# Magnitude > Updated 2026-10-01 · type: tool · category: ai-infrastructure · status: active · rev 1 Magnitude self-optimizes an LLM inference engine to your exact hardware, compiling kernels on-device so open models run up to 2x faster than llama.cpp. - Open source: yes (Apache-2.0) - Self-hostable: yes - Pricing model: free - Best for: A team running open models locally or on its own infrastructure behind agents, that wants more tokens-per-second out of the hardware it already has — Apple Silicon, NVIDIA, AMD, or plain CPU — without changing models. - Not for: A team that only calls hosted model APIs and never runs inference itself, or one that needs a turnkey managed endpoint rather than an engine it compiles and tunes on its own machines. - Last verified: 2026-10-01 - **Canonical:** https://gtmstacker.com/registry/tool/magnitude/ - **Source:** [magnitudedev · GitHub](https://github.com/magnitudedev/magnitude) - **Tags:** ai-infrastructure, mcp-agents, inference, local-models, self-hostable, performance - **Repository:** https://github.com/magnitudedev/magnitude ## Is Magnitude open source? Yes, Magnitude is open source under the Apache-2.0 license. ## How much does Magnitude cost? Magnitude is free to use. ## Can I self-host Magnitude? Yes, Magnitude can be self-hosted (the source is available under the Apache-2.0 license). ## Alternatives & related - [AgentRun](https://gtmstacker.com/registry/tool/agentrun/) --- Magnitude is an open-source inference engine for agents that optimizes itself for your exact hardware: it compiles and tunes its own kernels on your device so open models run faster on the machine you already have. Open source: yes (Apache-2.0); self-hostable; free OSS. It has ~6k stars and is very active (~1,000+ commits on main), and the maker reports open models running up to 2x faster than llama.cpp across Apple Silicon, NVIDIA, AMD, or CPU. ## What it does Most local-inference stacks ship generic kernels and hope they suit your chip. Magnitude instead treats the hardware as part of the problem: it compiles and tunes its kernels on the device where it runs, so the same open model is served with code shaped to that specific CPU or GPU. The maker's benchmark puts the result at up to 2x the throughput of llama.cpp, with coverage spanning Apple Silicon, NVIDIA, AMD, and plain CPU. For an agent that makes many model calls, more tokens-per-second on owned hardware is the difference between a snappy loop and a slow one — and because it is self-hosted and Apache-2.0, the engine, the models, and the tuning all stay on infrastructure you control. Open source: yes (Apache-2.0); self-hostable; free OSS. ## Provenance - Apache-2.0 per repo (license read from GitHub metadata, not guessed); ~6,020 stars; ~1,061 commits on main; self-hostable (github.com/magnitudedev/magnitude, verified 2026-10-01). - Self-optimizing inference engine: compiles and tunes its kernels on the user's device; maker reports up to 2x llama.cpp throughput on Apple Silicon / NVIDIA / AMD / CPU. - Surfaced in the GTM Stacker studio pull (2026-10-01 pass); license/facts independently verified 2026-10-01. - The speedup figure and hardware-coverage claims are the project's own benchmarks (vendor-claim); not independently reproduced here. ## Why it matters for a GTM stack Agents are only as cheap and fast as the inference underneath them, and the moment you run open models yourself the per-call economics are set by how well the engine uses your hardware. Magnitude's bet is that device-specific kernel compilation beats generic binaries — and if the 2x-over-llama.cpp claim holds on your box, that is real money and latency back on every agent step, with nothing leaving your infrastructure. For a GTM team self-hosting models behind prospecting, enrichment, or support agents, it is worth a head-to-head. The honest read: the speedup is a vendor benchmark and will vary with your model, quantization, and chip, and on-device tuning adds a first-run cost — so measure it against your current stack before you commit, rather than taking the 2x at face value.