Anubis OSS v3.9
Benchmark local LLMs. See the watts.
Anubis runs a model on your Mac, measures it properly, and correlates every token with what the silicon actually did: GPU, CPU, Neural Engine and DRAM power sampled live through IOReport while the model generates.
Free and open source under GPL-3.0. No account, no telemetry, nothing leaves your machine unless you choose to publish a run to the community leaderboard.
$ brew install --cask uncsoft/anubis/anubis-oss
macOS 15 Sequoia or later · Apple Silicon only · signed, notarized, and self-updating via Sparkle
The numbers were audited against the tools they claim to match
The whole measurement pipeline was checked against how Ollama, llama.cpp and the standard harnesses (llmperf, genai-perf) define their metrics, then cross-checked against every public leaderboard submission.
Counts every generated token
Thinking tokens included, matching Ollama's eval rate and llama.cpp's predicted_per_second. The old visible-output-only formula could explode on reasoning-heavy runs, which is where the leaderboard's impossible 5,000–29,000 tok/s rows came from. They're filtered server-side now too.
Excludes model load time
Anchored at request dispatch. A cold start reports the load separately (Prefill: 848 tok/s · cold +7.2s) instead of silently inflating time-to-first-token by seconds.
Timing from the backend where it exists
oMLX usage extensions, llama.cpp's timings block, and LM Studio's native chat API, giving decoded tok/s, TTFT, load time and exact token counts. A green seal on the Tokens/sec card marks a server-verified number.
Latency can't disagree with throughput
Average token latency is exactly 1000 ÷ tok/s. Alongside it: a model prep picker (as-is / warm-up / cold start), because cold-vs-warm was the largest uncontrolled variable in TTFT comparisons.
What's in it
Everything needed to benchmark, compare and understand local inference on a Mac. Every run is stored locally, exportable, and published only if you ask.
Benchmark dashboard
Streaming output with live tok/s, TTFT, prefill speed, joules per token and a per-model trend against your last run of the same config, so the dashboard doubles as a regression detector.
Flow Builder
A Shortcuts-style drag-and-drop sequencer. Set Model, Repeat ×5, Run, Unload, played back hands-off. Six templates ship, including Cold vs Warm and Q4 vs Q8 sweeps. Save as .anubisflow to share.
N-rep groups with confidence intervals
Run the same config 1–20 times and get mean ± 95% bootstrap CI (1,000 resamples) for tok/s, TTFT, J/tok, power and peak memory. Pick random or fixed seeds to separate hardware variance from sampler variance.
Power telemetry
GPU, CPU, ANE and DRAM watts straight from IOReport's energy model, plus GPU frequency from P-state residency. Per-channel unit scaling was corrected in 3.5; earlier builds undercounted system power.
Arena
Run two models side by side on the same prompt and watch both streams and both power traces resolve in real time.
System monitor & floating HUD
GPU core-level activity, thermal pressure and memory pressure, with a detachable always-on-top HUD so you can watch the machine while you work in another app.
Vault & Ollama browser
Inspect installed models, unload them to reclaim memory, and search, browse and pull new ones from ollama.com without leaving the app.
Process monitoring
Finds the process actually serving your model by port, identifies the backend, and measures phys_footprint, the same figure Activity Monitor shows, including Metal buffers.
Reports & export
Share-ready 1920×1080 or square report cards at 2× retina, per-model CSV, and full run history with filtering, multi-select and bulk delete.
Why it exists
The local LLM ecosystem on macOS is fragmented. Chat wrappers focus on conversation, monitors are CLI-only, and almost nothing connects inference to the hardware underneath it.
Throughput without power is half a number
120 tok/s at 8 W and 130 tok/s at 34 W are not the same result. Anubis reports joules per token so you can compare quantizations and backends on efficiency, not just speed.
One run proves nothing
Thermal state, scheduler decisions, sampler seed and background load all bleed into a single measurement. N-rep groups report the spread, so you can tell a real 5% gap from noise.
Your data stays yours
Everything is local by default: SQLite on your disk, no account, no analytics. Publishing a run to the leaderboard is an explicit, per-run action.
And the dataset is open
Every published run is queryable in the browser, exportable as CSV or JSON, and free to reuse. No sign-up wall in front of the numbers.
Screenshots
macOS 15+, native SwiftUI, light and dark.
Supported backends
Anything that speaks the OpenAI chat API works. Ollama is auto-detected on launch; the rest are two fields in Settings.
| Backend | Type | Port | Setup |
|---|---|---|---|
| Ollama | Native | 11434 | Install from ollama.com, auto-detected on launch |
| Apple Intelligence | On-device Foundation Models | n/a | macOS 26+ with Apple Intelligence on; appears automatically |
| LM Studio | OpenAI-compatible | 1234 | Enable the local server in LM Studio settings |
| mlx-lm | OpenAI-compatible | 8080 | pip install mlx-lm && mlx_lm.server |
| vLLM | OpenAI-compatible | 8000 | Add in Settings |
| LocalAI | OpenAI-compatible | 8080 | Add in Settings |
| Docker Model Runner | OpenAI-compatible | your choice | Add in Settings |
| llama.cpp | OpenAI-compatible | 8080 | Server-reported timings are read when available |
Only Ollama exposes the full set of datapoints. Some backends don't report prompt evaluation time, so a few columns stay empty for them. The dataset is robust either way.
Hardware metrics
Nine channels sampled in real time via IOReport and system APIs. If IOReport access is unavailable (in a VM, for instance) Anubis degrades gracefully to inference-derived metrics rather than failing.
| Metric | Source | What it is |
|---|---|---|
| GPU utilization | IOReport | GPU active residency percentage |
| CPU utilization | host_processor_info | Usage across all cores |
| GPU power | IOReport Energy | GPU consumption in watts |
| CPU power | IOReport Energy | P-cores + E-cores in watts |
| ANE power | IOReport Energy | Neural Engine consumption |
| DRAM power | IOReport Energy | Memory subsystem consumption |
| GPU frequency | IOReport GPU Stats | Weighted average from P-state residency |
| Process memory | proc_pid_rusage | phys_footprint, including Metal/GPU allocations |
| Thermal state | ProcessInfo | System thermal pressure level |
Get started
Four steps, about five minutes if you already have a backend.
Install a backend
Ollama is the easiest and reports the most complete metrics.
# install and start Ollama $ brew install ollama $ ollama serve # pull something to benchmark $ ollama pull qwen3:8b
Install Anubis
Homebrew installs the signed, notarized build from the latest GitHub release. Or download the zip and drag it to /Applications.
$ brew install --cask uncsoft/anubis/anubis-oss
Run your first benchmark
Anubis auto-detects Ollama on launch. Pick a model, choose a preset prompt or write your own, set repetitions if you want confidence intervals, and hit Run.
Publish it, if you like
One button in the toolbar adds your run to the public leaderboard. Entirely optional, and off by default. Nothing is uploaded unless you ask.
Anubis updates itself via Sparkle: open the app and accept the prompt, or
Settings → Check for Updates. Through Homebrew instead:
brew upgrade --cask anubis-oss --greedy
(--greedy is required because the cask is marked
auto_updates).
Requirements
The results are public and so is the data
Every published run is queryable in the browser and exportable in full: 1,740 runs across 59 fields of real Apple Silicon inference telemetry. No sign-up, no API key, no wall in front of the numbers.