The "Linux kernel for AI agents": OAK runs your agent 100% on-device, responds in 87ms
What does this project actually do
OAK (Open Agent Kernel) sets out to be the "Linux kernel for AI agents": it lets you build agents that run 100% on-device, with no cloud dependency, no data leakage, and no network latency. Where most agents ship your request to a remote model and wait, OAK crams the model, the tools and the memory all onto the local device — a car head unit, a phone, or an IoT gadget. One spoken command, "turn on the AC, set it to 22°C, interior mode," finishes in 87 milliseconds, and most of the time it never even wakes the large language model.
Why it is gaining attention now
On-device AI is the year's big theme, but most "local agents" are either slow or just hide a cloud call behind a friendlier name. OAK's pitch is harder-edged: 80% of requests take the deterministic path and resolve in 0.02 milliseconds without invoking a model at all; only the remaining 20% go to local LLM inference. Add 14× speedup on a Qualcomm NPU at one-third the power, plus the privacy promise of "data never leaves the device, no telemetry, no cloud calls," and it hands developers who want private, low-latency, offline-capable agents a surprisingly complete answer.
Technical highlights
The architecture has a clean Route Decision split: the deterministic side is handled by a Skill Engine (YAML-defined pattern matching, 0.02ms, no model needed), while the inference side goes to the LLM (87ms on NPU, 1200ms on CPU). On top of that it ships some genuinely hardcore capabilities. Constrained Decoding uses GBNF grammars to force valid JSON, which means zero hallucinated tool calls. A DAG Orchestrator executes multi-step plans with dependency resolution. Formal Plan Verification runs CTL* model checking to block unsafe plans before they execute. On-device continual learning uses DP-SGD with Rényi differential privacy (ε = 2.1, budget 8.0) for private fine-tuning. Crash safety rests on a write-ahead log with three terminal states — COMMITTED, FAILED, UNKNOWN — including an explicit UNKNOWN reconciliation that cloud frameworks simply do not offer.
The performance numbers are eye-catching too. Against LangChain (2–5s) and AutoGPT (3–10s), OAK's typical latency is 87ms. Speculative execution lifts cache-hit rate to 73.2% for a 3.71× latency speedup. Formal verification catches 71.4% of unsafe plans with zero false positives. Constrained decoding pushes valid output rate to 100% versus 16.7% unconstrained.
Who it is for
It targets embedded and on-device developers, teams that want agents on car head units, smart-home hubs or IoT edges, and privacy-sensitive users who refuse to send their data to the cloud. It fits best the scenario where "it must work offline and latency has to stay under a hundred milliseconds" is non-negotiable. It also appeals to hobbyists who want a private voice assistant that never phones home — no API key, no monthly bill, no transcript sitting on someone else's server.
Quick start
Install the CLI globally with npm install -g @sparx/cli, then sparx init my-agent && cd my-agent, pull a small model with sparx pull qwen2.5-0.5b-instruct (about 530MB), and sparx run to launch it — and notably it answers with deterministic skills even without loading a model. For a demo, just run sparx demo automotive.
How it compares to alternatives
The biggest difference from LangChain and AutoGPT is that those essentially orchestrate cloud API calls, whereas OAK runs the entire agent — model, tools and memory — on the device. Against Apple Intelligence, OAK is fully open source, can ride the NPU, and brings crash recovery and formal verification that Apple's closed stack does not expose. Its positioning is also disciplined: it does not try to be a complete operating system, only the kernel layer, so car makers, phone vendors and robot companies can build their own agent OS on top. In plain terms, it is betting that agents will eventually sink down to the device's lowest layer.
Where OAK fits in the broader picture is interesting: most agent tooling this year is racing up into the cloud and into bigger models, while OAK deliberately races down into the silicon. That makes it a natural fit for automotive and robotics, where a dropped network connection is not an inconvenience but a safety event. If your agent must keep working in a tunnel, OAK's local-first design is not a nice-to-have, it is the whole point.
One honest caveat: OAK is young (v2.1.x), and its headline numbers come from the project's own benchmarks on specific Qualcomm silicon; the Snapdragon 8 Gen 3+ path is still marked "TBD" for Q4 2026, and you do not need Qualcomm hardware to develop since CPU inference via llama.cpp works everywhere. Treat the 87ms figure as a best case on NPU, not a guarantee on every laptop. Even so, the design philosophy — deterministic-first, crash-safe, privacy-by-default — is what makes it worth watching.