OMLX Hits 47 Tokens a Second on a Mac, LM Studio Only 16

Local AI on a Mac just crossed a real threshold: a new runtime called OMLX is hitting 47 tokens per second on Apple Silicon, compared to roughly 16 tokens per second on LM Studio, by rethinking how a Mac's unified memory handles a model's context instead of just running a smaller model faster. Paired with a wave of native apps and a new generation of small, dense models, running frontier-class AI entirely offline on a Mac has gone from a compromise to a genuinely competitive setup for solo builders and small teams.

Two weeks before this episode, SpaceX confirmed its roughly $60 billion acquisition of Cursor, one more data point in a broader pattern of cloud AI valuations climbing into the stratosphere. At the same time, a much quieter shift has been building on individual desks: solo developers and small businesses increasingly want their code and data to never leave the machine they're sitting at, and they're building the tooling to make that practical instead of just aspirational.

That tension, consolidation at the top of the market against decentralization at the edge, is the backdrop for why local AI tooling has gotten so much sharper in the last few months. The clearest evidence of the pull toward local is Nativ, a Swift, MLX-based Mac app built by Prince Canuma that hit Hacker News's front page with 377 points and 131 comments and has since crossed 1,200 stars and 35 open issues on GitHub. Canuma has described the pitch directly: "OpenWebUI is a server product wearing a chat UI. Nativ is what local AI on Mac should feel like: a true native app, built on mlx-vlm for Apple Silicon. Download, open, chat." The app has since spread well past its initial developer audience, showing up in Thai-language TikTok explainers pitching it as a way to skip a $20-a-month ChatGPT Plus subscription and in Instagram reels racking up nearly 30,000 views.

The hardware reason local AI works on a Mac at all comes down to unified memory and zero-copy arrays. On a traditional PC, the CPU and GPU sit on physically separate memory pools, so running an AI model means constantly shuttling massive amounts of data back and forth over a comparatively slow bridge between them. Apple Silicon puts the CPU and GPU on the exact same physical memory pool instead, which removes that shuttling bottleneck entirely.

Think of a traditional PC like a restaurant where the chef works in the kitchen but the pantry is down the hall in another room: every ingredient costs a trip. Apple Silicon is closer to an open-plan kitchen where everything the chef needs is within arm's reach. The catch is that the open-plan kitchen has limited counter space. You get lightning-fast access to whatever's loaded, but a real ceiling on how much you can hold in memory at once, and AI models are notoriously memory-hungry. That ceiling is exactly the problem the next generation of local runtimes had to solve.

Two tools are doing most of the work of closing that gap. Nativ runs multimodal models locally while exposing OpenAI-compatible endpoints, so existing cloud scripts and automations can point straight at a local machine with no rewrite required. But the deeper architectural breakthrough is OMLX, a runtime built directly on Apple's MLX framework that uses a two-tier memory system: it keeps immediate, actively-used context in RAM while freezing colder data, like massive system prompts and tool definitions, directly to SSD in SafeTensors format instead of holding everything in memory at once.

The performance difference is the headline number: reviewers are clocking OMLX at roughly 47 tokens per second against LM Studio's 16 on comparable hardware, a gap large enough to change what's actually usable for real work rather than demos.

The obvious objection to constantly reading and writing system prompts to an SSD is that it should be slow, and that once you max out memory on a consumer machine, you'd expect a crash. In practice, hitting a memory ceiling on a standard Mac still produces a context-window overflow error, but OMLX's SSD cache changes what happens next. Because it freezes the exact context state to disk rather than discarding it, the runtime doesn't truncate the prompt and start guessing, which is usually what produces hallucinated output. Instead it offloads the cold data and picks the session back up with the full reasoning context intact, functionally recovering from a context-limit hit instead of just failing past it.

The other half of making this practical is model choice. A standard Mac Mini will choke if you try to run the same massive models the frontier labs run in the cloud, so local setups lean on quantized models: dropping a model's mathematical precision, commonly from 16-bit down to 4-bit, cuts memory requirements dramatically while preserving most of the model's reasoning capability.

Based on recent benchmarking discussed in the episode, Google's Gemma 3 in its 4-billion-parameter configuration is currently the strongest option for this setup, consistently beating larger local models like Llama 3.1 and Qwen 2.5 on coding and reasoning puzzles, largely because Gemma 3's architecture is unusually dense and tuned for logic relative to its size. The broader local ecosystem backs this up: Better Stack's "Local AI On Apple Silicon Uses 7X Less RAM" video, covering a 26-billion-parameter model running usably on 2GB of RAM at 23 tokens per second, has pulled in nearly 150,000 views and over 4,300 likes, and TikTok accounts like github.signals have covered the same class of memory-efficient local models to audiences in the tens of thousands.

The mental model worth keeping: running local AI isn't about cramming a Formula One car onto a go-kart track. It's choosing the tuned-down street version that actually fits in your garage and still gets you to work, reliably, for free, and without your code or data ever leaving your machine.

If you're a solo creator or small business owner deciding whether to bother with any of this yet, the practical takeaway is that local AI on a Mac has crossed from "interesting experiment" to "genuinely usable tool for daily tech debt," particularly for coding and reasoning tasks that don't need frontier-model ceiling performance. The setup that works right now: an Apple Silicon Mac with enough unified memory, a runtime like OMLX or Nativ that manages context intelligently instead of just hoping it fits, and a quantized model in the Gemma 3 4B class rather than the biggest thing you can find. None of it requires a subscription, an API key, or your data leaving the building.

What is OMLX and how is it different from LM Studio?
OMLX is a local AI runtime built on Apple's MLX framework that uses a two-tier memory system, keeping active context in RAM while freezing older or larger context (like system prompts) to SSD instead of holding it all in memory. That architecture is why it's currently benchmarking at roughly 47 tokens per second versus LM Studio's 16 on comparable Apple Silicon hardware.

What is Nativ and who built it?
Nativ is a native macOS app, built in Swift on top of mlx-vlm, that lets you chat with, serve, and monitor local MLX models from one app while exposing OpenAI-compatible endpoints. It's built by Prince Canuma and has crossed 1,200 GitHub stars since launching on Hacker News, where it hit the front page with 377 points.

Why is a 4-billion-parameter model beating larger local models?
Google's Gemma 3 4B is winning recent local benchmarks against larger models like Llama 3.1 and Qwen 2.5 because its architecture is unusually dense and optimized for logic relative to its size, which matters more for local coding and reasoning tasks than raw parameter count once a model is quantized down to fit consumer memory.

Do I need expensive hardware to run local AI models well?
No. The whole point of this shift is that a consumer Apple Silicon Mac with unified memory, paired with a quantized model and a context-aware runtime, now handles real coding and reasoning work locally, no dedicated GPU rig or ongoing cloud subscription required.

Full episode: Watch on YouTube

Want the daily breakdown of what's actually working in AI agents, coding tools, and local AI? Head to joebuildsai.com for more.