MLX or GGUF on Mac: Which Format to Run a Local Model In
How the MLX format differs from GGUF on Apple Silicon, why mlx-lm won't open a .gguf, what to pick for your MacBook, and how unified memory affects model size.
On Apple Silicon, a local model has two paths. You can take the familiar .gguf — the
same file that works on Windows and Linux. Or you can take a version in the MLX
format, built specifically for Apple's chips.
The usual question is which one is faster. "What do you give up by picking one of them" is more practical, and the answer is less obvious.
What MLX is
MLX is Apple's framework for computation on its own chips. On top of it sits the
mlx-lm package, which developers describe plainly: text generation and fine-tuning
of language models on Apple Silicon.
The key phrase is Apple Silicon. MLX exists only for Macs on M-series chips. On Intel Macs, Windows, and Linux, it doesn't exist and won't — it's tied to an architecture where the CPU, GPU, and memory live on one chip.
Models come from Hugging Face, mostly from the mlx-community organization — that's
where converted and quantized versions of popular models live. mlx-lm can also
quantize on its own, including the familiar 4-bit option.
What GGUF is
GGUF is the format of the llama.cpp engine, which almost the entire local AI
ecosystem sits on: Ollama, LM Studio, desktop apps. One file per model, quantization
tags like Q4_K_M, and it runs on anything — CPU, NVIDIA, AMD, Apple.
There's a detailed breakdown of the format and which quantization to pick separately.
The main difference in practice
The formats aren't interchangeable. mlx-lm won't open .gguf files. These are
different ways of storing weights, and converting between them is a separate
operation, not a settings toggle. If a model only exists in GGUF, you can't run it
through MLX, and vice versa.
The first practical consequence follows from that: picking a format is picking which catalog of models you have access to. A GGUF version shows up for almost everything notable, almost immediately. MLX versions get published by the community, and for less common models one might not exist at all, or might show up later.
On speed
MLX is built for specific hardware and works directly with Apple's unified memory, so on the same model it often ends up faster than GGUF on the same Mac. But "often" isn't "always" — the result depends on the model, quantization, context length, and whatever else is using memory.
Better to not trust anyone else's numbers and measure your own. Take one model in both formats, the same prompt, and compare tokens per second. LM Studio on Apple Silicon supports both engines, so the comparison happens inside one app with no extra setup.
The noticeable difference more often shows up not between formats, but between "the model fit in memory" and "it didn't." Once swapping starts, whatever advantages an engine had stop mattering.
How much memory to budget
On Apple Silicon, memory is shared: the system, apps, and the model all split the same pool of gigabytes. There's no separate VRAM, and that's both an advantage and a limitation — a lot is available to the model, but it's competing with everything else.
The practical rule is the same as everywhere else: look at the model file's size and
leave headroom for context and the system. For large models, mlx-lm's
documentation has a specific caveat — you may need to raise the system's "wired"
memory limit, and some features require macOS 15.0 or newer.
If a model doesn't fit in memory, drop to a lighter quantization instead of looking for a faster engine. General guidance on sizes is in the article on hardware for local models. If you'd rather not work it out by hand, there's also a model picker calculator.
You don't always have to pick this by hand: Doka on Mac checks the available memory itself and suggests something that fits — on Apple Silicon, with its shared memory pool, that removes the most common mistake, where a model technically downloaded but is unusable.
What to pick
| Situation | Format |
|---|---|
| Need a specific new model, no MLX version yet | GGUF |
| Working only on Mac and squeezing out speed | MLX |
| Want one file across all your machines | GGUF |
| Fine-tuning a model on your own data right on the Mac | MLX |
| Just need it to work | GGUF |
That last row isn't a joke. GGUF is the default format for local AI: every tool understands it, there's documentation and ready answers for almost any error. MLX gives a speed win on its own hardware, but it's a choice for someone willing to convert models by hand sometimes and wait for the community to publish the version they need.
Where format stops being the main question
Choosing between MLX and GGUF matters as long as the task is running a model. Once it becomes about actual work, the question shifts: what matters more isn't the weight format, but what the tool around the model can do.
Doka runs on Mac and uses GGUF, precisely because it's the format that always has the model you need. It downloads inside the app, and from there the actual point begins: the agent reads and edits files, works in the terminal, connects MCP servers, and carries a task through to a result.
A useful exercise: time how much of your typical task is generation, and how much is copying results between the chat and your files. For most people, it's the second one. What usually gets optimized is the first. Download it and watch those proportions shift once nothing needs to be carried over by hand.