Local reasoning models on Mac: GGUF vs MLX vs token-efficient builds

The smallest verified 27B-class local reasoning option in this source set for Apple Silicon Macs is a 5.9 GB GGUF file. In the same set, the smallest MLX safetensors build is 8.6 GB, while Swift-Qwen3.8-27B starts at 9.1 GB and its model card says the Swift adapter uses 58.3% fewer thinking tokens than the BF16 base with less than 1% loss.

What the verified model files say about Mac fit?

If you want a local reasoning model on a Mac today, the first useful filter is file size, not marketing copy. The verified floor in this source set is Ternary-Bonsai-2-27B-PTQ1_0.gguf at 5.9 GB, and the same Hugging Face page’s arithmetic says to plan roughly 6.8 GB of RAM or VRAM including a modest context cache.

That is not a Mac benchmark. It is still useful because it separates models that can plausibly load from models that plainly will not. The same Ternary-Bonsai GGUF page lists PQ2_0 at 7.2 GB with a 8.3 GB plan and F16 at 53.8 GB with a 61.9 GB plan. On the MLX side, prism-ml/Ternary-Bonsai-2-27B-mlx-2bit publishes 8.6 GB on disk and says the model has 27.4 billion parameters, with 54.7 GB at 16-bit, 27.4 GB at 8-bit and 13.7 GB at 4-bit for weights alone.

The wider public size ladder in this evidence base is GGUF. Unsloth’s Qwen3.8-27B-GGUF runs from UD-IQ1_S at 6.2 GB with a 7.1 GB plan to Q8_0 at 29.0 GB with a 33.4 GB plan. UkisAI’s Swift-Qwen3.8-27B-GGUF spans IQ2_XXS at 9.1 GB and 10.5 GB planned memory up to Q8_0 at 29.1 GB.

Those figures are enough to make one firm point. Apple Silicon Macs can run local 27B-class models, but the format and quantisation tier decide whether the project is routine or marginal.

Assumption used once for the arithmetic above: the published “planned memory” figures add a modest context cache to the file size on the same Hugging Face pages. They are not hardware tests.

Source: file listings and model cards at prism-ml/Ternary-Bonsai-2-27B-gguf, prism-ml/Ternary-Bonsai-2-27B-mlx-2bit, Swift-Qwen3.8-27B-GGUF and Qwen3.8-27B-GGUF.

Which runner supports what on Apple Silicon?

llama.cpp is the best-supported route in this source set for GGUF on Apple Silicon. The project says Apple Silicon is a first-class target, optimised through ARM NEON, Accelerate and Metal, and it supports 1.5-bit through 8-bit integer quantisation. It also documents direct model fetches with llama cli -hf and an OpenAI-compatible server with llama serve -hf.

For Apple-native MLX serving, vLLM Metal is the strongest primary source here. Its README says it enables vLLM to run on Apple Silicon Macs using MLX as the primary compute backend, requires macOS 15 (Sequoia) or later, on Apple Silicon, and now supports Qwen3.8 on Metal. It also says mlx-community/Qwen3.8-27B-8bit serves a 27B hybrid model on a single Apple Silicon Mac.

The source set is thinner for Ollama and LM Studio. A developer comparison on Dev.to says Ollama is a CLI plus REST API aimed at developers, while LM Studio is a desktop GUI for browsing, running and chatting with local models. The same piece says LM Studio can use MLX-optimised models on Apple Silicon. That is useful compatibility guidance, but it is not as strong as a project README or vendor documentation.

The Swift model card adds one concrete cross-runner point. It says its llama.cpp settings also work in LM Studio, koboldcpp and Jan AI. It also provides a ready-made llama-server command for Q4_K_M and says llama.cpp downloads the multimodal projector automatically with -hf.

What public speed data exists, and what does not?

There is public Apple Silicon performance data in the public record, but not the apples-to-apples comparison many readers will want. The strongest primary-source figure comes from vLLM Metal, which says version 0.2.0 made its unified paged varlen Metal kernel the default attention backend and delivered 83x TTFT and 3.6x throughput compared to v0.1.0. That is a project-internal before-and-after result, not a comparison with llama.cpp, Ollama or LM Studio.

The Swift model card makes another speed claim, but it is about model behaviour rather than Mac runtime. UkisAI says Swift uses 58.3% fewer thinking tokens and gets a 1.95x speed-up on several tasks against the Qwen3.8-27B BF16 base. The same card publishes benchmark-level token reductions, including 41.0% on GPQA-Diamond, 46.2% on MMLU-Pro and 24.3% on LiveCodeBench v6. Those are vendor benchmark claims, but they are specific and reproducible enough to matter.

External practitioner reports go a bit further on Mac speed. One Mac Studio operator writes on Substack that in 2026 MLX is consistently 10% to 25% faster than llama.cpp. A Dev.to comparison says MLX often outperforms llama.cpp on Apple hardware. Those are plausible signals. They are still weaker than matched benchmarks from the projects themselves, so they should guide experiments, not procurement.

Recommendations by Mac memory tier

The practical decision for most readers is not “GGUF or MLX” in the abstract. It is which file to try first on the Mac you already own. The table below sticks to verified file sizes and to runner compatibility that appears in this source set.

Source: file listings and model cards at prism-ml/Ternary-Bonsai-2-27B-gguf, prism-ml/Ternary-Bonsai-2-27B-mlx-2bit, Swift-Qwen3.8-27B-GGUF and Qwen3.8-27B-GGUF.

Why Swift matters more for runtime than for download size?

Swift is not the smallest model family in this set. Its case is different. UkisAI says Swift-Qwen3.8-27B uses 58.3% fewer thinking tokens while keeping performance within <1% loss against the BF16 base. The same card says that produces a 1.95x speed-up on several tasks.

For Mac users, that matters because long reasoning runs are often bottlenecked by token generation, not just weight loading. The Swift card’s benchmark table makes that concrete. It reports median-token reductions of 58.3% on GPQA-Diamond, 28.3% on MMLU-Pro, 50.5% on IFBench and 45.8% on LiveCodeBench v6.

The same page also gives unusually specific context-cache maths. It says Swift’s hybrid architecture keeps KV cache to 64 KiB per token, which works out to 0.5 GB at 8k, 2.0 GB at 32k, 4.0 GB at 64k and 8.0 GB at 128k. That is why its card warns that full context will not fit on smaller machines unless you lower -c.

What we could not verify?

We could verify published file sizes, some model-card benchmarks, and project-level Apple Silicon support statements. We could not verify the earlier draft’s claim that 24 GB is the practical floor for daily local reasoning across this source set. The strongest support for that idea is narrower: UkisAI recommends Q4_K_M for 24 GB cards, everyday use, and the model files below that size plainly exist. That is enough to recommend 24 GB for some 27B-class setups, not enough to declare it the universal floor.

We also could not verify matched tokens-per-second benchmarks across llama.cpp, Ollama, LM Studio, MLX and vLLM Metal on the same Apple Silicon hardware. the public record has project claims and practitioner reports, but not one reproducible cross-runner benchmark suite. We could not verify public Mac benchmarks for the Ternary-Bonsai builds either. Finally, we could not verify wider compatibility matrices for Ollama or LM Studio from primary sources in the public record; the strongest runner-specific support there comes from the Swift model card’s note that its settings also work in LM Studio and from the external Dev.to comparison.

What to try first on a Mac, based on the evidence here?

If you own a 16 GB Mac and want to test local reasoning, start with the smallest verified GGUFs: Ternary-Bonsai PTQ1_0 at 5.9 GB, Qwen3.8-27B UD-IQ1_S at 6.2 GB or Swift IQ2_XXS at 9.1 GB. Use llama.cpp if you want the clearest primary-source path on Apple Silicon.

If you own a 24 GB or 32 GB Mac, Swift Q4_K_M at 18.0 GB is the most clearly documented everyday pick in the public record, because the vendor pairs a memory-tier recommendation with token-efficiency and quant-quality tables. If you want a simpler baseline, Qwen3.8-27B Q4_0 at 16.1 GB or Q4_1 at 17.5 GB are the obvious GGUF alternatives.

If you want the Apple-native MLX route, the clearest documented serving path in this source set is vLLM Metal, which requires macOS 15 or later on Apple Silicon. The next open question is not whether Qwen3.8-class models run on Macs. It is which public benchmark suite will publish matched speed and quality results across GGUF, MLX, Ollama, LM Studio, llama.cpp and vLLM Metal on the same machines.

Sources

• https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit
• https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf
• https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF
• https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
• https://github.com/vllm-project/vllm-metal
• https://huggingface.co/docs/huggingface_hub/en/package_reference/cli
• https://docs.astral.sh/uv/
• https://github.com/ggml-org/llama.cpp
• https://spicyneuron.substack.com/p/a-mac-studio-for-local-ai-6-months
• https://dev.to/bspann/running-llms-locally-on-macos-the-complete-2026-comparison-48fc