Local Qwen on Apple Silicon: a short field note

by Davide Pasca

Qwen 3.5 on Apple Silicon (MLX)

I put together this repo to make running Qwen locally on Apple Silicon (via MLX) repeatable. This setup was developed with GPT-5.3 Codex via Codex CLI, mainly to keep the scripts and experiments easy to reproduce.

For context, these runs were done on a MacBook Pro (Apple M2, 32GB RAM).

Quickstart

git clone https://github.com/dpasca/local-llm-lab.git
cd local-llm-lab
./scripts/setup_mlx_env.sh
./scripts/run_model.sh "Say hello in five words."

If you want to use this setup from a coding assistant UI, you can run the included local OpenAI-compatible server and point OpenCode at it (the repo includes an opencode.json for convenience):

# from the local-llm-lab repo
./scripts/mlx_server.sh start
opencode

Local models will matter more and more

Local models are getting strong enough to run practical agent-style loops, including tool calling. Not having to depend on external services (with API costs and rate limits) opens the door to unlimited automation and experimentation, even if the harder tasks still need large cloud-hosted models.

Physical constraints mean local models are not going to match cloud-hosted models in the foreseeable future. But they will likely reach the "good enough" threshold for many everyday tasks soon, and can act as a broker: handle easier work locally, then offload tougher problems to larger paywalled models.

35B-A3B or 27B?

The repo defaults to mlx-community/Qwen3.5-35B-A3B-4bit, but I also tested mlx-community/Qwen3.5-27B-4bit.

Qwen3.5-35B-A3B is larger overall (35B total parameters), but it is a sparse Mixture-of-Experts checkpoint: only a small subset of experts is active for each token at inference time (and the active experts can vary token-to-token). The model card summarizes this as ~3B activated parameters.

Qwen3.5-27B is a dense checkpoint, so all parameters participate for each token. In practice, the surprise in my runs was the speed (especially after warmup). For quality notes, see the report below.

I compared the two on the same long prompt (~5,975 tokens), running each twice with temperature 0.

On this M2 MacBook Pro (32GB RAM):

  • 27B run times: 120.489s, 126.996s
  • 35B run times: 25.486s, 19.751s

So for now, 35B stays the default profile in the repo.

Full benchmark note: Qwen3.5 35B-A3B vs 27B report

Notes

RAM headroom

RAM headroom matters. If the system starts swapping or compressing memory, throughput can get noisy, so I try to keep the machine lean while testing. I generally prefer simpler TUI-based tools (Codex CLI, tmux, NeoVim) over heavier Electron apps like VS Code, which can easily eat gigabytes of RAM.

Qwen "thinking" templates

One detail that mattered a lot here is:

CHAT_TEMPLATE_ARGS='{"enable_thinking":false}'

This avoids hidden reasoning output consuming the completion budget in this workflow, and helps keep responses directly usable for content output.

Reference: in Qwen3.5, this corresponds to chat_template_kwargs.enable_thinking=false (see the "Instruct (or Non-Thinking) Mode" section in the official model card).