Engineering notes
Qwen3.6-35B-A3B hits 63.8 tokens/s on a 32 GB Mac mini
Native MLX with MTP decodes Qwen3.6-35B-A3B at up to 63.8 tokens/s on a base 32 GB M4 Mac mini, up to 2.5x what llama.cpp managed on the same box.
Qwen3.8 hits 81.4 tokens/s on an RTX 3090
Local Qwen3.8-27B peaks at 81.4 tokens/s with a few perf adjustments.
Muse Glimmer is a flop
Muse Glimmer is a flop: chatty, prone to hallucinations, bad at tool calling, slower than Qwen and only half of Qwen 3.6's context.
Muse Glimmer 30B with DFlash hits 133.7 tokens/s on an RTX 3090
Muse Glimmer 30B fits an RTX 3090 with its vision projector and DFlash drafter, DFlash averaged a 2.29x decode speedup, and the context stretched to 98,304 tokens for 442 MiB of extra KV cache.
Subscribe
New posts by email, roughly twice a month.
✓ Subscribed.
RSS