Codex 校核（2026-10-09） 本页是 7月14日发布版本与社区评论的采集快照。PrismML 原始发布页所列 5.9GB/3.9GB 是量化权重大小；运行还需要 KV cache、激活和框架内存，262k 是模型上下文规格，不能据此保证手机或任意 16GB 笔记本都能运行完整上下文。发布页的 iPhone 演示使用缓存/预填充图片上下文。其 15 项 thinking-mode 评测是厂商自报；80→74/66 的工具分数分别下降 6/14 个百分点，不能把整体约90–95%的均值保留率当作所有任务下降不超过5%。RTX 5090、M5 Max 和手机演示的硬件/内核条件各异，权重免费许可也不证明免费 API 预览在当前账户持续可用。
Bonsai 27B — A 27B-Class Model That Runs on a Phone
Summary
PrismML announced Bonsai 27B on July 14, 2026 — the first 27B-class model capable of running on a phone. Based on Qwen3.6 27B, available in two extreme-quantization variants under Apache 2.0.
- Ternary Bonsai 27B — 1.71 effective bits/weight (5.9GB), runs on any laptop
- 1-bit Bonsai 27B — 1.125 effective bits/weight (3.9GB), fits iPhone 17 Pro memory
Both are multimodal (vision + text), support 262K-token context, speculative decoding.
Key Benchmarks (15-benchmark suite)
| Category | Qwen3.6 27B | Ternary 27B | 1-bit 27B |
|---|---|---|---|
| Math | 95.3 | 93.4 | 91.7 |
| Coding | 88.7 | 86.0 | 81.9 |
| Agentic/Tool-calling | 80.0 | 74.0 | 66.0 |
| Instruction Following | 78.4 | 71.8 | 65.8 |
| Knowledge/STEM | 83.1 | 77.0 | 73.4 |
| Vision | 72.6 | 65.2 | 59.6 |
| Overall | 85.0 | 80.5 | 76.1 |
What Makes It Important
- 90-95% retention at 7-10% of original size (54GB → 3.9GB). The 1-bit variant is the first true 27B-on-phone.
- Agentic performance is the biggest drop (80→66 for 1-bit). HN commenters flagged tool-calling degradation as the real-world pain point.
- No higher-precision escape hatches — the low-bit representation runs end-to-end through everything (embeddings, attention, MLPs, LM head).
- HuggingFace models are live but GGUF/MLX support in LM Studio is incomplete (per simonw’s testing).
- Apple in talks with PrismML (CNBC report) — strategic acquisition interest.
HN Community Sentiment
- Bullish side: “Paradigm shift” — on-device capability kills “Privacy GPT” startups. Works on M1 Pro 16GB. KV-cache efficiency praised for multi-agent coding.
- Skeptical side: Blog post may be AI-generated. 5% tool-call drop matters in real use. CPU inference on Ryzen 5700X: binary = 6 t/s, ternary = 0.7 t/s (too slow).
- Technical takeaway from hham: This is “accelerant #3 and #4 converging” — local models finally good enough to matter + hardware layer ready.
- Interesting comparison: Ornith 9B on Ollama claims to match Gemma 4-31B and Qwen 3.6 35B.
Relevance to Hermes Stack
- Local inference breakthrough: A 27B-class model running on WSL laptop means Hermes could run much smarter local agents without GPU. Ternary variant at 5.9GB fits well within 16GB RAM laptops.
- Coding agent impact: 86.0 coding score (ternary) is competitive with many larger models. However, 74.0 agentic/tool-calling is a significant gap from 80.0 baseline.
- KV-cache efficiency praised for multi-agent workflows — directly relevant to Hermes session management across multiple profiles.
- Skill system integration: If 1-bit models reach 66.0 on BFCL/TauBench, they’re usable for tool orchestration but may hallucinate tool schemas.
Key Links
- Article: https://prismml.com/news/bonsai-27b
- HN Discussion: https://news.ycombinator.com/item?id=48910545
- HuggingFace: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf
- Apple Talks: https://www.cnbc.com/2026/07/14/apple-prismml-ai-compression-iphone.html
- Community Benchmark: https://github.com/ArmanJR/PrismML-Bonsai-vs-Qwen3.5-Benchmark