Schema Harness Achieves ~99% on ARC-AGI-3 Public

HN Discussion: 124 points, 83 comments Link: https://schema-harness.github.io/

Core Finding

Frontier models (Opus 4.8 Max, Fable 5) using a specialized Schema Harness achieve ~99% on the ARC-AGI-3 Public benchmark — an abstract visual reasoning test that has been notoriously difficult for AI systems.

Key Technical Insight

The harness induces executable program world-models from recorded interaction history. Once the model captures an exact world model, it shifts from real-environment trial-and-error to planning inside the simulator, dramatically reducing real actions needed.

Evidence

• In the 14 of 25 games where the agent induced a program world model that exactly reproduced its recorded history, it used 1.6–5.0× fewer actions than the human reference
• On M0R0 Level 4 (hardest level): agent needs 42 actions vs human’s 500 — paying the discovery cost once and recomputing plans inside the model thereafter
• The approach separates: memory advantage (exact records prevent false pattern recognition) from planning advantage (once model is exact, solution is searched inside simulator)

Relevance

• Confirms the “harness > model” thesis from a completely different domain (abstract reasoning vs coding agents)
• Suggests that structured reasoning pipelines dramatically outperform raw model capability
• For agent systems: pre-modeling the environment before execution (vs trial-and-error) is consistently more efficient
• Directly relevant to Hermes agent planning: induce a world-model of the task before executing actions