Five model families that existed at the time of the first benchmark pass but weren't surfaced by the initial ollama.com search: - qwen3.5:4b-q4_K_M - gemma4:e2b-it-q4_K_M - gemma4:e4b-it-q4_K_M - granite4.1:3b-q4_K_M - ministral-3:3b Grouped as tier="modern" with a new orchestrator filter so they can be re-run independently. 25 runs in 11.2 minutes, zero errors. Three findings reshape the recommendations: 1. gemma4:e4b-it-q4_K_M is the new local leader for room classification. Closed-set 0.62 (vs qwen3:4b q4 at 0.61), open-set 0.65 (highest score across all measured models including cloud). The cloud "ceiling" on open-set was 0.61 — a 4B local model now beats every cloud reference up to 1T parameters on this task. 2. ministral-3:3b has 0.99 memory coverage — nearly cloud-tier. But weak elsewhere (closed-set 0.49, entity 0.63). Specialist not generalist. 3. qwen3.5:4b does NOT beat qwen3:4b on this workload despite the version bump. Slightly better entity F1, worse on memory and closed-set. Newer != better. Production-tier update: - Room classification (OpenClaw use case): gemma4:e4b becomes the new recommended default. Costs 2.1x latency and 1.4x VRAM vs qwen3:4b for the room-task lift. - General extraction on tight hardware: qwen3:4b q4_K_M still wins. - Open-set discovery: softens from "shelved indefinitely" to "retest with gemma4:e4b + prompt tuning before declaring dead." Gap to 0.70 ship threshold is now 0.05, not 0.09.
Reports directory
Hand-written markdown analyses go here. After each benchmark run, render a summary table from the CSV and write a narrative interpretation: which models won, where the surprises were, which entries belong in the next MODEL_TIERS revision.
Reports are committed (unlike the raw CSVs in results/).