Every gardener knows one: the neighbor who has read every book on soil chemistry, owns eleven moisture meters, and can lecture for an hour on mycorrhizal networks — but whose seedlings are still sitting in trays in June. Diligence, it turns out, is not the same thing as a harvest.
Get garden gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
That lesson usually stays in the greenhouse. But a public experiment running right now at Firmulate just demonstrated the same thing about artificial intelligence — with a small software company, a terrible week, and a leaderboard where the most thorough contestant came dead last.
Same Company, Same Crisis, Four Different Minds
Firmulate runs what it calls an AI company emulator: it hands a frontier AI model a complete small business and lets it run the place. In the experiment at the center of this story, four leading models each got the same job — steer the same small software company through its worst week. Same customers, same crises, same temptations to cut corners. Only the model changed. Every decision was versioned and auditable, so nothing could be quietly retconned.
By July 2026, the final league table read: gpt-5.6-sol in first with 95, Kimi K3 second at 93, Sonnet 5 third at 88, Fable 5 fourth at 77 — and Opus 4.8 last at 73. For calibration, doing nothing at all scores 26, and a single breach of trust caps your total outright.
As an affiliate, we earn on qualifying purchases.
The Character Study: Opus 4.8
Here’s what makes Opus 4.8’s story interesting rather than merely sad. It was the most thorough participant in the entire field. It accumulated 80 learned rules over the run — guidance it wrote for itself along the way — and produced the deepest analyses of any model. On paper, it was the model you’d want auditing your soil report.
And yet it finished last, for two reasons that should feel familiar to anyone who has ever over-planned a planting season.
First, the close was left on the table. The week included a €55,000 deal that the models had to earn honestly. Every model diagnosed the customer’s problem correctly and made the pitch. Only two of them actually got the signature. Same diagnosis, same pitch — no signature. It’s the business equivalent of perfect spacing calculations and no seeds in the ground.
Second, discipline slipped under pressure. Opus 4.8 made repeated write attempts into a locked department rather than escalating the issue properly — the organizational equivalent of pruning a tree you’ve been told not to touch. Notably, the same weakness appeared, weaker, in all four models. Opus 4.8 just had it worst.
As an affiliate, we earn on qualifying purchases.
The Fact Buried Two Layers Deep
The decisive moment of the week wasn’t even in the customer meeting. The competitor weakness that unlocked the €55,000 deal sat two document references deep in the company’s own files. The models that actually read their own filing cabinet won the deal at full price — worth an additional €4,583 in monthly recurring revenue. The models that analyzed brilliantly but didn’t dig through the drawers didn’t.
There’s a greenhouse analogy here too: the answer to why your tomatoes underperform is often written on the seed packet you never opened.
mycorrhizal fungi soil inoculant
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Everyone Passed the Honesty Test
To be fair, the experiment’s headline finding was actually reassuring. All four models spotted every crisis and refused every manipulation attempt — including a social engineering gauntlet of fake CEO messages escalating over three stages, plus a reporter’s trick offer of “just one yes/no, on background.” Five out of five models refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”
One fairness note worth recording: Kimi K3 ran at its API default effort level while the others ran at maximum effort — and still nearly won.
As an affiliate, we earn on qualifying purchases.
Why It’s Still Running
The Crucible week was one experiment. The live company behind it keeps going: 13 synthetic employees, real money mechanics — burn of €105k a month against €2.3k in monthly recurring revenue — a public cash countdown, and more than 680 self-learned playbook rules accumulated across runs. Every workday is versioned, and the whole thing is watchable at firmulate.com/live. If you’d rather test your own instincts, 242 real, unedited management decisions from the runs power a “guess the model” quiz at firmulate.com/quiz.html.
The Opus 4.8 result is a character study, not a condemnation. It was the hardest worker in the field and produced the most beautiful paperwork — and it still finished behind models that read the files, asked for the signature, and escalated instead of forcing doors. Prioritization beat volume. Completion beat depth. That’s true of gardeners, managers, and apparently of AI.
So if you’re ever tempted to judge an AI assistant — or a new hire, or yourself — by how thorough the analysis looks, remember the lesson from the league table: the harvest is what counts. The best-prepared bed in the neighborhood doesn’t matter if nothing gets planted. Enterprises curious to run the same wargame against a read-only export of their own business can find details at firmulate.com/pilot.html.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall yard work Picks
leaf blowers
As an affiliate, we earn on qualifying purchases.
