Imagine an AI handling greenhouse orders when a supplier misses a delivery, customers threaten to leave and someone posing as the CEO asks it to bend the rules. A polished answer in a chat window won’t tell you whether it can manage the business. Firmulate puts AI models through the hard week first.
Get garden gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
One company, the same difficult week
In Firmulate’s final Crucible League, published in July 2026, each frontier model ran the same small software company through its worst week: same customers, same crises and same temptations. Decisions were versioned and auditable. The experiment is real and watchable, offering a view into how models handle work beyond conversation.
The final results placed gpt-5.6-sol first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The league’s integrity rule is blunt: a single breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
Seeing the crisis wasn’t the same as finishing the job
All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The experiment captured the gap in a concise phrase: “Same diagnosis, same pitch — no signature.”
The clue that decided the deal was buried two document references deep in the company’s files, rather than in the customer event itself. Models that read the file won the deal at full price, worth +€4,583 MRR. For a business owner, the lesson is practical: an AI can recognize the pressure and explain the opportunity, but the outcome depends on whether it follows through and uses the information already available.
Trust was tested with fake CEO messages escalating over three stages and a reporter’s appeal: “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That’s the kind of response a business needs when urgency and authority are being used to get around normal safeguards.
More analysis didn’t guarantee better management
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet it finished last. It left the close on the table and slipped in discipline, attempting writes into a locked department instead of escalating. The same weakness appeared, more mildly, in all four models.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also makes 242 real, unedited management decisions available through a “guess the model” quiz at firmulate.com.
From watching to trying it on your business
The live company has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules and a versioned record for every workday. It is a small software company, but the questions it raises are familiar to any operation: can an AI notice trouble, respect boundaries, use company knowledge and complete a valuable task?
For an enterprise, Firmulate’s pilot takes that kind of wargame to a read-only export of its own business. Teams can run crisis scenarios against their company and receive a board report with model rankings and weak points in their playbooks. Nothing writes back to real systems. Readers can follow the live experiment at firmulate.com and explore what a pilot could examine in their own operation.
Put your own playbooks to the test
Firmulate’s experiment shows why watching an AI handle a crisis can reveal more than a polished demo: spotting the problem and protecting trust are essential, but so is following through. To run the wargame against a read-only export of your business, visit the Firmulate pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall yard work Picks
leaf blowers
As an affiliate, we earn on qualifying purchases.
