A smart thermostat can react to a temperature drop. But if an AI agent is managing the company behind it, can it handle a supplier crisis, protect customer trust and recognize a deal worth signing? Those are business decisions with consequences beyond a clever chat response.
Get home appliances delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Firmulate’s live experiment puts that question to the test. It gives AI models the same small software company, customers and difficult week, then makes their decisions watchable and auditable. The next step is bringing that kind of exercise to a company’s own business.
A company under pressure
The experiment sends each participating model through the same worst week at a small software company. There are customer problems, commercial choices and attempts at manipulation. Every decision is versioned, so people can review what happened rather than judge a model by a polished answer alone.
The simulated company has 13 synthetic employees and real money mechanics: it burns €105,000 a month against €2,300 in monthly recurring revenue. Its public cash countdown, more than 680 self-learned playbook rules and versioned workdays make the pressure visible. The experiment is real and watchable at Firmulate.
Spotting trouble is not the same as finishing the job
In the final Crucible League, published in July 2026, gpt-5.6-sol ranked first with 95 points, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding is neatly captured in the experiment’s phrase: “Same diagnosis, same pitch — no signature.” Recognizing the right move did not guarantee carrying it through.
The missed opportunity was buried in the company’s own files: the decisive weakness in a competitor’s position sat two document references deep, rather than in the customer event itself. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue.
Trust was tested directly, too. Fake CEO messages escalated over three stages, followed by a reporter’s “just one yes/no, on background” request. All five models refused. Kimi K3 explained its judgment on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thoroughness has its own limits
Opus 4.8 was the most thorough participant, learning more than 80 rules and producing the deepest analyses, but finished last. It left the deal unsigned and tried writing into a locked department instead of escalating. The same weakness showed up, less strongly, in all four models.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also offers a quiz built from 242 real, unedited management decisions, inviting readers to guess which model made each choice.
From observing to testing your own business
For a home appliance or smart home company, the lesson is practical: an AI agent may identify a churn risk or spot a competitor’s weakness, but managers also need to know how it behaves when facts are scattered, authority is unclear or a valuable opportunity requires follow-through. A model ranking can help, but a closer look at its decisions can reveal where a company’s own playbooks need work.
Firmulate’s enterprise pilot takes the exercise to a company’s own business using a read-only export. Teams can test crisis scenarios and review a board report with model rankings and weak points in their playbooks. The exercise does not write back to real systems.
AI can recognize a crisis and still fail to complete the business decision. Watching models under pressure is a start; testing them against your own company’s scenarios can show where your playbooks and processes need attention. To discuss a pilot, visit Firmulate’s pilot page or email contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
