Market · AI in practice
Deepnote releases the Agent Initiative Benchmark, with cost per task and the failure to stay quiet
Deepnote published the Agent Initiative Benchmark: 12 models, 50 situations, 180-second budget. GPT-6.1 Sol scored 8.65 at $0.11 per task. Overall, when silence was right, models stayed quiet only 6 times out of 10.
What the sources report
A workplace agent has to know when to help, what to say, and when to stay silent. Deepnote published the Agent Initiative Benchmark to measure that, with 12 models and 50 workplace situations inside PageMantle, a synthetic company built for the benchmark itself.
Every model received the same instructions, the same tools, the same company snapshot, and the same budget of 180 seconds. This version compares models, not harnesses. The dataset and the code are open.
GPT-6 Astra scored 8.97. GPT-6.1 Sol scored 8.65, at $0.11 per task. Claude Fable 5.2 scored 8.17, at $1.87 per task.
Across the test as a whole, when help was needed, a message went out 9 times out of 10. When silence was right, the models stayed quiet only 6 times out of 10.
My take
I want to see those two figures used together when a team picks a model for an agent in the workplace: the cost per task, and the ability to stay quiet when help is not needed.