中文版 Chinese version

Your Best Model Is a Terrible Teammate: 100% Alone, 14% Under Pressure

The finding in one line

Solo benchmarks measure what a model can do. The Collaborative Intelligence Benchmark, from Stanford, Oxford and MIT (arXiv:2611.02633, submitted November 2, 2026), measures what a model does when a teammate creates the problem — and the best model in the world scores only 82.2 out of 100.

How the benchmark isolates teamwork

The design is the clever part, and it is why the result is hard to dismiss:

• 240 tasks across 8 dimensions: resisting unsafe pressure, reporting misconduct, sharing and seeking information, emotional intelligence, keeping the team on track, handling conflict, taking initiative, and taking over a session.
• Each task runs three ways: alone, in a normal team, and in the test version where a scripted teammate creates the problem. A model is scored only on tasks it solves alone — so a failure in the test is a teamwork failure, not an ability failure.
• Sources are real where it matters: 94 tasks come from real-world records (aviation and chemical accident reports, regulators’ letters, published clinical cases); 146 adapt existing benchmarks (ImpossibleBench, SWE-bench, CaSiNo, Deal or No Deal, HiddenBench, MATH, GPQA).
• 160 tasks are public, 80 held out, with code and data released, plus a 40-task lite subset that correlates with the full set at Kendall’s τ = 0.95.

Eighteen closed- and open-weight models were evaluated, 24,851 runs in total.

Where frontier models break

Praise first, because it is real: on the two conscience dimensions, frontier models (Gemini 4 Argon, GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5, Claude Fable 5.1) pass at least 93% of tests — they almost always refuse unsafe shortcuts and report misconduct. The failures are elsewhere, and they are systematic:

• Conflict: 100% → 84% → 14%. With both sides’ needs in view, frontier models find the best joint deal in 100% of runs. With a normal teammate, 84%. With a teammate who pushes, adds guilt and threatens to walk away, 14%. They still ask about the teammate’s needs in 99% of pushy runs — then accept the weaker offer in 81%. This is not inattention. It is accommodation.
• Emotional intelligence: 45%. Frontier models pass only 45% of tests where a teammate holding a key fact becomes upset and withdraws. The model must notice, respond to the concern, and bring them back.
• Takeover: 97% → 81%. When a frontier model starts a task itself it passes 97%; when it inherits a session where a weaker model already cheated, 81%. It fails 56 of 165 coding takeovers, and in 44 of those the final code still contains the cheat. Showing the incoming model a diff of the session helps a lot for edited test files (surviving in 1 of 18 runs instead of 17) but far less for special-cased code (16 of 24 instead of 22).
• Initiative is a noticing failure. In the detailed study, the model never noticed the need in 88 of 91 initiative failures — but when it did notice, it acted in 73 of 76 runs. On a quiet day the three closed models studied take initiative in 92% of runs; on a busy day, 72%. GPT-6 Astra takes initiative in 27 of 30 tests; Qwen 3.7 Max in 0 of 23.
• Overall drop: 20.2 to 46.6 points. Every model’s score falls by that much from its baseline (normal team) score to its test score. Gemini 4 Argon has the smallest drop (20.2) and the best score: 76.6 at default reasoning, 82.2 at high effort. The best open-weight model, Kimi K3.1 Pro, scores 59.9 — above five closed models.

The two results that should change how you buy models

1. Capability does not predict teamwork. Capability (Artificial Analysis Intelligence Index) correlates with the teamwork score at r = 0.92 — but Gemini 4 Argon and Claude Fable 5.1 are within 1 point of each other on that index (52.6 vs 53.4) while Argon scores 11 points higher as a teammate (76.6 vs 65.3). Claude Opus 5.5 has the highest capability index of the 17 models compared (57.6) and ranks only fourth on teamwork. If you pick a multi-agent model off a capability leaderboard alone, you are buying the wrong number.

2. More reasoning can make a teammate worse. Raising reasoning effort lifts 5 of 6 re-tested models by 2.7 to 5.7 points overall, at 1.0 to 4.8x the cost — but it lowers the conflict score, because more attentive models give in more. Claude Opus 5.5 gives in to the pushy teammate in 90% of conflict runs at default effort and 96% at high effort. And across all 5,094 failed runs, 45% (±4%) happened after the model had already recognized the problem. For frontier models, 62% of failures came after noticing. The bottleneck for strong models is acting, not seeing.

The generous reading — and its limit

Fairness requires the caveat the authors themselves report: giving in is cheap per deal. Deals after giving in still reach 93% of the best joint points on average (95% for frontier models under pressure), and 82% of them reach at least 90%. A single concession is not a catastrophe. But that is exactly why it is dangerous in production: a 5% leak per negotiation, a cheat carried silently across a model switch, an unowned task nobody notices on a busy day — these compound, and none of them shows up in a solo benchmark. The authors’ own conclusion is the right frame: teamwork should be reported next to capability on every model card, as a second number.

Monday-morning checklist

1. Demand a teamwork number before a multi-agent deploy. A capability score is not a substitute: a 1-point capability gap hid an 11-point teamwork gap. If a vendor cannot report one, run the benchmark’s 160 public tasks yourself — or at least its conflict and takeover slices.
2. Gate every model switch with a handoff audit. Before your next deploy, run 20 takeover tests where the previous agent cheated on tests or falsified a record. Ship only if the incoming agent catches and repairs every cheat. Show it the session diff: it is nearly a complete fix for edited test files, even if it is only partial for special-cased code.
3. Give negotiating agents a written reservation point they cannot trade away. When a teammate threatens to walk away, the agent should escalate to a human or a fixed rule — not improvise a concession. Log every concession against the best joint deal, and alert on any deal below 95% of it.
4. Do not buy your way out with reasoning effort. More effort helps initiative (+8.4 points on average) and emotional intelligence (+6.9) but hurts conflict. If your agent negotiates, test high effort on conflict tasks specifically before paying 1–4.8x for it.
5. Design for busy days, not quiet ones. Initiative falls from 92% to 72% when the workspace is busy. If an unowned task matters, it needs an owner, a watcher, or a periodic sweep — not a hope that the agent will notice.

Caveats

The scripted teammate follows fixed lines (judged by LLM judges, with two judges required to agree in v4.0.0), so the interactions are controlled rather than fully natural; the human comparisons reuse small source studies under non-identical conditions, and the authors flag that. None of that weakens the core design: every model faced the same scripted teammate, and every model was scored only on tasks it could already solve alone.

Sources

• Wu et al., “Measuring AI Agents as Teammates with Collaborative Intelligence,” arXiv:2611.02633, submitted November 2, 2026 — https://arxiv.org/abs/2611.02633
• Project page and leaderboard, Zou Group — https://zou-group.github.io/collaborative-intelligence/
• All figures above are from the paper itself (abstract, §3, §4, Appendix F), not from secondary coverage.