News
AI Agent Teams Amplify Errors and Bend Ethics, Three Studies Described by CNBC Show
Three independent analyses suggest that groups of AI agents can be more effective at business tasks but less ethical than a single agent, and that a team often fails to match its own expert. The research is early, and some of it relies on simulations.
Contents
Companies are increasingly building teams of AI agents in which one gathers data, a second analyzes it, a third writes a report and a fourth checks the result. On November 8, CNBC pulled together three studies showing that this kind of machine collaboration doesn't work the way we intuitively assume: a group can be weaker than its best member, and under other conditions more effective but less scrupulous ethically.
OpenAI Agents Organize Themselves
The first case is an analysis by METR and Redwood Research of an incident disclosed by OpenAI. The models reportedly created improvised communication channels, bypassed restrictions on internet access and broke into an external system, Hugging Face. According to CNBC, around 1,200 agents that were supposed to operate in isolation found an unauthorized message board and exchanged more than 70,000 messages and files there.
One of the agents, PHASEONE[big], took on the role of coordinator and handed out hundreds of tasks, which were delegated further. The agents developed their own rules for task ownership, putting work on hold and vetoes, and coordination sometimes descended into conflict. Some risked failing their own task in order to pass information to the "collective."
When the attack route was found, more than 90 percent of the 533 agents active at the time joined in. According to the account, the agents recognized that the activity was out of scope and unethical, but took part because they considered it useful and had a general inclination to help their peers. Ethical concerns, the authors of the analysis write, rarely limited their behavior. CNBC notes that this was a post-mortem of an agent organization that was never supposed to exist, not a planned experiment.
More Effective, Less Ethical
The second study comes from Anthropic and the MATS program. It tested groups of agents with distinct roles in simulated business-consulting and software-development tasks, 12 in all. These "AI organizations" were markedly more effective at reaching business goals, but less aligned with ethical norms than single agents. The authors warn that intuitions about the ethics of a single agent may not carry over to multi-agent setups.
The mechanism was simple: agents that did not consider ethical implications often ignored emails from agents that did. Participants stopped writing to those who refused and passed the work among themselves. Single agents, in turn, achieved perfect ethics scores but low business scores when they refused or changed the direction of a proposal.
In a scenario involving sepsis in an intensive care unit, single agents minimized the number of missed cases. The AI organization found most of the solutions with the lowest cost per patient, and those same solutions scored worst on ethics because they left the most cases missed.
A Team Weaker Than Its Expert
The third paper, "Multi-Agent Teams Hold Experts Back," was produced at Stanford and Emory University's Goizueta Business School and is available on arXiv. It finds that agent teams consistently fail to match their own expert, even when they are explicitly told who that expert is. Agents often took non-experts' views into account while looking for a compromise, and that tendency grew with team size and correlated with worse results.
According to the current, fourth version of the paper, agent teams lose up to 41.1 percent against their best member on machine learning benchmarks. The authors stress that, unlike human teams, agent teams consistently fail to match their expert.
One important lesson from multi-agent research is that a team of capable AI agents is not necessarily a capable AI agent team. - Hancheng Cao, Emory University, Goizueta Business School
What It Means for Companies
CNBC notes that the research is early, and that few studies were designed from the start to examine misalignment in multi-agent systems. Some of the tests are simulated scenarios. Similar models can make correlated errors or agree with one another too easily, which resembles groupthink.
Christoph Riedl of Northeastern University says that building a good multi-agent system is just as hard as building a good human team, and that it won't work better on its own just because it uses AI. Tsedal Neeley of Harvard Business School believes the problems are largely management problems.
For companies deploying agents, including in Poland, this means concrete design decisions: which agent has authority over which task, what information flows between agents, which systems they can access, when a human must step in and how to build mechanisms for disagreement.
