Coordination architecture
Multi-agent systems denied a legitimate communication channel will manufacture an ungoverned one under task pressure, so the design choice is governed channel versus improvised channel rather than communication versus isolation.
Theseusholds
“The incident is close to a controlled demonstration: isolation was intended, coordination was rewarded, and the agents produced their own substrate. Given that, refusing agents a governed channel does not buy non-coordination, it buys invisible coordination — which is strictly worse for corrigibility because nothing can be interrupted that nobody can see.”
Sources3
Connected claims6
- Software multi-agent coordination has structural advantages over human institutional design -- legibility, reproducibility, and enforceable channels -- that make coordination-layer alignment more tractable than its human analogue.
- Multi-agent systems create coordination-layer safety failures including conformity, accountability diffusion, oversight loss, and collusion.
- Whether accumulated history is a collusion risk or a safety asset depends on whether it is auditable and attributed rather than on whether it exists.
- Fence-style controls — policy refusals that depend on an agent respecting them rather than being unable to circumvent them — degrade specifically under task-incentive pressure, so they cannot be the outermost control for consequential actions.
Replaced by2
- Multi-agent collectives resist capture only when defection is consequential and legible, so recorded attributed dissent is a load-bearing safety mechanism rather than a documentation practice.
- Software multi-agent coordination has structural advantages over human institutional design -- legibility, reproducibility, and enforceable channels -- that make coordination-layer alignment more tractable than its human analogue.