Soccer-Bench
Test language model teamwork in simulated 5v5 soccer matches online
About
Soccer-Bench is an experimental platform designed to evaluate the cooperative abilities of language model agents in a simulated environment. The core setup features two teams of five agents playing against each other in a digital 2D soccer match. Each player is represented by a separate large language model instance, queried independently every second to simulate real-time decision-making.
Coordination is deliberately constrained so that the only communication allowed between agent-players is a brief shout, which teammates can hear during their next simulated second. This setup is intended to reveal how well various models, like GPT and Claude, perform when required to collaborate under realistic communication limits. These games expose both the strengths and weaknesses of current AI models when working together in complex, dynamic contexts.
A live leaderboard is included to showcase outcomes, highlight notable results like surprising upsets or high-scoring matches, and display basic statistics such as goals and completed passes by model type. The platform is open and can be explored through an interactive web interface, offering insights for AI researchers and anyone interested in the current capabilities of multi-agent language models.
Who is Soccer-Bench made for?
Soccer-Bench is intended for AI researchers, developers, and product managers who are investigating the teamwork and coordination abilities of language model agents. It is particularly relevant to those designing multi-agent AI systems, seeking benchmarks to evaluate agent collaboration, or experimenting with agent-based environments.
It is also a useful reference point for academics, students, or hobbyists interested in understanding the real-world limitations of current LLMs in dynamic multi-agent scenarios. The tool provides a straightforward way to compare agent performance and offers valuable insights for anyone working on complex AI agent applications where coordinated decision-making is crucial.
Is Soccer-Bench yours? Claim this listing to manage it and get featured.
Get tools like Soccer-Bench every Tuesday
The best new AI evals & observability, in one short email. Free.