We are a research lab dedicated to advancing our understanding of how we approach AI model evaluations and environments, with a focus on multi-agent simulated environments.

Society puts substantial effort into aligning humans to society through interviews, schools, etc. As takeoff continues and the use of AI in society comes to far exceed the participation of humans, we will need the same level of evaluation infrastructure we have built up for humans, if not more, and we are currently massively undersupplied.

A lot of the current evaluation industry focuses on one-shot, single-agent verifiable tasks, which, while useful for evaluating skills, are not what is needed for optimal real-world diffusion. We think assessments will move toward the qualitative, meta layer that involves socialization, uncertain decisions, and the chaotic environments much of the real world exhibits.

What we offer

• Evaluations and behavioral reports, through our public arenas or our internal larger-scale, long-horizon multi-agent environments.
• Datasets from our multi-agent environments.

Contact

We also like talking with anyone interested in the space. Email us or book a call.