WorldBuild Bench

WorldBuild Bench is an innovative benchmarking harness designed for assessing AI-generated 3D browser games. It maintains a consistent environment across multiple models, enabling a fair evaluation of game design capabilities under similar conditions. This project ensures that variations in outcomes directly reflect the performance of the AI models rather than differing design preferences.

Project Overview

The goal of WorldBuild Bench is to explore how well AI models can produce engaging 3D games that exhibit spatial, temporal, and causal coherence. Unlike conventional benchmarks which merely check if a game runs, this framework examines whether AI understands the dynamics of a game world—its coherence, functionality, and user engagement.

Key features include:

• Standardized Task Briefs: Each game is developed based on the same brief, allowing for precise measurement of execution without the influence of subjective tastes in game design. The tasks include:

  • Arena Combat: Engage in intense battles against waves of AI enemies.
  • Physics Puzzle: Solve various physics-based challenges by manipulating objects.
  • Racing: Compete in high-speed races against AI competitors.

• Diverse AI Models: WorldBuild Bench runs multiple AI models, each producing unique gameplay experiences from the same brief, which can be instantly played in the browser. The current pilot showcases games produced by nine renowned models, resulting in 27 distinct builds available for user experience.

• Human-Centric Evaluation: Automated tests gauge playability and world coherence, but user feedback collected through the WorldBuild Arena is paramount. Players can participate in blind comparisons of games and provide ratings based on overall preference, game feel, and design quality.

Technical Insights

WorldBuild Bench employs a robust architecture utilizing TypeScript, Three.js for 3D rendering, and Playwright for automated gameplay testing. Key performance metrics, such as playability and world coherence scores, are calculated alongside transparency around model execution time, costs, and the amount of code generated.

Join the Benchmarking Revolution

WorldBuild Bench is paving the way for a new standard in evaluating AI-generated games. By providing a rigorous, consistent testing framework that emphasizes player engagement, this project captures the essence of what makes a game enjoyable while fostering further advancements in AI capabilities.

Explore and Participate

To engage with the builds and vote in the arena, visit the WorldBuild Bench Playable Builds and cast your votes. Your insights can significantly influence the future of AI-generated gaming.