1The University of Hong Kong 2LIGHTSPEED 3The Chinese University of Hong Kong 4Tsinghua University
‡ Project Leader † Corresponding Author
Most game benchmarks score a foundation model the first time it meets a game, and rarely measure what the model can make of the game once it has played it and reflected on its play. We measure a model at three levels, a cold-start score at its first contact with a game, an Improvement Dynamics Curve (IDC) of its score after every round of reflection, and a transfer test that re-runs the skill it learned through reflection on variants of the game. These levels need games that no model has seen and that ship with variants of themselves. We built OmniGameArena, a single Unreal Engine 5 build of twelve such games played from the rendered frame, covering solo, adversarial and cooperative play, each with four variants that alter it in one respect. Across 72 curves of six VLMs, reflection raises the best round more than 0.1 above the cold-start score in 42, yet in half of these the first round shows at most 0.05 of the gain and in a quarter the last round ends more than 0.1 below the best. The learned skills gain 0.19 on average on the game they were written from and 0.03 on its variants, and the size of the gain on the practiced game explains about a tenth of what carries over. Neither pattern is visible in a single score, whether it is taken at first contact or after one round of reflection.
Each game targets a different mix of visual perception, spatial navigation, reaction, memory, planning, adversarial interaction, and cooperation.
Each measurement builds on the one before: round 0 of every curve is the cold-start score, and the transfer test re-runs the skill of the best round of the curve.
The agent plays from the rendered frame under an empty skill prompt. The game is paused while the model thinks, so the score reflects the decisions alone. A cell is five episodes of one agent on one game, and the cold-start score is their mean.
Each round the agent plays five episodes under the current skill prompt, and the same model then reflects on the trajectories and writes the next skill. Ten rounds of reflection give the curve of round scores S0 to S10, where S0 is the cold-start score.
The learned skill, the skill of the best round, is run on each of the four variants of the game for five episodes. The reference is the cold-start score of the same agent on the same variant. The variants were fixed before any transfer run and are the same for every agent.
Multi-round reflection
Reads the round's trajectories through sandboxed read-only tools (list_dir, read_text, read_image, grep).
Commits an explicit list of failure modes, separating causes from prescriptions.
Proposes an updated skill to an independent LLM judge, which rejects proposals that memorize map content or contradict the diagnosis; up to five tries.
Finalizes the accepted skill for the next round and, optionally, adds durable observations to the notebook.
Twelve agents on the twelve games. Round 0 of every curve below is one of these cells.
Gameplay recordings
Of the 72 curves, 42 have a best round more than 0.1 above R0. On LastStand, CueChase and CrystalGuard all six curves rise; on MidlineClash none does.
Of the 42 rising curves, 21 gain at most 0.05 in the first round of reflection and reach their best round between rounds 2 and 9.
Of the 42 rising curves, 10 end more than 0.1 below their best round and 3 more than 0.2 below it.
The 72 skills gain 0.19 on average on the practiced game and 0.03 on its variants, where they score below the cold-start score in two cells out of five.
In part: the size of the gain accounts for about a tenth of the variance in what carries over (Pearson r = 0.34 across the 72 skills).
Claude Opus 4.7 and GPT-5.5 gain 0.62 and 0.55 over R0. On the variants the skill of GPT-5.5 scores above the cold-start score on all four, the skill of Claude Opus 4.7 below it on all four.
@article{lin2026omnigamearena,
title = {OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics},
author = {Lin, Mingxian and Qian, Shengju and Liu, Yuqi and Huang, Yi-Hua and Wang, Yiyu and Huang, Wei and Li, Yitang and Zhang, Fan and Hu, Zeyu and Zhu, Lingting and Wang, Xin and Qi, Xiaojuan},
journal = {arXiv preprint arXiv:2606.09826},
year = {2026}
}