oaustegard/experiments · doom-planner

Doom by Rollouts

This Doom player, called the planner below, decides what to do by trying moves out in copies of the game. Four times a second it copies the game, tries a different move in each copy, and plays whichever move worked out best. Each copy played out this way is called a rollout. The planner reads the game's internal state and never looks at the screen.

The whole experiment in 87 seconds, narrated by Muninn.

Each decision tries 30 options in 90 copies of the game

Doom updates its world 35 times a second, and each update is called a tic. Every 8 tics, about a quarter of a second, the planner pauses the real game and does this:

  1. Copy the game 90 times. The game is a program running on Linux, which can duplicate itself (the fork system call). Each copy holds the whole game as it stands: the map, every monster, every door, and the player's position, health and ammo.
  2. Give each copy one option to try. There are 30 options, such as walk forward, turn left while firing, strafe right, walk into a door pressing "use", turn 90° and then run, or keep doing what it was already doing. Each option is tried in three copies.
  3. Roll different dice in each of the three. Doom uses random numbers to decide whether a shot hits, how much damage it does and where a monster wanders. Each of the three copies has its random numbers reseeded differently from the other two, so they show three ways the next moment could go. The reseeding is the same in every run, so a run repeats exactly. None of them uses the real game's random numbers, so the player never sees the future that will actually happen.
  4. Let each copy play for a second and a half, then score it. After 52 tics, each copy reports points for damage it dealt, kills, items picked up, walls it saw for the first time, and getting closer to parts of the map it has not seen yet. "Closer" is walking distance, measured through the map's rooms and doorways, so a wall between the player and an unexplored room does not count as close. It loses points for damage it took, ammo it used and standing still. A copy that reaches the exit, which the game itself reports, earns a large bonus, and dying costs a large penalty.
  5. Pick the option, play it, repeat. Each option's three scores are averaged, and the average is pulled halfway toward the worst of the three, so a move that is usually good but sometimes fatal loses to a steadier one. The best option is played in the real game for 8 tics, and then the player decides again.

A 5-minute game takes about 1,300 decisions and 117,000 copies, which run in about four minutes on four CPU cores. The player knows the map's layout, which it needs to measure walking distance, but only the exit-assisted run below is told where the exit is. The first level, E1M1, ends at a wall switch. The map tells the player where every wall switch is, and once nothing unexplored is left within reach it goes and presses the ones it has not tried.

Haiku chose every move in two of the six runs

The question that started this was whether Haiku, Anthropic's smallest Claude model, can play Doom. Four kinds of run separate what Haiku did from what code did.

Every Haiku call uses Haiku at low reasoning effort. In every run, Haiku is told it is playing Doom, on which level and difficulty, that the goal is the exit, and what the actions do. Code, not Haiku, presses "use" to get past the screen between levels and to restart after a death.

Four of the five E1M1 runs reached the exit within five minutes

Times are game time. Each video is the game as it was played, every frame, at normal speed, on the middle difficulty ("Hurt Me Plenty"). The caption at the top left shows each decision: the option played (marked "plan", "HAIKU" or "JUDGE"), the planner's score for it, and the planner's next two options with their scores. In the screen-only run there are no scores and every number reads 0. After the exit, each run carries on into the second level, E1M2. A death restarts the level with its monsters, so kills count from the last restart.

Exploring without exit knowledge

The planner explores, fights, and once nothing unseen is reachable it goes to press the switches it has not tried.

E1M1 exit
2:43
E1M1 kills
6 of 6
Deaths in 5 min, any level
0
Decisions
1,302
Rollouts (copies played out)
117,174

Exit-assisted

The same planner, also told where the exit switch is and given points for walking toward it.

E1M1 exit
0:58
E1M1 kills
3 of 6
Deaths in 5 min, any level
1
Decisions
1,302
Rollouts (copies played out)
117,171

Haiku choosing every move from played-out results

The 90 copies run as before, but Haiku makes every choice. It sees the screen and, for each of the 30 options, what happened in its three copies: damage dealt and taken, kills, deaths, exits, new walls and areas, items, ammo, distance moved and distance gained toward unexplored space. It does not see the planner's combined score or ranking. About 5.4 s per answer. A two-hour limit on background jobs stopped the run 2.8 s of game time short of five minutes.

E1M1 exit
4:27
E1M1 kills
6 of 6
Deaths in 5 min, any level
2
Decisions
1,292
Rollouts (copies played out)
116,268
Haiku calls
1,292 (5.36 s each)
Haiku picked the planner's top option
17%

Haiku choosing every move from the screen

No copies and no scores. Every 8 tics Haiku sees the screen, its health, armor and ammo, and the list of 30 options, and picks one. The game waits for each answer, about 4.8 s.

E1M1 exit
not reached
E1M1 kills
0 of 6
Deaths in 5 min, any level
1
Decisions
1,311
Haiku calls
1,311 (4.8 s each)

Exploring with Haiku on close calls

As the first run, but when the two best options score within 10% of each other, Haiku makes the call. It sees a picture of the screen now and of where each close option leads 1.5 s later, with their scores, plus a one-line note the previous Haiku call left about where the player was heading. It runs at low reasoning effort, about 5.6 s per call, which stretched this run to 24 minutes of real time. Haiku's answers can vary, so this run would not repeat exactly.

E1M1 exit
3:37
E1M1 kills
6 of 6
Deaths in 5 min, any level
0
Decisions
1,301
Rollouts (copies played out)
117,084
Haiku calls
197 (5.56 s each)
Haiku kept the planner's top pick
66%

Left to play for 15 minutes without exit knowledge, the same planner finished E1M1 at 2:43 (the same game, since runs repeat exactly) and spent the remaining 12 minutes on E1M2 without finding its exit, with 0 deaths and 4 kills there. Watch the 15-minute run.

Haiku choosing every move cleared E1M1 only with the played-out results

Given only the screen, Haiku saw about as much of E1M1 in five minutes (318 wall segments) as the planner saw before its exit at 2:43 (322), mostly by walking forward and turning, and noted correctly what it saw ("Fists out, no monsters visible. Green wall on the right, open grass area to the left."). It never found the exit, ran out of ammo, died once and ended with no kills in its last life. Given the outcomes of each option in three played-out copies, Haiku cleared the level at 4:27 with all six monsters killed, after dying twice. It chose the planner's top option only 17% of the time, so it weighed the outcomes its own way. It was slower and less safe than the planner's arithmetic.

The planner alone, with no idea where the exit was, cleared E1M1 at 2:43 with no deaths; knowing where the exit is cut that to under a minute. Each setting ran once, so these runs show what each player can do; how often it does so is unmeasured.

The second level, E1M2, stopped the planner. That map has a door that needs the red key, which the walking-distance measure treats as an ordinary door, so it can pull the player toward a door it cannot open. The map also has lifts, which the walking-distance measure leaves out. Which of these stands between the start and the exit has not been traced.

Five rules keep the planner from stalling