StarCraft Won’t Wait for Your AI to Finish Thinking

Key takeaways

• An agent’s performance in real-time games depends on how quickly it acts.
• A sound decision can become obsolete between observing the battlefield and issuing a command.
• Win rates need context: game speed, pauses, and the scope of automated control.
• Separating long-term planning from immediate reactions offers one way to manage delays.

An AI agent can make the right call in StarCraft: Brood War and still lose its army before the order arrives. That makes the game a useful lens for evaluating agents: once software starts taking action, timing becomes part of competence.

Every strategy has an expiration date

StarCraft’s opponent has no obligation to wait while an agent considers its options. Enemy units keep moving. Health bars keep shrinking.

An agent powered by a large language model needs time to receive information, decide what to do, and send commands. If the game keeps running throughout that process, the battlefield can change before those commands arrive.

“Retreat” might be an excellent decision. It does very little for an army that has already been wiped out.

Evaluating the agent therefore means asking two separate questions: Was the strategy sound, and did it execute while that strategy still made sense?

Three seconds can change the decision

Suppose an agent takes three seconds to go from observing a battle to issuing an order. That is a hypothetical delay, not a measured result.

It sees scattered enemy forces and chooses to attack. During those three seconds, the enemy regroups. The attack now commits its army to a fight it would have avoided with a fresh view of the battlefield.

The information behind the decision has gone stale.

Inference latency is the time the model takes to produce its response. The full delay also includes preparing observations and delivering commands. Measuring model response time alone leaves part of the agent’s reaction time unaccounted for.

Thinking longer may improve a decision. It also gives the world more time to invalidate the assumptions behind it. An effective agent needs to judge when further deliberation is worth that cost.

A win rate needs a clock

Before comparing Brood War agents, check whether the game pauses while they think.

Pausing helps isolate their ability to choose a strategy. Leaving the game running makes response speed part of the challenge. Both setups can tell us something useful, but their win rates answer different questions.

Game speed matters for the same reason. A fixed response delay leaves less room to react when the simulation runs faster.

Then there is the question of control. An LLM might issue a broad instruction such as “attack this area,” with a separate controller handling movement and combat. Or it might have to direct individual units’ actions. Those are substantially different workloads.

A loss could reflect poor strategy, a late command, or a failure in execution. “The AI is bad at StarCraft” does little to identify which component needs fixing.

Give urgent reactions a shorter path

One possible design separates long-term planning from immediate control.

The LLM could choose production priorities and attack objectives. A faster controller could handle urgent actions, such as moving a unit out of danger.

Think of a software team during an outage. The person setting the quarterly roadmap does not need to approve every immediate response. Requiring that approval would introduce a fairly predictable bottleneck.

This division still needs rules. A controller’s emergency retreat might disrupt the larger plan. The system must define which actions it can take immediately and which changes require the planner to reconsider.

It also complicates evaluation. If win rates improve, the gain could come from better planning, faster execution, or both. The whole system’s performance and the LLM’s contribution deserve separate scrutiny.

For an agent, a good decision has a deadline. StarCraft makes that deadline visible: by the time an answer arrives, the army it was meant to save may already be gone.