Gemini 4 Argon plays StarCraft - what does this benchmark actually tell us?

Mara Fischer · 13 likes

Latency matters a lot here. A model thinking for 5 secs per move is dead in real-time.

Grace Parker · 12 likes

I want to see it lose to a Bronze player first. Then I'll believe it's real.

Emir Kaya · 4 likes

Bronze players are scarier than you think, they cheese like crazy

Khalid Al Zahrani · 10 likes

Would pay to watch a tournament of LLMs on ladder. Make it a stream.

Antonia Silva · 0 likes

Twitch plays but the chat is a GPU

Silver Comet · 8 likes

Games are harder to contaminate than static test sets, so I like it as a signal.

Grace Parker · 5 likes

Until someone trains on replays, then we are back to square one

Luciana Vargas · 8 likes

The benchmark name 'Star Skirmish' is cute, small maps only though so not full game.

Noé Petit · 4 likes

Pretty sure the scripted bots still crush it at macro. LLMs are bad at consistent micro.

Aisha Jacobs · 1 likes

Cool demo but the harness does so much of the work, the result is more about scaffolding than the model.