BattleCast

A few months ago I wrote about building a D&D combat simulator . Once BattleCast could run whole battles on its own, a question started nagging me: if I handed the controls to a language model, would it actually play well? Not “can it talk about tactics” - can it stand on a bridge, look at eight combatants, and make the right call? That question turned into D20bench, an open-source benchmark where LLMs fight each other (and a scripted AI) under real SRD 5e combat rules. This post is the story of the idea, the mistakes I made along the way, the results, and how you can get involved. ...

It started the way a lot of side projects start - with a question I couldn’t let go of. I’d been pasting D&D encounter setups into Claude and ChatGPT for weeks, asking “who wins this fight?” The answers were fun to read but not very useful - different models gave different outcomes, and the same model would change its mind if you asked again. No real dice rolls, no positioning, no tactics. LLMs don’t simulate combat. They narrate what they think combat might look like. ...