Slay the Spire 2

Slay the Spire 2 learning pipeline

Teaching a combat agent to play the real game—and checking what it is learning.

The live pipeline connects Slay the Spire 2 to Python through a C# bridge. It reads the combat state, chooses from legal actions and records what happened next. Those replays become the material for learning.

Watch the agent play

The Regent playing through the C# bridge, with Python choosing legal combat actions and recording the results.

A comparison between the incorrect 40 HP reward and the corrected 1 HP reward — 01 The reward-accounting bug: a 1-HP kill was credited as 40 HP of damage.

The interesting part quickly became the plumbing around the model: making sure a move has finished before choosing another one, keeping worker saves separate, and being able to resume an experiment without quietly changing it.

The slime worth forty HP

One replay gave 40 HP of damage credit for killing a slime with 1 HP left. Dead enemies had disappeared from a compacted target list, so comparing enemies by position mixed up their identities.

Fixing that accounting changed 14,960 of 164,118 reward labels. It is a good example of why a learning curve can look convincing while the thing being rewarded is wrong.

Candidate scoring time falls from 38.86 milliseconds to 30.69 milliseconds — 02 The same scoring calculation, from 38.86 ms to 30.69 ms with matching outputs.

Make the repeated work cheaper

Cached replay encoding and shared-state scoring cut a controlled scoring benchmark from 38.86 ms to 30.69 ms, with matching answers. That is roughly 21% less time spent on the same choices.

Checkpoints also keep optimiser state, replay priorities and sampling state, so restarting does not throw away the machinery behind the weights.

Compare on the same worlds

Frozen agents are compared on matched seeds, with the initial worlds checked before the results are combined. The 16-seed pilot did not show a reliable gameplay gain, so the candidate stayed separate. The faster pipeline makes the next comparison easier to run properly.

Under the hood

A live-game bridge, isolated worker profiles, legal candidate actions and retained replay transitions.

A reward audit that corrects phantom damage when enemy indices change.

Persistent optimizer, target-network, replay-priority and sampler state across incremental updates.

Cached replay encoding, shared-state candidate scoring and bounded worker cleanup.

Matched-seed evaluation with recorded world fingerprints, uncertainty intervals and protected manual saves.

Correcting enemy identities changed 14,960 of 164,118 reward labels. Reusing shared calculations then reduced scoring from 38.86 ms to 30.69 ms without changing the answers. Warm update batches took roughly 12–14 seconds.