Jack Tricker / Writing

Spire Network 2/5: Teaching a network to play the Ironclad, then letting it think ahead

Jack Tricker, 4 November 2026


Part 2 of 5. Part 1 covered the engine.

In this post a win means beating the Act 3 boss, or both bosses at ascension 20. The Ironclad networks never collected the keys, so they never fought the Heart.

The setup

The engine loads as a Python module that holds thousands of games at once and steps them in parallel. The network is a small transformer of about 1.07 million parameters. In a fight it sees each hand card with the damage and block the game would display, each monster with its intent, and the piles, relics and potions. Between fights it sees the deck and whatever is on offer: a card reward, a shop, the next rooms on the map. Every decision is one of 171 action slots with the illegal ones masked out. Training is PPO on one consumer graphics card at about 9,000 decisions a second.

At ascension 0 it learned quickly. A first network trained only on fights with random decks won 78% of them. Whole runs started from those weights reached 64% of A0 runs won in under four hours and 77% overnight.

Habits it picked up

Human runs

Mega Crit published a dump of run histories in 2020. From it I took 2.17 million usable Ironclad runs, 9.4% of them won, and 8.77 million decisions from the winners. The dump records choices between fights only: which card the player took, where they rested, what they bought.

Copying those choices didn't work by itself. A network trained to imitate them agreed with held-out human choices 60% of the time, and its win rate fell to 44.5%, because its fighting didn't suit the decks humans build. The dump was more use as practice material. Boss fights from 8,685 winning A20 runs, each set up with the human's deck and relics, became fights for the network to practise.

The climb at ascension 20

A20 stood at 7.6% when I turned to it. Four changes moved it, in this order:

  1. Small rewards for the habits of human winners, such as upgrading at rest sites and not drinking potions in easy fights: 10.8%.
  2. A reward of 3 for a win where it had been 1: 17 to 18%.
  3. Lowering the learning rate over the run: 19.2 to 19.5%.
  4. A longer horizon (gamma 0.9995): 22.0%.

That network is on the site as Ironclad v1. It wins 93% of runs at A0, 75% at A10 and 22% at A20.

Looking ahead

By then 68% of its A20 losses were boss fights, and it won 69% of the boss fights that humans had won with the same decks. The fights were the main limit.

The first fix needed no training. At each move, take the network's few likeliest moves and try each one on copies of the fight with the draw pile reshuffled, so it can't see what's coming. Let the network play each copy on to its next turn, and keep the move with the best average. With that, A20 went from 22% to 27.5% and the human boss fights from 69% to 76%, at about 60 times the computing per move.

An earlier attempt had lost badly, taking A0 from 88.9% to 70.1%. It scored each move by its best continuation, which favours whichever move has the most continuations, so it kept drinking potions and playing cards.

Next came two networks in place of one: a fighter, and a deck builder with an extra input for every option, the change in the fighter's predicted chance of beating the bosses ahead. With a new fighter trained on more styles of deck, the pair won 27.1% of A20 runs by themselves. With a look-ahead of two turns, the 8 likeliest moves and 32 copies, they won 49.5%.

That version isn't on the site. It needs about 1,200 passes through the network for each move, which in a browser would be about 7 seconds a move.

What didn't work

Teaching the look-ahead's moves back to the network failed every way I tried. An extra loss pulling the network towards the search's moves flattened its policy, and one version took A20 from 21.9% to 10.8%. Playing the search's moves during training was stable and gained nothing.

I also built a search tree for fights of the kind chess engines use, where positions reached by different move orders share a node. On 919 boss fights from human A20 runs:

Fights won
The network alone61.6%
Look-ahead, 2 turns70.4%
Tree, 2 turns72.8%
Look-ahead, 4 turns74.2%
Tree, 3 turns74.4%

About 7% of those fights turned out to come from runs that look cheated, almost all of them easy wins. Removing them lowers every row by 1 to 2 points and leaves the order as it is.

The tree beat the look-ahead at the same depth. The plain look-ahead one turn deeper tied it every time, in a bit over half the computing. How far ahead the search plays before it trusts the network's judgement mattered more than how many branches it tried.

Whole runs took most of that gain away. Six extra points on the Act 3 boss fights came to about one point in runs won: 52.7% against 51.4% on a different set of 512 seeds, which is within the noise.

Two bugs came out of this stretch. The search preferred not to finish fights, because the network valued a fight just won slightly below one still going. And with Blue Candle, one playout played Necronomicurse over and over, since it costs 1 HP and comes straight back, until I capped playouts at 30 moves.

Next: part 3, the other three characters and the Heart.

[ Return to top ] [ All writing ]


Jack Tricker / me@jacktricker.com
Last modified: 4 November 2026