How Strong a Shogi Engine
Can Codex Build for the Game Boy?

The Assignment Given to Codex

At the start of development, the repository contained only a README, a license, and Git ignore settings. There was not a single line of shogi source code, and the README defined only these boundaries:

• Run on a Game Boy and fit the ROM in 32KB.
• Let a human play the AI and choose to move first or second.
• Offer CPU thinking times of 5 and 60 seconds.
• Provide a UI that a shogi player can understand without learning custom symbols.
• First make a working program, then add measurements and improve its strength quantitatively.

The implementation language, board representation, search method, and development tools were unspecified. Codex began with a C11 host engine, then expanded the work to SM83 code, tests, emulator control, and match runners. The first implementation commit added 17 files and about 1,700 lines; many small measurement and design commits followed.

Saying only that Codex wrote most of the code misses the division of responsibility. The human set the objective and acceptance criteria; Codex designed, implemented, and verified within that frame. Usability and playing strength in particular left questions that the program alone could not settle.

The human ran the ROM and reported concrete failures: pressing Down alone started a match, parts of the cursor remained on screen, and the display flashed white. Codex reproduced these reports in emulators, isolated their causes, and turned them into regression tests that included long key presses. For strength testing, the human chose meaningful opponents and stopped experiments when the comparison no longer answered the intended question.

This division is best understood not by counting lines of code, but by asking who framed the questions and who decided which results were acceptable.

Making Shogi Work on a PC

Before writing Game Boy code, Codex built a host engine from the same C sources. Investigating rule discrepancies only in a slow emulator would make every position expensive to debug. On a PC, unit tests run quickly and legal move sets can be compared with an external engine.

A position contains an 81-square array, seven types of pieces in hand for each side, the side to move, both king locations, the ply count, and repetition history. Board pieces use signed bytes: positive values belong to Black, negative values to White, and both share the same piece numbers. Only the pre-move differences are saved in Undo; search repeatedly makes and unmakes moves while reusing one position object.

A move fits in 16 bits. The source and destination use seven bits each, and one remaining bit marks promotion. Source values 81 through 87 represent drops of the seven piece types.

/* bits 0..6: destination, 7..13: source, 14: promotion */
typedef uint16_t Move;
#define MOVE_TO(m)       ((uint8_t)((m) & 127u))
#define MOVE_FROM(m)     ((uint8_t)(((m) >> 7) & 127u))
#define MOVE_PROMOTES(m) (((m) & 0x4000u) != 0)

The legal move generator follows the movement directions for each piece and emits pseudo-legal moves one at a time. Where promotion is optional, it emits promoted and unpromoted variants; pawns and lances entering the last rank, and knights entering the last two ranks, retain only mandatory promotion. Drops exclude dead-end squares and nifu.

Each candidate is then played, and any move that leaves the moving side's king in check is discarded. Only when a pawn drop checks the opposing king does the engine recursively test whether the opponent has no legal reply; if so, the drop is rejected as illegal pawn-drop mate. This order separates movement generation from rules that depend on the whole position.

Ordinary Search Does Not Fit in 8KB of RAM

Running on a PC and running on a Game Boy in practical time were different problems. The Game Boy has only 8KB of work RAM. The first implementation allowed as many as 600 legal moves, so a 16-bit move array alone consumed 1,200 bytes. Placing such an array at every recursive depth was impossible.

The engine therefore stopped storing all legal moves and kept only a small generator state that yields one move at a time. Search scans the generator in phases: the transposition-table move, captures and promotions, then other moves. Finding a category can require several board scans, but no recursive level consumes a 1,200-byte move list.

The speed limit was lower than expected as well. With the same four-ply quiescence search as the PC engine, the first measurement visited only 13 nodes in 5 seconds and did not complete depth 1. An early fixed budget of 12,000 nodes still had not finished after 20 seconds. The fixed-node design was replaced with a hardware timer that stops search after 5 or 60 seconds.

static const int piece_value[] = {
    0,100,300,320,450,520,650,750,20000,
    520,520,520,520,850,950
};

if (base == PAWN) v += advancement * SHOGI_PAWN_ADVANCE;
else if (base == LANCE) v += advancement * SHOGI_LANCE_ADVANCE;
else if (base == KNIGHT || base == SILVER)
    v += advancement * SHOGI_MINOR_ADVANCE;

uint8_t to = ray_next[square][direction];
int8_t target = p->board[to];
if (!target || owner_of(target) != side) {
    if (type == BISHOP || type == HORSE) {
        int f = square_file[to];
        int edge_distance = f > 4 ? f - 4 : 4 - f;
        mobility += 5 - edge_distance;
    } else ++mobility;
}

if (base == KING) for (int d = 0; d < 8; ++d) {
    uint8_t guard_square = ray_next[i][d];
    if (guard_square >= 81) continue;
    int8_t guard = p->board[guard_square];
    if (king_guard_owner[guard + 14] == side)
        v += SHOGI_LOW_KING_GUARD;
}

if (!checked) {
    stand = search_evaluate(p, s);
    if (stand >= beta) return beta;
    if (stand > alpha) alpha = stand;
    if (!qdepth) return alpha;
}
while (checked ? next_legal_do(p, &g, &m, &u)
               : next_tactical_legal_do(p, &g, &m, &u)) {
    int score = -quiesce(p, -beta, -alpha,
                         qdepth - (qdepth != 0), s);
    /* update alpha and beta */
}

for (int phase = tt_move ? 0 : 1; phase < 3; ++phase) {
    MoveGen g; movegen_init(&g);
    while (phase == 1 ? next_tactical_legal_do(p, &g, &m, &u)
                      : next_legal_do(p, &g, &m, &u)) {
        /* phase 0: TT move, 1: captures/promotions, 2: quiet moves */
    }
}

if (qdepth > 1 && phase == 2 && depth <= 2 &&
    !checked && move_count >= 11) {
    position_undo(p, m, &u);
    goto moves_done;
}

score = -negamax(p, child_depth, -alpha - 1, -alpha, s);
if (!s->stopped && score > alpha && score < beta)
    score = -negamax(p, child_depth, -beta, -alpha, s);

if (!s.stopped) {
    result->best = best;
    result->completed_depth = depth;
} else if (best != principal &&
           best_score >= principal_score + partial_margin) {
    result->best = best;
}

Faster Alone Is Not Enough

On an 8-bit CPU, a change that visits more nodes in the same time looks attractive. But node count measures speed, not move quality. Changing order or pruning can find important moves with fewer nodes, or race faster down bad branches.

An attempt to double the transposition table to 1,024 entries illustrates the distinction. Compressing entries from eight to seven bytes solved fixed mandatory-promotion and mate-in-one tests in fewer nodes. Yet at the current depth-3 budget it performed significantly worse than the 512-entry version on standard positions, Floodgate positions, and the 16-position suite. Because the tactical improvement did not transfer to overall play, the engine returned to 512 entries in 4,096 bytes.

A small machine-learned piece-square table failed similarly. On separate training and validation Floodgate games, the raw model raised winner-prediction accuracy from 52.02% to 61.77%. Quantizing it for ROM reduced accuracy to 53.90%, increased the nodes needed for fixed tactics, and slowed the Game Boy. Both time settings also lost their 32-game pilots, so predictive accuracy was not treated as playing-strength improvement.

Root PVS won narrowly when both versions received the same 734 nodes. On the Game Boy, however, the fixed cost of re-search after narrow-window failures did not appear in that node count and reduced the work completed in 60 seconds by about 7.5%. The wall-clock comparison scored 158 wins, 113 draws, and 241 losses, while the ROM grew by 263 bytes, so root PVS was rejected.

Tests of roughly 32 games were used only to eliminate clearly bad candidates. When a small difference remained, 512 distinct starting positions extracted from Floodgate games were color-swapped for 1,024 games. Reports included a Wilson 95% confidence interval and a two-sided binomial-test value, and each color-swapped pair shared the same starting position and random seed.

Statistics still did not decide the design automatically. Selecting only the best among many candidates can overfit the same position set. Some evaluation coefficients were rejected when gains failed to reproduce on a separate set of 512 positions never used for selection. Distinguishing what speed, fixed tactics, matches, and holdout positions measured became a condition for spending the next bytes of ROM.

Repairing the Opening with a 3KB Book

Even after search was simplified, five seconds barely compared all initial moves at depth 1. The evaluation function knew little about coherent piece development, so it often left the king unmoved and merely brought golds and silvers closer. The first few moves were therefore supplemented with an opening book before search began.

The source was the public 2026 CSA game archive from the Floodgate online computer-shogi server. From 72,127 games, the project selected 15,279 in which both players were rated at least 3,000, at least 40 plies were played, and the game ended by resignation or entering-king declaration. It aggregated the first 24 plies, 366,696 moves total, weighting a winner's move by 3 and a loser's by 1.

The final build stores 312 positions and 512 candidates. Each six-byte candidate contains a 16-bit position key, 16-bit move, weight, and ply, so the data occupies 3,072 bytes. Each position keeps at most two choices and samples them by weight instead of fixing one move.

Randomness uses an 8-bit xorshift. It requires neither multiplication nor large state, and real hardware can seed it from the timer and scanline position. Providing the same seed reproduces the same branch, so color-swapped test pairs use identical seeds.

A 16-bit position key can collide across different positions. A book move is therefore never played unchecked. The current legal moves are generated first, the selected move must exist, and another candidate with the same key is tested after a collision.

The book was not intended to lock the engine into one opening. After the first move, branches open the bishop diagonal, advance the rook pawn, push central or edge pawns, and develop the king or golds. The final first move is nevertheless strongly biased: P-7f has probability 255/256 and P-2f probability 1/256. Shallow search handled positions after leaving a double-wing-attack line poorly, while biasing toward P-7f was more robust in pilot games.

This bias had a cost. After the same book move, both opponent and Game Boy often followed the same deterministic continuation. Repeated identical records in the final matches could no longer be counted simply as “twenty trials.”

Running in an Emulator Is Not Completion

Automated rule and search tests did not catch every defect seen by a person holding a key. The clearest example was the Windows BGB emulator starting a match when Down alone was pressed immediately after startup.

Codex first scanned 724 combinations of press start time and hold length in headless PyBoy. None started a match. The mGBA debugger likewise read Down as Down, without START.

Failure to reproduce did not prove the input code correct. Using the BGB configuration and exact input conditions supplied by the human, Codex ran BGB 1.6.6 under Wine. Holding Down for 0.5 seconds advanced the pre-fix ROM to the board.

The cause was the Game Boy JOYP input register. Eight keys share four bits arranged in two rows; bit 3 means Down in the direction row and START in the button row. The old code read once immediately after switching rows, while the prior row's Down value still remained and was interpreted as START.

Discarding one read after each switch fixed Down but left Right occasionally turning into A and making an unintended move. The final code discards four reads after selecting a row and accepts only the fifth. Long-press tests cover every direction/button pair sharing a bit, not only Down and START.

Automated tests were adjusted to human hold times. Alongside one-frame taps, they verify that 10-, 30-, and 120-frame holds produce exactly one action. A separate defect mistook a transient zero during release for a completed release and accepted one physical press twice; the engine now waits for two consecutive VBlanks with no input before accepting another action.

The one-frame white flash occurred because the LCD was stopped for every cursor move. Coordinate calculation and tile selection were moved into a work-RAM staging step, leaving only a few predetermined VRAM writes during VBlank. Recorded video was checked for a single white or half-updated frame, not merely for a correct final screenshot.

Making the Game Boy ROM Play Matches

Strength could not be measured by substituting the lightweight PC build. Even with identical C sources, SM83 move-generation time, timer interrupts, and display load change which move survives a timeout.

A dedicated evaluation ROM therefore ran under mGBA. It exposes a mailbox in work RAM through which external code writes positions and commands using the mGBA debugger. The PC side presents this control path as a USI engine and connects it to a normal match runner.

match runner
    ↓ USI
mGBA debugger
    ↓ WRAM mailbox
evaluation Game Boy ROM
    ↓ bestmove
cshogi legality and termination checks

The normal and evaluation ROMs link the search core from the same object file. Every move returned by the ROM is applied to a cshogi position, checking legality, side to move, mate, and termination reason. This makes repeated games use the actual instruction speed of the Game Boy search.

Merely creating an evaluation ROM did not reproduce normal timing. The first version stopped the LCD and searched 90 nodes in five seconds, while the normal ROM searched 88. VBlank interrupts and sprite DMA consume CPU time in the displayed game.

Running the LCD and the same VBlank/DMA load in the evaluation ROM aligned the initial-position results at 88 nodes for 5 seconds and 1,061 for 60. Records produced before that correction were excluded. Even a two-node advantage from hiding the screen matters when the entire five-second search visits only 88 nodes.

Playing YaneuraOu at Its Minimum Setting

Opponents included random, which chooses uniformly from legal moves; MaterialLv1, the pure-material evaluation build of YaneuraOu; and Suisho 5. The original plan was to adjust each opponent's node budget until it played evenly with the Game Boy.

The gap remained large even at the smallest command accepted by YaneuraOu and Suisho 5, go nodes 1. In a 32-game color-swapped pilot over 16 positions, the 5-second build scored 1 win, 1 draw, and 30 losses against Suisho 5, and 1 win and 31 losses against MaterialLv1. The 60-second build scored 1 win and 31 losses against Suisho 5, and 12 wins, 1 draw, and 19 losses against MaterialLv1. Every condition was below 50%.

The equal-strength node budget lay below the positive integer range available to the opponents. Further Suisho 5 games, holdout validation, and precise estimation of an equal point stopped there.

Instead, twenty games from the standard initial position were saved for each condition. They are illustrative records, not estimates of a general win rate.

All twenty random games at each time setting had distinct move sequences. The MaterialLv1 games at each setting had effectively only three sequences, repeated 10, 7, and 3 times after both deterministic engines followed the same continuation from a book branch.

The three wins at 60 seconds must not be read as three successes in twenty independent trials. All three replayed the same record: one winning line repeated three times. The table describes saved files, but the sample cannot support a population win rate with a confidence interval.

The Misleading Phrase “It Lost to Material Alone”

The name MaterialLv1 may suggest a simple one-ply opponent. But Lv1 describes only the static evaluation function, not the whole searcher. “Material” means it returns the difference in piece values without positional, king-safety, or attack-map terms.

The searcher using that score is normal YaneuraOu. It enumerates every root move, uses iterative-deepening alpha-beta, and recursively searches captures, promotions, checks, and evasions in quiescence. Mate-in-one detection, a transposition table, SEE filtering of losing exchanges, and capture ordering remain enabled. SEE (static exchange evaluation) cheaply estimates the material outcome of repeated captures on one square.

Furthermore, go nodes 1 did not mean “make one move and stop immediately.” YaneuraOu v7.6.3 enters quiescence before checking the node limit at depth zero, and does not check that limit inside quiescence. Its first depth-1 iteration therefore completes every root move and each move's quiescence before stopping.

In saved standard-position games, MaterialLv1 reported an average of 86.0 nodes per move against the 5-second build and 104.2 against the 60-second build. Despite a requested budget of one, it processed as many as 319 nodes and completed depth 1 on every move.

The Game Boy checks timeout at every normal and quiescence node. Among 715 non-book moves at 5 seconds, 359 stopped at depth 0 without comparing all legal root moves at one depth. Similar totals near 90 nodes therefore represented different amounts of completed work.

The Game Boy evaluation has more terms than pure material: advancement, approximate rook/bishop mobility, and nearby king defenders. More terms do not guarantee greater accuracy. Piece-value ratios, coarse features, incomplete search, and short quiescence can combine to select worse moves than a pure-material opponent.

This was not a comparison in which “the positional Game Boy lost to one-ply material.” It compared a modern searcher that completes every candidate and tactical exchange even at its minimum setting with a search aggressively shortened for an 8-bit CPU and interrupted by time. Comparing only the evaluation functions would require a separate experiment that swaps them into the same searcher.

Strengths and Weaknesses Visible in the Games

The saved records cannot estimate win rates, but they do show which moves the engine chose. The following examples take one game each against random and MaterialLv1.

The diagrams were produced by replaying the saved records in ShogiHome.

5 Seconds vs. random (Game Boy as Black)

At five seconds, the engine leaves its king on the starting square and brings golds and silvers nearby. This is not natural development after leaving the book; it exposes the coarseness of measuring king safety only by adjacent defenders.

In the endgame it nevertheless brought a rook, dragon, horse, and gold around the opposing king and delivered mate. Random has no positional objective and chooses checks and material gains with the same probability as any legal move. The Game Boy's castle is strange, but material, advancement, and search for checks still give its play direction.

One game cannot establish a win percentage against random. Still, the 40 saved games with distinct sequences produced 37 wins, showing play more purposeful than uniform legal-move selection.

60 Seconds vs. MaterialLv1 (Game Boy as White)

At sixty seconds, the engine still tends to leave the king in place and gather golds and silvers. Deeper search does not change the king formation preferred by the evaluation function itself.

In the illustrated win, the Game Boy gave the opponent many pieces but eventually mated with a dragon. MaterialLv1's static evaluation does not count king safety, but its normal searcher's mate-in-one logic remains active. This game is an example of a legal mate, not a win caused by an accidental illegal move.

All three 60-second wins used this same move sequence. The engine did not discover three different ways to win; it replayed one line from the same book branch.

How Old Were the Techniques Put into the 1989 Hardware?

The Game Boy launched in Japan on April 21, 1989.³ Calling the project a port of modern shogi AI to an old console may suggest that its search methods were invented after the hardware. Publication dates tell a different story: the skeleton of this ROM's search predates the Game Boy.

What Codex Could Do and What the Human Decided

Codex built not only the shogi engine and ROM, but the machinery for testing correctness and strength. It checked legal generation with perft and an external engine, controlled emulators through debuggers, and transformed Floodgate records into a book and starting positions. It also left an evaluation ROM, USI bridge, parallel match runner, statistical summaries, reproduction steps, and records of failed candidates.

Large candidate comparisons were especially suitable for a coding agent. It could change one constant or implementation, then run rule tests, perft, fixed tactics, hardware-node measurements, and matches in sequence, reverting poor candidates. Because rejected ideas were documented, a later experiment under different conditions could compare against their original assumptions.

The human, however, first noticed what felt wrong during actual use. Neither the mistaken Down input nor cursor trails appeared in final-screen comparisons alone. After the human supplied screenshots and exact operations, Codex converted them into reproducible tests.

The human also decided what a strength comparison meant. They selected opponents that would make the result understandable and stopped precise measurement once even the minimum setting was far stronger. Adding 108 games completed during selection would have increased the number, but keeping a selection-stage sample out of the formal result was another human decision.

“How strong is it?” is not answered automatically by adding games. The analysis had to avoid counting dependent records, inspect what the opponent's nodes 1 actually executed, and limit the conclusion to what the measurement supported.

Impressive for a Game Boy, Far Too Weak as a Shogi Engine

The completed ROM fits in 32KB and plays shogi on a Game Boy with promotion, drops, nifu, self-check, and illegal pawn-drop mate. It searches 88 nodes in five seconds and 1,061 nodes in sixty, and won 37 of the 40 saved random games.

Even after thinking for sixty seconds, however, it remained far below MaterialLv1, a modern normal searcher using pure material evaluation. It won only 3 of 40 saved games, and those three were replays of the same record. Because it was weaker than the minimum-node settings of both Suisho 5 and MaterialLv1, the original plan to estimate an equal-strength point could not be carried out.

Rather than ending the diagnosis at “because it is a Game Boy,” the project measured the missing search coverage. At five seconds, roughly 90 nodes still fail to complete depth 1 on half of non-book moves. MaterialLv1, by contrast, completes every root move and recursive quiescence before stopping even when asked for one node. That difference in stopping behavior and completed search could not be closed by adding a few evaluation terms.

Codex autonomously handled design, implementation, tests, emulator automation, and large match runs. The human still had to notice display and input anomalies, question what comparisons meant, and decide when to stop an experiment.

It worked hard by Game Boy standards, but it is far too weak as a shogi engine. Even so, the project measured and explained not only how much could be built, but where the same ROM stopped being competitive.