Pikafish FAQ
Table of Contents
- Interpretation of the Pikafish evaluation
- Optimal settings
- The Elo rating of Pikafish
- Pikafish crashed
- Does Pikafish support xiangqi variants?
- Can Pikafish use my GPU?
- Executing Pikafish opens a CMD window
- What is depth?
The evaluation of a position that results from search has traditionally been measured in pawns or centipawns (1 pawn = 100 centipawns). A value of 1, implied a 1 pawn advantage. However, with engines being so strong, and the NNUE evaluation being much less tied to material value, a new scheme was needed. The new normalized evaluation is now linked to the probability of winning, with a 1.0 pawn advantage being a 0.5 (that is 50%) win probability. An evaluation of 0.0 means equal chances for a win or a loss, but also nearly 100% chance of a draw.
Some GUIs will be able to show the win/draw/loss probabilities directly when the UCI_ShowWDL engine option is set to True.
The full plots of win, loss, and draw probability are given below. From these probabilities, one can also obtain the expected match score.
| Probabilities | Expected match score |
|---|---|
 |
 |
The probability of winning or drawing a game, of course, depends on the opponent and the time control. With bullet games, the draw rate will be lower, and against a weak opponent, even a negative evaluation could result in a win. These graphs have been generated from a model derived from Fishtest data for Pikafish playing against Stockfish (so an equally strong opponent), at 60+0.6s per game. The curves are expected to evolve, i.e. as the engines get stronger, an evaluation of 0.0 will approach the 100% draw limit. These curves are for Pikafish 2023-03-05.
To get the best possible evaluation or the strongest move for a given position, the key is to let Pikafish analyze long enough, using a recent release (or development version), properly selected for the CPU architecture.
The following settings are important as well:
Set it to the maximum - (1 or 2 threads).
Set the number of threads to the maximum available, possibly leaving 1 or 2 threads free for other tasks. SMT or Hyper-threading is beneficial, so normally the number of threads available is twice the number of cores available. Consumer hardware typically has at least 4-8 threads. Pikafish supports hundreds of threads.
More detailed results on the efficiency of threading are available.
Set it to the maximum - (1 or 2 GiB RAM).
Set the hash to nearly the maximum amount of memory (RAM) available, leaving some memory free for other tasks. The Hash can be any value, not just powers of two. The value is specified in MiB, and typical consumer hardware will have GiB of RAM. For a system with 8GiB of RAM one could use 6000 as a reasonable value for the Hash.
More detailed results on the cost of too little hash are available.
Set it to 1.
A value higher than 1 weakens the quality of the best move computed, as resources are used to compute other moves.
More detailed results on the cost of MultiPV are available.
"What is the Elo of Pikafish?": A seemingly simple question, with no easy answer. First, the obvious: it is higher than any human Elo, and when PF 2023-03-05 ranked with more than 3678 Elo on some rating lists, Bilibili knew. To answer the question in more detail, some background information is needed.
In its simplest form, the Elo rating system predicts the score of a match between two players, and conversely, a match between two players gives information about the Elo difference between them.
The Elo difference depends on the conditions of the match. For human players, time control (blitz vs. classical TC) or variant (standard xiangqi vs. Jieqi) are well-known factors that influence the Elo difference between two players.
Finally, given an Elo difference between two players, one needs to know the Elo rating of one of them to know the Elo rating of the other. More generally, you need an anchor or reference within a group of opponents, and if that reference is different in different groups, the Elo value cannot be compared.
The same observations apply to the calculation of Pikafish's Elo rating, with the caveat that top engines play at an extremely high level and, from the starting position or any balanced opening, have achieved a draw rate approaching 100% against other engines of similar strength, especially at rapid or longer time controls, and even more so on powerful hardware. This results in small Elo differences between top engines, e.g. +19 -2 =79 is a convincing win, but a small Elo difference.
Carefully constructed books of starting positions with a clear advantage for one side can significantly reduce this draw rate and increase Elo differences. The book used in the match is therefore an important factor in the computed Elo difference.
Similarly, the pool of opponents and their rankings has a large impact on the Elo rating, and Elo ratings computed with different pools of opponents are difficult to compare, especially if weaker (but different) engines are part of that pool.
Finally, to accurately compute Elo differences at this level, a very large number of games (typically tens of thousands of games) are needed, as small samples of games (regardless of time control) will lead to large relative errors.
Having introduced all these caveats, accurately measuring Elo differences is central to the development of Pikafish, and our Fishtest framework constantly measures with great precision the Elo difference of Pikafish and its proposed improvements. These performance improvements are accurately tracked over time on the regression testing wiki page. The same page also links to various external websites that rank Pikafish against a wide range of other engines.
Rating Pikafish on a human scale (e.g. FIDE Elo) has become an almost impossible task, as the difference in strength between it and humans is now so large that this difference can hardly be measured. This would require a human to play Stockfish long enough to have at least a handful of draws and wins.
Pikafish may crash if fed incorrect fens, or fens with illegal positions. Full validation code is complex to write, and within the UCI protocol, there is no established mechanism to communicate such an error back to the GUI. Therefore Pikafish is written with the expectation that the input fen is correct.
On the other hand, the GUI must carefully check fens. If you find a GUI through which you can crash Pikafish or any other engine, then by all means report it to that GUI's developers.
The official Pikafish engine only supports standard xiangqi. However, various forks based on Stockfish support variants, most notably the Fairy-Stockfish project.
No, Pikafish is a chess engine that uses the CPU only for chess evaluation. Its NNUE evaluation (see this in-depth description) is very effective on CPUs. With extremely short inference times (sub-micro-second), this network can not be efficiently evaluated on GPUs, in particular with the alpha-beta search that Pikafish employs. However, for training networks, Pikafish employs GPUs with effective code that is part of the NNUE pytorch trainer. Other chess engines require GPUs for effective evaluation, as they are based on large convolutional or transformer networks, and use a search algorithm that allows for batching evaluations. See also the Pika Xiangqi Zero (Px0) project.
Pikafish is a command line program, when you execute it, you might notice that it simply opens a Command Prompt (CMD) window. This behavior is intentional and serves as the interface for interacting with the engine.
If you prefer a more user-friendly experience with a xiangqi board and additional features, you can consider using a graphical user interface (GUI) alongside Stockfish. To set up a GUI, you can visit the Download and Usage page.
The CMD window allows you to input various commands and receive corresponding outputs from Pikafish. If you want to explore the available commands and their explanations, you can refer to the Commands page but this is only recommended for advanced users and developers.
First, we need to understand how minimax search works. We will go with the vanilla one because explaining what Alpha-beta is does not do much.
Each player tries to maximize the score in their favor. White wants the evaluation to be as positive as it can, and Black as negative as it can - we do this all the time when we play chess. Search works in a similar way - you explore your moves, explore the opponent's replies, assign a value called evaluation to each resulting board position (which is not precise but tries to be), and find a sequence where White plays some move that has the maximum evaluation for the best opponent's reply.
Then you search one ply (half-move) deeper - exploring your reply to the last opponent's replies. This process is called iterative deepening - you explore a position up to depth 1, then to depth 2, then to depth 3, and so on - you deepen your search with each iteration and this is why it is called this way.
So, for now, "depth" is a perfect thing - it means you fully calculated the search tree up to this "depth" and you know everything that can happen within it. For a mate in 5, you will need depth 9 to see it (because depth is written in half-moves). But chess has a lot of possible moves, 20 from the starting position and usually many more from any middlegame position. Even if you can evaluate millions of positions per second as engines like Stockfish do, you will still hit a wall in what depths you can realistically reach, and it would not be that high - depth 8, maybe 10.
How to combat this? With a thing called pruning. Pruning splits into quite a lot of different heuristics, but they mostly serve one purpose - remove branches in search that do not look too desirable, to re-explore them later when iterative deepening depth goes higher. So this is where "depth" starts to mean less - because you do not search the entire game tree and modern engines prune large percentages of branches.
But then there is also a thing called "extensions" which is more or less the opposite of pruning. With extensions, you start by searching "important" branches deeper (for example, checks) than what is needed to complete the iterative deepening iteration.
With all of this, instead of a search tree that is strictly cut off at this "depth", you have most of the branches ending really early and a lot of branches searched deeper than the given "depth". Pikafish is the most aggressive engine in both pruning and extensions, so its search tree looks nothing like what you usually see on Wikipedia.
Coming back to how much Pikafish prunes, there is some data here. The branching factor indicates how many moves you calculate on average per depth increase, calculated as