Evidence · Limitations · Reproducibility

Chess ML Research and Model Cards

Public technical evidence for the chess machine-learning systems built by Uldis Briedis, with clear boundaries around what has—and has not—been validated.

Evidence status

Published now

Deployed artifact fingerprints, the tactics-classifier test report, ONNX export agreement, system boundaries, and known limitations.

Still required

Leakage audit, playing-strength matches, slice results by phase and rating, and repeated-run confidence intervals for the human-style bots.

Detector status

The fair-play score is a heuristic engine-likeness index. A public detector benchmark is planned; the current score is not proof of cheating.

What runs on the site

The named human-style bots use project-specific ONNX model artifacts in the browser. Separate files and tuning profiles create different practical behaviors. This fact alone does not prove playing strength or human-likeness; those require controlled evaluation.

Stockfish is a separate, optional analysis component. It is used where an engine reference is needed and is not presented as one of the custom human-style models.

Training code, raw datasets, training checkpoints, and the complete feature recipe remain private. Browser inference artifacts are necessarily downloadable by visitors. Public evidence can still be meaningful through immutable artifact hashes, declared test protocols, aggregate metrics, baselines, limitations, and versioned reports.

Tactics classifier V9: current test evidence

These are position-level tactical-classification results, not bot playing strength and not chess-cheating detection. They come from the saved V9 test report for the exact deployed ONNX artifact fingerprinted below. The public manifest records the test-file fingerprint, evaluation seed, report fingerprint, and limitations.

100,683

Test positions

93.14%

Precision

73.98%

Recall

4.45%

False-positive rate

85.86%

Accuracy

Confusion matrix: TP 33,476 · FP 2,464 · TN 52,969 · FN 11,774.

Export check: PyTorch-to-ONNX validation recorded maximum absolute differences of 2.30e-7 for the binary probability and 5.78e-6 for motif probabilities.

Limitation: stored train/validation/test files and a fixed seed are present, but the original split construction has not yet passed an independent leakage audit. Treat these as internal held-out-test results, not independently reproduced evidence.

Evaluation manifest (JSON)Full test report (JSON)ONNX export check (JSON)

Deployed artifact fingerprints

SHA-256 fingerprints identify the exact browser model used in a report without publishing the private recipe. A changed model produces a different fingerprint.

Tactics classifier V9

tactics_v9_react.onnx

Position-level tactical-signal classification used by training commentary.

SHA-256

764a3ae626bf2ee412fa7dcec5771274d30ecf07f0ce9701b9b21d759bd8fcdf

Beginner coach model V16

nnue_human_beginner_v16.onnx

Move evaluation for the beginner-oriented custom bot.

SHA-256

ad567cd2cd5200b89fd8d73a6b867a3de2df14a31774ab8559852fd24e48fadd

Endgame model V15

nnue_human_endgame_phase_v15_best.onnx

Move evaluation for the endgame-oriented custom bot.

SHA-256

e285b21e55e85cdc22a85027db47d51d50a343cf5d90033704644d5420d51b90

Defensive model V5

nnue_human_phase_v5.onnx

Move evaluation for a defensive custom-bot profile.

SHA-256

63ba4a069b468278fad7b6aa858b664d97f4cc3aab66785ad3ae0993d372b31c

Attacking model V6

nnue_human_phase_v6.onnx

Move evaluation for an attacking custom-bot profile.

SHA-256

be7f0c6c7b3cac47678e309658d618cb5ec7863ff946c41108c27a04383ee751

Practical model V8

nnue_human_phase_v8.onnx

Move evaluation for a practical custom-bot profile.

SHA-256

285f137aff1b2647b9596046df8ab53827c5c301c55a842f82eee28c79b341ad

Defensive model V18

nnue_human_super_defensive_v18.onnx

Move evaluation for a more defensive custom-bot profile.

SHA-256

ee0b4ff9c9aa384fd462d5ef9f9c2128c1a9f385f4f8621221e875b6135f0704

Fair-play analyzer validation plan

  1. Freeze and fingerprint the scoring formula before testing.
  2. Build documented human and controlled engine-assisted corpora with game-level split separation.
  3. Blind labels and report confusion matrices, precision, recall, false-positive rate, ROC-AUC, PR-AUC, and calibration.
  4. Publish slices by rating, time control, phase, game length, clock availability, and complexity with confidence intervals.
  5. Compare with simple baselines and release a non-sensitive evaluation manifest with the aggregate report.

Until that benchmark is complete, score bands are interface labels for the current formula, not validated probabilities. No single-game result should be used as an accusation.

Next public evidence release

The next release will focus on a leakage audit and a reproducible detector benchmark. Results will be published even when they expose weak slices or regressions.

Maintained by Uldis Briedis · Updated July 25, 2026