Reproducible benchmark results.

This page renders results from github.com/sovantica/engrava-benchmark, a public, MIT-licensed reproduction of standard agent-memory benchmarks. Numbers here are read from that repository at a pinned commit at build time — there is no live API and no client-side fetch on this page.

Scores are only comparable to each other within the same segment below — the same benchmark, dataset revision, split, harness, reader, judge, and scorer. Different segments are never merged into one ranked table, because changing any of those axes changes what the number means.

No language model runs inside the memory pipeline itself — retrieval is embeddings only, and the reader and judge that turn retrieved context into a graded answer sit outside it. Every verified row was produced by the public engrava package under the official scorer, and carries its own audit trail: hypotheses, judge labels and a checksummed artifact, committed in the repository alongside the row. That is what lets a third party check a score without re-running it.

Engrava 0.5.0 and 0.6.0 sit in separate segments because they ran on different harness commits, the only axis separating them. 0.6.0 is the lower of the two, by four graded questions out of five hundred. Judge verdicts carry non-zero error, and the benchmark’s methodology treats small gaps as noise even inside a single segment; these two rows are not in one. Neither an improvement nor a regression is demonstrated.