Writ by Infinri
Writ retrieval, measured
Maintainer-run results from October 7, 2026. A test of rule ranking and abstention, with the misses included.
What was tested?
The isolated test graph was queried with a maintainer-authored set of 193 queries. Of those, 169 target rules the ranked channel can return; 21 target always-on rules and 3 target routed rules. The 47 ambiguous queries are a subset, not an additional sample. Twenty separate negative queries should return no ranked rules.
The table compares the shipped abstention threshold of 0.30 with the same retrieval setup without that gate. It is not a comparison with another product.
| Metric | Gate at 0.30 | No abstention gate |
|---|---|---|
| Expected rule in top 5 (169 queries) | 0.929 [0.888, 0.965] | 0.941 [0.905, 0.970] |
| MRR@5, ambiguous subset (47) | 0.603 [0.486, 0.716] | 0.635 [0.521, 0.743] |
| nDCG@10 (169 queries) | 0.832 [0.789, 0.873] | 0.844 [0.803, 0.881] |
| False injection (20 negatives) | 4 of 20 0.20 [0.08, 0.42] | 20 of 20 1.00 [0.84, 1.00] |
How should I read the numbers?
Hit@5 asks whether the expected rule appears in the first five results. MRR@5 rewards placing it earlier. nDCG@10 describes ranking quality across the first ten results. False injection counts a negative query that still receives a rule.
Abstention reduced false injection on this set while also missing some positive targets. Four of twenty negative queries still received rules. Twenty negatives is a small sample and its interval is wide.
Method and limits
The report uses 10,000 percentile bootstrap resamples with seed 0 for ranking intervals, and Wilson intervals for false injection. The same project authored the corpus and the queries. Each gold query names one expected rule, so another useful result can still score as a miss.
These are published maintainer measurements, not an independent replication or a fresh run for this website. They do not measure code quality, rule compliance, agent behavior or security containment. No controlled evidence here establishes that Writ makes an agent ship better code.
Inspect the source
Sources are pinned to Writ commit 8cd4f81c7a22. The October 7 report's before/after columns test a separate refactor and are equal; the table above uses its gated and ungated arms.