OMBench results
Each mode, measured. And measured again when its setup changes.
How many known bugs each OhMyBug mode finds, against how long a review takes. Every point is a stored run of OMBench, our held-out benchmark, with its date; when a mode's setup changes, its earlier points stay on the chart.
The full held-out set
All four modes on the full held-out set, as the modes page prints them today, and the figures it printed before, and an estimate.
what the site prints todayindicativeestimatethe same mode over time
| Mode | Price | Found, of the known bugs | Hard cases found | Precision | Findings per change: medium+ / all | Server time: typical / 9 of 10 | Client wait: typical / 9 of 10 |
|---|---|---|---|---|---|---|---|
| lite | $1 | 61.5% | 52.5% | 79.9% | 30 / 49 | 85 s / 2.1 min | 2.1 min / 2.6 min |
| snap | $2 | 57.7% | 47.5% | 82.7% | 27 / 45 | 34 s / 49 s | 56 s / 83 s |
| fast | $4 | 76.5% | 77.5% | 82.5% | 35 / 61 | 1.9 min / 5.4 min | 2.3 min / 6.3 min |
| deep | $12 / $24 | 86.1% | 77.5% | 85.0% | 40 / 80 | 4.6 min / 10.0 min | 4.9 min / 10.7 min |
Lite, snap and fast come from later runs. Hard cases: 40 bugs that earlier reviews missed, in 23 of the changes.
| Mode | Measured | Setup | Found, of the known bugs | Precision | Findings per change: medium+ / all | Server time: typical / 9 of 10 | Client wait: typical / 9 of 10 | Price today |
|---|---|---|---|---|---|---|---|---|
| lite | Nov 16, 2026 | earlier setupindicative: measured on an older server version | 53.4% | 84.1% | 15 / 24 | 84 s / 2.1 min | 2.1 min / 2.6 min | $1 |
| lite | Nov 15, 2026 | earlier setupindicative: measured on an older server version; scored with another judge setup; measured on fewer of the changes | 56.6% | 82.0% | 16 / 25 | 1.6 min / 5.7 min | 2.0 min / 6.1 min | $1 |
| lite | Nov 12, 2026 | earlier setupindicative: measured on an older server version; scored with another judge setup | 58.0% | 81.8% | 18 / 29 | 2.3 min / 5.6 min | 2.6 min / 6.0 min | $1 |
| snap | Nov 12, 2026 | earlier setupindicative: measured on an older server version; scored with another judge setup | 46.4% | 84.8% | 11 / 19 | 31 s / 40 s | 45 s / 58 s | $2 |
| fast | Nov 12, 2026 | earlier setupindicative: measured on an older server version; scored with another judge setup | 72.4% | 85.5% | 18 / 34 | 2.0 min / 6.6 min | 2.3 min / 7.0 min | $4 |
| deep | Nov 17, 2026 | estimate | 87.5% | 79.8% | 56 / 104 | 4.6 min / 10.0 min | 4.9 min / 10.7 min | $12 |
How to read it
- Found. The share of the known bugs a review found, all bugs of the set pooled. Known bugs are checked against the code before the run, in changes never used to tune any mode; a bug counts as found only when a finding points to the same defect at the same place. Nobody knows all the bugs in the code, so this is a share of the known ones.
- Precision. The share of a review's findings that point to a real bug, averaged over the changes.
- Findings per change. Your agent reads them all, checks each against the code and drops the false ones. The medium-or-worse count is what a fix starts from.
- Server time. The chart's time, and the same measure as the modes page: from the moment the review starts on our side to the result. Typical is the median change; 9 of 10 is the time 9 changes in 10 stay under.
- Client wait. What the person or agent who asked actually waits: from the start of sending the change to the answer. It adds packing and sending the change to the server time.
- Earlier setup. One a mode ran before; each mode's solid point is the figure the site prints today.
- Indicative. A point that shows the direction, not a like-for-like comparison with the solid point; the table says why.
- Hard cases. Bugs that earlier reviews missed, a small sample.
- Price. The price of one review in each mode today. When a review is charged is in the FAQ.
How the benchmark is built, verified and scored: OMBench. Which mode to run when: the modes page.