← All updates

LPR A/B benchmark: run both engines, see which wins on your cameras

Set a camera to Both and every vehicle pass is read by both LPR engines at once. The desktop Benchmark screen puts them head to head so you can see which detected, which missed, and where they differ.

BackendDesktop lprbenchmarkdetection

I don't want to tell you which plate engine is better for your setup, because I can't. It depends on your camera angle, your lighting, your plates. So instead of picking for you, Crumb lets you run both at once and watch them compete.

Set a camera's Engine to Both and every vehicle pass gets read by both the Frigate native engine and the local crumb-alpr worker, each read tagged by its source. The desktop app grows a Benchmark screen, reachable from the Plates screen whenever at least one dual-engine camera exists.

What the screen shows

GET /lpr/ab-report clusters the raw reads into vehicle passes at report time (there's no stored "pass" entity), pairing a Frigate read and a worker read that fall within a short window (8 seconds by default). Then:

  • Two side-by-side stat cards, one per engine: reads, passes seen, hit rate,

average confidence, and accuracy.

  • A newest-first list of paired passes. Each row carries the pass's context

frame and the tight plate crop, both click-to-enlarge, plus both engines' plate and confidence and a match / differ / miss verdict.

So at a glance you see: on this pass, Frigate read 7ABC123 at 0.71, the worker read 7ABC123 at 0.94, verdict match. On the next one, Frigate missed entirely and the worker caught it. That's the wide-angle case that made me write the worker in the first place, and now you can prove it on your own footage instead of taking my word for it.

Confirm the true plate for real accuracy

A verdict of "differ" doesn't tell you which engine was right, so an admin can confirm the true plate for a pass. That confirmation (POST /lpr/ab-confirm) is stored normalized and keyed to the pass, and it's what anchors the accuracy numbers on the stat cards. The confirm prompt shows both images again so you can read the plate before you type it.

Every image is authenticated

The context frames and crops in the report ride an authenticated source, either the detection-event snapshot or the stored worker crop. Never an unauthenticated provider URL. Same rule as the rest of Crumb: media goes through scoped tokens.

The point

Run both for a week on the camera you care about, look at the miss column, and keep whichever engine (or both) actually earns its keep on that angle. Then you can drop the loser back to a single engine and reclaim the idle. See the LPR docs for the full report shape.

Don't miss the next one.

Watch the repo on GitHub for releases, or grab the RSS.

Get it on GitHub