How do I compare two test runs by stable case ID to isolate new failures from known ones?
Each ID appears once. Skipped after-runs need a separate review and are not counted as passes.
Open Regression ComparatorTry the worked example
These are the example inputs from Regression Comparator. Replace them in the tool to run your own case.
| Test case | Before | After |
|---|---|---|
| Login | pass | fail |
| Export | fail | pass |
| Search | fail | fail |
| Archive | pass | skip |
Example result
1 new failure · 1 recovery
Who uses this?
QA coordinator. The free online tool runs the calculation in your browser. To edit the files and use the tools offline in one business, see the €49 offline edition.
Where the idea came from
Compare two test runs by stable case ID to isolate new failures from known ones. The demonstration requires actual execution before work can be marked complete. Benchcraft designed this tool’s inputs, decision rule and example for this office task.
Original video