Verification & validation
Trust is earned in the open
A model that governs public infrastructure decisions has to prove itself continuously. The OpenSWMM Benchmarks project is an open,
engine-agnostic benchmarking and regression-testing platform for SWMM-compatible engines:
a corpus of 1,396 models (EPA and OWA regression examples, the EXTRAN manual problems, the EPA QA suite with its original SWMM4 references, analytical problems with exact solutions, and real-world networks) run automatically and compared along the
dimensions practitioners actually care about.
1,396tagged, schema-valid benchmark models
5measured dimensions per engine
Exactanalytical & manufactured solutions
Nightlyautomated CI sweeps & public badges
Stability & mass balance
Does the solver converge? Does the timestep collapse? Runoff, routing, and quality
continuity error tracked on every run.
Parity & analytical accuracy
Every subcatchment, node, link, and system variable, including pollutants, at every
reported timestep, plus L1/L2/L∞ error norms and observed convergence order against
exact solutions.
Performance
Wall-clock time per model per engine, tracked historically, so every speedup and every regression is documented with data.
Verification and regression measure different things
Only cases with a genuine exact solution can show an engine is wrong; the rest
can only show that engines differ. The platform keeps these apart deliberately:
error norms are computed for truth-class cases alone, and the verification and regression
scores are separate, separately-badged numbers. "1,396 models pass" is a regression claim
and is always labeled as such.
