Skip to content

Verification

A test suite tells you whether the code still does what it did yesterday. riskpy.verify answers a different question: whether what it does is right. It computes a value with the library and compares it to something the library had no hand in — a closed form, a published table, a mathematical identity, or the same quantity by an unrelated route.

from riskpy import verify

report = verify.run()
print(report.summary())
assert report                    # False if any check failed

python -m riskpy.verify              # the same from the command line
riskpy-verify --bench                # with timings, for speed regressions
riskpy-verify --json report.json     # for a dashboard
riskpy-verify -m life -m reserving   # a subset

What a check is

Every check is a name, the value the library produced, the reference it should match, and an absolute tolerance. The tolerance is chosen per check and stated: machine precision for exact identities, the printed precision for values transcribed from a book, and a few standard errors for anything Monte Carlo. Every stochastic check runs on a fixed seed, so it is reproducible, and is loose enough never to fail by chance.

Some of what is checked, by module:

module examples
core FactorModel against hand arithmetic; the hand-checked chain ladder; the compound Poisson FFT with unit severity is Poisson(λ); the C++ aggregate-loss mean is λ·E[X]
special Φ(0) = ½; Φ⁻¹(Φ(x)) = x; P(1, x) = 1 − e⁻ˣ; I_x(a, b) + I_{1−x}(b, a) = 1; the t with one degree of freedom is Cauchy; ψ(1) = −γ; ψ₁(1) = π²/6; and, when SciPy is installed, agreement with it to 1e-13 (1e-8 for the Student-t quantile)
mc cdf(ppf(q)) = q for every family; sample means within four standard errors of the analytic mean; Latin hypercube within one; Iman–Conover recovers the target Spearman and leaves the marginals untouched; a truncated density integrates to one
quant the canonical Black–Scholes value; put–call parity; delta as a central difference; implied vol round trip; Heston → Black–Scholes as vol-of-vol → 0; Heston put–call parity; the GBM terminal mean
life the SULT's l_100, ä_40 and A_40 against AMLCR Appendix D; A_x = 1 − d·ä_x; A_{x:n} = A¹_{x:n} + ₙE_x; Makeham's closed form against numerical integration; M_x / D_x = A_x; ä_xy + ä_x̄ȳ = ä_x + ä_y
reserving GenIns factors, total reserve and Mack standard errors against Mack (1993); se² = process² + parameter²; agreement with the compiled LossTriangle; Bornhuetter–Ferguson collapsing to the chain ladder
rates par at par; ytm(price(y)) = y; the bootstrap round trip; Nelson–Siegel limits; both short-rate bonds at σ = 0; Vasicek against Monte Carlo
credit equity + debt = assets; equity as a Black–Scholes call; the ASRF at ρ = 0; the credit triangle; the Basel correlation limits; the BCBS risk-weight table
viz every palette colour clears 3:1 contrast on its surface; the sequential ramps are monotone in lightness; eight categorical slots and no more

Without SciPy the suite runs 111 checks; with it, the 14 oracle checks bring it to 125. NumPy is required for the mc, quant and reserving checks and Matplotlib for viz. On an install without them those modules are reported as skipped, with the extra that enables them (pip install "open-riskpy[sim]"), and the run still passes: a missing extra is a fact about the environment, not a wrong number. Asking for such a module explicitly (riskpy-verify -m mc) fails, because nothing you asked for could run.

Where it runs

  • CI, on every push and pull request — a wrong number fails the build.
  • Nightly, with the SciPy oracle switched on and the benchmarks included, keeping the JSON report as an artifact so a slow drift in accuracy or speed leaves a trail even on days nobody pushes.
  • Here, regenerated on every docs build. The table below is the report from the commit this site was built from.

Adding a check

life, reserving, rates and credit contribute checks through a private hook in their own module. Checks for the core, special, mc, quant and viz are written directly in riskpy/verify.py.

def _verification_checks():
    """Return a list of (name, value, reference, tolerance) tuples."""
    return [
        ("whole life A_x = 1 - d * ä_x", A, 1 - d * a, 1e-12),
    ]

The rule for what belongs here: pick things that would catch a class of error, not a single typo. Put–call parity catches every sign error in a pricing formula; one reference price catches one.

Latest report

RiskPY 0.4.0 — 111 of 111 checks pass (1.4s)

module check value reference error tolerance
core FactorModel: 1000 × 3.0 (FL) × 2.0 (age 19) 6000 6000 0.0e+00 1e-09 ✅
core FactorModel: unmatched inputs leave the base rate alone 1000 1000 0.0e+00 1e-09 ✅
core ActuarialMath: PV of 20-year annuity-immediate at 5% 623111 623111 0.0e+00 1e-06 ✅
core ActuarialMath: FV of 30 payments at 6% 395291 395291 0.0e+00 1e-06 ✅
core LossTriangle: hand-checked chain ladder IBNR 2540 2540 0.0e+00 1e-06 ✅
core FourierTransform: compound Poisson with unit severity = Poisson(2) 1.66533e-16 0 1.7e-16 1e-12 ✅
core FourierTransform: PMF has unit mass 1 1 0.0e+00 1e-12 ✅
core FourierTransform: convolution [1,2,3]*[4,5] 0 0 0.0e+00 1e-10 ✅
core ExperienceRating: full credibility at k claims 1 1 0.0e+00 1e-12 ✅
core ExperienceRating: √(n/k) below the standard 0.499538 0.499538 0.0e+00 1e-12 ✅
core ExposureRating: ILF at the base limit is 1 1 1 0.0e+00 1e-12 ✅
core ExposureRating: burning cost of a 100 xs 100 layer 62.5 62.5 0.0e+00 1e-09 ✅
core RateAnalyzer: indicated change at a 77% loss ratio 0.1 0.1 0.0e+00 1e-12 ✅
core RateAnalyzer: on-level factor compounds rate changes 1.1319 1.1319 0.0e+00 1e-12 ✅
core MonteCarloSimulator: aggregate mean = λ·E[severity] (200k trials, ±3 s.e.) 16918.222331 16889.339658 2.9e+01 6e+01 ✅
special norm_cdf(0) = 1/2 0.5 0.5 0.0e+00 1e-16 ✅
special norm_cdf(-37) keeps its tail (erf-based forms give 0) 1 1 0.0e+00 0e+00 ✅
special norm_ppf(norm_cdf(x)) = x at x = -3.7 -3.7 -3.7 4.4e-16 1e-13 ✅
special norm_ppf(0.975) = 1.959964 1.959964 1.959964 0.0e+00 1e-14 ✅
special gammainc(1, x) = 1 - e^-x 0.899741 0.899741 1.1e-16 1e-14 ✅
special gammainc + gammaincc = 1 1 1 0.0e+00 1e-14 ✅
special gammainc(n, x) = Poisson survival (n=4, x=2.5) 0.242424 0.242424 5.6e-17 1e-14 ✅
special betainc(1, 1, x) = x 0.37 0.37 0.0e+00 1e-14 ✅
special betainc symmetry I_x(a,b) = 1 - I_{1-x}(b,a) 1 1 1.1e-16 1e-14 ✅
special betainc(a, 1, x) = x^a 0.216 0.216 2.2e-16 1e-14 ✅
special t_cdf with 1 d.o.f. is Cauchy: ½ + atan(x)/π 0.791286 0.791286 1.1e-16 1e-13 ✅
special t_ppf(t_cdf(x)) = x, 4 d.o.f. 2.1 2.1 4.4e-16 1e-09 ✅
special chi2_cdf with 2 d.o.f. = 1 - e^{-x/2} 0.77687 0.77687 2.2e-16 1e-14 ✅
special digamma(1) = -Euler–Mascheroni -0.577216 -0.577216 1.9e-14 1e-13 ✅
special trigamma(1) = π²/6 1.644934 1.644934 2.4e-14 1e-13 ✅
special brent solves x³ = 2 1.259921 1.259921 0.0e+00 1e-12 ✅
special simpson ∫₀^π sin = 2 2 2 6.8e-10 1e-09 ✅
special nelder_mead finds (1, -2) 7.15219e-11 0 7.2e-11 1e-05 ✅
special kolmogorov_sf(1) = 0.26999967 0.27 0.27 0.0e+00 1e-12 ✅
mc Normal: cdf(ppf(q)) = q 1.38778e-17 0 1.4e-17 1e-09 ✅
mc LogNormal: cdf(ppf(q)) = q 1.38778e-17 0 1.4e-17 1e-09 ✅
mc Gamma: cdf(ppf(q)) = q 1.11022e-16 0 1.1e-16 1e-09 ✅
mc Beta: cdf(ppf(q)) = q 3.88578e-16 0 3.9e-16 1e-09 ✅
mc Weibull: cdf(ppf(q)) = q 1.38778e-17 0 1.4e-17 1e-09 ✅
mc StudentT: cdf(ppf(q)) = q 4.16334e-17 0 4.2e-17 1e-09 ✅
mc Pareto: cdf(ppf(q)) = q 2.91434e-16 0 2.9e-16 1e-09 ✅
mc PERT: cdf(ppf(q)) = q 1.66533e-16 0 1.7e-16 1e-09 ✅
mc Triangular: cdf(ppf(q)) = q 1.11022e-16 0 1.1e-16 1e-09 ✅
mc Exponential: cdf(ppf(q)) = q 0 0 0.0e+00 1e-09 ✅
mc Uniform: cdf(ppf(q)) = q 8.67362e-19 0 8.7e-19 1e-09 ✅
mc Gamma: sample mean = analytic mean (200k, ±4 s.e.) 4.25047 4.25 4.7e-04 2e-02 ✅
mc Gamma: LHS mean within 1 s.e. (variance reduction) 4.249999 4.25 8.7e-07 6e-03 ✅
mc LogNormal: sample mean = analytic mean (200k, ±4 s.e.) 4.242976 4.241852 1.1e-03 3e-02 ✅
mc LogNormal: LHS mean within 1 s.e. (variance reduction) 4.241807 4.241852 4.5e-05 8e-03 ✅
mc Poisson: sample mean = analytic mean (200k, ±4 s.e.) 140.000835 140 8.3e-04 1e-01 ✅
mc Poisson: LHS mean within 1 s.e. (variance reduction) 139.99998 140 2.0e-05 3e-02 ✅
mc NegativeBinomial: sample mean = analytic mean (200k, ±4 s.e.) 139.973535 140 2.6e-02 2e-01 ✅
mc NegativeBinomial: LHS mean within 1 s.e. (variance reduction) 139.999945 140 5.5e-05 4e-02 ✅
mc Weibull: sample mean = analytic mean (200k, ±4 s.e.) 1.804603 1.805491 8.9e-04 1e-02 ✅
mc Weibull: LHS mean within 1 s.e. (variance reduction) 1.805488 1.805491 2.4e-06 3e-03 ✅
mc Iman–Conover: Spearman(a, b) = 0.7 0.700447 0.7 4.5e-04 1e-02 ✅
mc Iman–Conover: Spearman(a, c) = -0.4 -0.400389 -0.4 3.9e-04 1e-02 ✅
mc Iman–Conover: unrelated pair stays at 0 -0.000462066 0 4.6e-04 1e-02 ✅
mc Iman–Conover: marginal untouched (same draws, only re-paired) 0 0 0.0e+00 1e-09 ✅
mc Truncated: density integrates to 1 1 1 8.0e-10 1e-06 ✅
mc Mixture: mean is the weighted mean 12756.62529 12756.62529 0.0e+00 1e-09 ✅
mc Result.var(0.99) = np.percentile(99) 2.324256 2.324256 0.0e+00 1e-12 ✅
mc Result.tvar ≥ Result.var 1 1 0.0e+00 0e+00 ✅
quant Black–Scholes ATM call, S=K=100, T=1, r=5%, σ=20% 10.450584 10.450584 0.0e+00 1e-09 ✅
quant put–call parity 9.26043 9.26043 3.6e-15 1e-10 ✅
quant delta = ∂price/∂S (central difference) 0.693464 0.693464 2.3e-10 1e-06 ✅
quant implied_vol(price(σ)) = σ 0.27 0.27 2.0e-10 1e-07 ✅
quant Heston → Black–Scholes as vol-of-vol → 0 9.413432 9.413403 2.9e-05 1e-03 ✅
quant Heston put–call parity 12.002389 12.002389 0.0e+00 1e-06 ✅
quant parametric_var at 99% = 2.326348 σ 2.326348 2.326348 0.0e+00 1e-09 ✅
quant GBM terminal mean = S₀e^{μT} (100k paths, ±4 s.e.) 107.140709 107.250818 1.1e-01 3e-01 ✅
life SULT l_100 vs AMLCR Table D.1 6248.174333 6248.17 4.3e-03 1e-02 ✅
life SULT ä_40 at 5% vs AMLCR Table D.3 18.457757 18.4578 4.3e-05 5e-05 ✅
life SULT A_40 at 5% vs AMLCR Table D.3 0.121059 0.12106 7.9e-07 5e-06 ✅
life whole life A_x = 1 - d * ä_x 0.121059 0.121059 1.2e-15 1e-12 ✅
life endowment A_{x:n} = A¹_{x:n} + nE_x 0.381263 0.381263 0.0e+00 1e-12 ✅
life Makeham closed form 30_p_40 vs Simpson-integrated force 0.916892 0.916892 4.8e-15 1e-09 ✅
life commutation M_60 / D_60 = A_60 0.290282 0.290282 0.0e+00 1e-12 ✅
life joint life ä_xy + ä_x̄ȳ = ä_x + ä_y 30.963941 30.963941 0.0e+00 1e-12 ✅
reserving GenIns first volume-weighted factor (Mack 1993) 3.490607 3.4906 6.5e-06 5e-05 ✅
reserving GenIns chain ladder total reserve (Mack 1993) 1.86809e+07 1.86809e+07 3.9e-01 1e+00 ✅
reserving GenIns Mack total s.e. (Mack 1993 Table) 2.44709e+06 2.4471e+06 1.4e-01 2e+03 ✅
reserving GenIns Mack s.e. of the youngest origin 1.36315e+06 1.36316e+06 8.8e-02 1e+03 ✅
reserving Mack se² = process² + parameter² (origin 9) 1.85819e+12 1.85819e+12 2.4e-04 2e+06 ✅
reserving chain ladder vs compiled LossTriangle (max relative gap) 4.44089e-16 0 4.4e-16 1e-06 ✅
reserving Bornhuetter–Ferguson at ELR = CL ultimate / premium equals CL 1.86809e+07 1.86809e+07 3.7e-09 2e+01 ✅
reserving Cape Cod: Σ latest = ELR · Σ premium · %developed 3.43581e+07 3.43581e+07 0.0e+00 3e+01 ✅
rates par bond prices at face when ytm = coupon 100 100 1.4e-13 1e-10 ✅
rates ytm(price(y)) = y 0.043 0.043 1.5e-15 1e-10 ✅
rates zero-coupon Macaulay duration = maturity 7 7 0.0e+00 1e-12 ✅
rates bootstrap round trip: 10y par rate 0.04 0.04 4.9e-17 1e-10 ✅
rates flat curve df(5) = exp(-0.15) 0.860708 0.860708 0.0e+00 1e-14 ✅
rates forward consistency df(5) = df(2)·exp(-3f) 0.83733 0.83733 0.0e+00 1e-12 ✅
rates Nelson–Siegel z(0) = β0 + β1 0.02 0.02 0.0e+00 1e-15 ✅
rates CIR bond at sigma = 0 = exp(-∫r dt) 0.807927 0.807927 0.0e+00 1e-12 ✅
rates Vasicek bond at sigma = 0 = exp(-∫r dt) 0.807927 0.807927 0.0e+00 1e-12 ✅
rates Vasicek bond vs Monte Carlo exp(-∫r dt) 0.928147 0.928257 1.1e-04 3e-03 ✅
credit Merton: equity + debt value = assets 120 120 0.0e+00 1e-12 ✅
credit Merton: equity = Black-Scholes call on the assets 31.279803 31.279803 0.0e+00 1e-14 ✅
credit Merton: credit spread = 0 at zero asset volatility 5.55112e-17 0 5.6e-17 1e-14 ✅
credit ASRF: conditional PD at rho = 0 is the unconditional PD 0.02 0.02 0.0e+00 1e-15 ✅
credit Vasicek loss quantile = ASRF conditional PD 0.206066 0.206066 0.0e+00 1e-15 ✅
credit CDS par spread = credit triangle h(1-R) 0.006 0.006 3.5e-10 1e-08 ✅
credit Basel corporate correlation -> 0.12 as PD -> 1 0.12 0.12 0.0e+00 1e-15 ✅
credit Basel IRB risk weight, PD 1% / LGD 45% / M 2.5 0.923168 0.9232 3.2e-05 1e-03 ✅
viz dark palette: every series colour ≥ 3:1 on its surface (worst) 1 1 0.0e+00 0e+00 ✅
viz dark sequential ramp: lightness monotone 1 1 0.0e+00 0e+00 ✅
viz dark palette: eight categorical slots, no more 8 8 0.0e+00 0e+00 ✅
viz light palette: every series colour ≥ 3:1 on its surface (worst) 1 1 0.0e+00 0e+00 ✅
viz light sequential ramp: lightness monotone 1 1 0.0e+00 0e+00 ✅
viz light palette: eight categorical slots, no more 8 8 0.0e+00 0e+00 ✅