Get the latest tech news
Benchmarking Crimes Meet Formal Verification
There are multiple instances of authors comparing verification efforts of systems projects by looking at the ratio of proof to code size. I demonstrate why this is nonsense and constitutes a benchm…
If we look instead at the spec-to-proof ratio then we see that among the ITP systems (solid blue ovals), seL4 and SeKVM are quite similar (fitted by the green line), while CertiKOS has a surprisingly large amount of proof given its spec size. Nevertheless, experience shows that modularity may dramatically simplify constructing, and, importantly, reason about of complex systems, and it is intuitive that this will also somehow apply to the verification effort (and the above provides some over-simplified justification for this). Obviously, that’s the whole point of automated techniques: When they work, they are relatively cheap (if you accept the green and orange lines in the figure as predictors, which I don’t claim they are, the improvement would be a factor of five).
Or read this on Hacker News