NederlandsFormal verification
In formal verification, a computer uses mathematical logic to prove that software or rules meet a specification, for all possible situations. Ordinary testing only tries out some of those situations.
Formal verification is the mathematical proof that a program or a set of rules is correct. With ordinary testing, you try out a number of situations and see whether it goes well. Situations you do not think of are not checked either. With formal verification, you prove that something works for all possible situations.
It is comparable to a civil engineer who does not just test a bridge with a few cars, but shows by calculation that it can bear every permitted weight.
How does it work?
First, it is written down precisely what has to hold, for example: this program does not crash on any file. Then comes a proof: a step-by-step account of why that is so. Such a proof can be enormous, sometimes hundreds of thousands of lines. A human does not read that. That is why a small checking program does it. It recomputes every step, and rejects the whole proof as soon as one step is wrong. Anyone who wants to trust the result only needs to trust that checking program and the specification of what had to be proved.
A proof does apply exactly to what was specified. It does not say that a program is error-free everywhere, only that it stays within the specification.
Why it matters
As AI writes and reviews more software, it becomes harder to read everything yourself. Formal verification has the AI supply a proof itself. Anthropic had Claude supply a mathematical proof for 123 widely used software building blocks, which a small program checks step by step, and hundreds of hidden errors came to light (link).
It is also useful for safety. Nvidia has AI agents work in a sandboxed environment and checks the rules in advance with such a mathematical method, so that they do not allow more than intended (link).
Articles about Formal verification
-
Anthropic builds FlashProve: cryptographic proof of which AI model produced an answer
Anthropic has released FlashProve, an open-source research prototype that uses zero-knowledge cryptography to show which model generated an answer, without revealing the model weights.
-
No need to take the AI's word for it: Claude delivers a mathematical proof for software
How do you know whether an AI's work is sound if nobody can read it all? Anthropic has Claude supply a mathematical proof that a small program checks step by step. This way Claude examined 123 widely used software building blocks and found hundreds of hidden bugs.
-
Nvidia lets AI agents do only what is allowed, and checks the rules in advance with mathematics
Nvidia presents OpenShell: free software that lets AI agents work in a sealed-off space. A mathematical check looks in advance at whether the rules allow more than intended, and according to Nvidia a separate chip can stop an agent within milliseconds.
