Debugging Proof Assistants

The use of tac­tics in mod­ern proof assis­tants seems to have the same fla­vor of usabil­ity prob­lem as trait errors. For exam­ple, a pro­gram­mer applies a tac­tic to some goal; the appli­ca­tion either suc­ceeds or gets sim­ply a “No” fail­ure. In the­ory, sim­i­lar tech­niques might be use­ful to diag­nose a trait error as to diag­nose a tac­tic fail­ure. How­ever, we strug­gled to find much related work on this sub­ject. In the space of proof assis­tants there are two styles of tool, proof visu­al­iz­ers and proof state visu­al­iz­ers. The for­mer empha­sizes writ­ing proofs, fol­low­ing a pat­tern sim­i­lar to that taken on paper, and the lat­ter empha­sizes under­stand­ing the state of the cur­rent proof.

Proof state visu­al­iz­ers ren­der the proof tree before and after each tac­tic appli­ca­tion, but do not help with under­stand­ing the appli­ca­tion itself. Notable tools in the space include Alec­tryon 26 for Lean and Coq-PSV 12, and Shi et al.2023 describe a “tac­tic pre­view” which can help read­ers of proofs iden­tify the goals solved by a given tac­tic. Even though proof state visu­al­iza­tion may seem tan­gen­tial to trait solv­ing, an exten­sion for Argus is to iter­a­tively step through the solver’s exe­cu­tion visu­al­iz­ing the type state at each step. We imag­ine this fea­ture would be more use­ful to com­piler writ­ers and library main­tain­ers.

Proof visu­al­iza­tion tools seem to take an approach of mod­el­ing sequent cal­cu­lus with Gentzen trees. These tools aim to help proof writ­ers by pro­vid­ing a more “paper-like” feel to mech­a­nized proofs. The old­est tool in this space, ProofTree, works with Coq and Proof­Gen­eral and dis­plays the tree top-down sim­i­lar to the Argus And–Or tree. show two addi­tional tools in the space, Paper­proof and Traf 18, 19 that use a more Gentzen-style nat­ural deduc­tion proof to help proof writ­ers.