
ALT We present the techniques that made agentic proof-oriented programming (PoP) effective—or ineffective—and characterize the role of the human expert, whose contribution reduces to providing natural-language problem descriptions, reviewing auto-generated specifications, and occasionally supplying a key invariant. Our findings suggest that this division of labor provides substantial leverage to the human expert in the loop: three experts, over the course of two weeks, completed case studies whose manual proof-engineering cost we estimate at roughly half a year. Table 1. Cost and effort summary across the three case studies. Manual estimate is each expert’s assessment of how long the same work would have taken without agentic assistance; Agent-assisted is the actual elapsed effort with Copilot CLI. See Appendix A for per-project breakdowns and methodology.

ALT Fig. 1. Human involvement per data structure, ordered by increasing human contribution. “Lines” counts verified code, specification, and proof. Agent autonomy scales inversely with proof complexity, yet the boundary shifts when the agent has a similar reference in the same codebase. Sequential structures required only a prompt; the reader-writer lock demanded a formal invariant, whereas the semaphore became tractable by referencing the lock. In every case the mechanical burden of proof construction falls on the agent. The promise of program proof is that it reduces what one has to trust about a piece of code to its specification. While agents have substantially lowered the bar for proof-oriented programming, one still needs domain expertise and some specification review skills to be productive. The agents will only keep getting better, but human understanding of agent outputs will remain a concern.

ALT Vacuous Verification. The most insidious failure mode we encountered is what we call vacuous verification (see Section 4): the agent produces code that type-checks and verifies, yet does not actually implement the intended functionality. Because the proofs go through, the failure is indistinguishable from success without careful auditing. The defense is systematic auditing at every phase boundary—reviewing not just that the code verifies, but that it actually implements the intended behavior. The idea of letting an agent run all night and have a verified system in the morning is appealing, but not yet realistic. The human must be in the loop to steer the process, catch mistakes early, and ensure that each phase produces a verifiable artifact meaningfully connected to the intended behavior.

ALT Furthermore, the human must still make crucial architectural decisions about how verified modules compose. The agent can check whether a composition works, and even explain why it does not, but it cannot anticipate the design intent behind reuse. Directing agents is itself a new discipline. The proof engineer’s job is no longer to write invariants line by line, but to tame the agent: decompose a task into phases the agent can handle, supply high-level proof ideas when it stalls, and—critically—audit for vacuous verification. The reader-writer lock (Section 2) is a good example: three hours of unaided effort failed, but a 30-line formal invariant unlocked a 1,200-line verified module. The new literacy of proof engineering is specification reading: understanding what a function promises, what it demands, and whether those contracts faithfully capture the intended behavior.























