Kibi

Started running Kibi's proof lane on Kibi itself. Each requirement gets labeled by what actually backs it: a test, or a proof that covers every case. Takeaway: writing the proof forced a few requirements to get more precise, because the vague version couldn't be proved at all.

Agree on 1. The useful part is not mixing the two up. We ended up labeling each requirement by what actually backs it: tested if it's E2E/simulation, proved only if a certificate covers every case. Most things stay "tested", and that's fine, as long as nobody reads it as proof.

"Within defined assumptions" is doing a lot of work there. In practice the hard part for us was making sure the proof is about the same thing the requirement says, and re-checking it when either side changes. Otherwise you get a stale proof that still looks green.

> agent doesn't deliver what it was asked to -> kibi

That's why Kibi went with Prolog knowledge base instead of traditional memory. Necessary ADRs and requirements are delivered to the agent in a progressive disclosure manner. All the relationships (like superseeded) are handled deterministically by Prolog.

You need requirements, evidence and proof in your development pipeline. Also well maintained and exhaustive E2E tests. Also run code gardening sessions from time to time.

FYI @PrologInfo :)

The theorems and the differential test: looted.github.io/kibi/guide/…

Kibi's numeric checks are Prolog: intervals.pl folds each comparison on a variable into an interval and asks if it's empty. Now those folds are proved exact in Lean 4, and CI runs 200k cases through the Prolog code, the Lean model and a CLP(Q)/CLP(FD) oracle.

Legacy codebase aside, creating formal proof in greenfield projects has upsides: the proof is created when the context is fresh. The problem with legacy codebases is really that context (in broad sense) is probably long gone, and the code doesn't necessarily express it (bugs).

Update: as of today something can. The formal proof lane is out: certificates that cover every case, re-checked by Kibi itself. nitter.gq/kibi_dev/status/211462…

Hey @ArthurLiu0725! It's up to you and your CI really. Contradictions get blocked at commit gate (git hooks), for proof levels you could set your CI to block on requirement/proof level. Haven't tried the latter though.

How certificates bind to claims: looted.github.io/kibi/guide/… Setup prompt: github.com/Looted/kibi#quick…

Passing tests are evidence, not proof. They sample cases. New in Kibi: the formal proof lane. A requirement is marked proved only when a certificate covers every case (every enum member, a typechecked switch, z3/cvc5 returning unsat) and Kibi re-checks it itself.

Hey Martin! In Kibi every requirement has a firmness level and only a person sign-off will give it a firm level. When it comes to testing, Kibi assigns every requirement an evidence level: looted.github.io/kibi/guide/…. Untested code will never reach tested, let alone proved.

Feel free to send a DM or skim through the docs: looted.github.io/kibi/. The formal proof work is still on develop but landing soon.

Each requirement gets a computed level: proved > tested > asserted > unverified. “Proved” is formal: requirements compile to typed facts, solvers check consistency and emit certificates a separate checker re-verifies.

Hey @MichaelObasa! Kibi never counts an agent’s claim as evidence. Agent-written requirements stay unverified until a person approves them. Kibi runs the tests itself and ties each receipt to a hash of the code.

11/ Kibi is a repo-local knowledge base for coding agents, checked by Prolog (bundled on Linux and macOS). npm i -D kibi-core kibi-cli kibi-mcp github.com/Looted/kibi Proof ladder: looted.github.io/kibi/guide/…