Eventum AI

Garan and colleagues studied heart failure phenotyping in a US health system. A sentence-level rule pipeline scored 4.9% F1 on HFpEF. A pipeline combining evidence across the longitudinal record reached 91.0%, often by linking a problem-list mention to an EF value elsewhere.

A generated cohort rule passed all three of its tests. Each test case carried all three qualifying signals, so none could separate "any one qualifies" from "diagnosis plus another". A diagnosis-only patient exposed the bug at once. Pick cases that separate rules.

A statin recommendation can report four true facts and still be wrong, because it never mentions the dialysis exclusion. A checker that verifies only what the explanation lists will accept it. A checker that pulls the required conditions from the approved rule will not.

Two rules for a 130/80 blood pressure target, one using AND and one using OR, agree on all ten of the sample readings we listed. Ask a solver for a case where they disagree and you get 134/76 and 124/84. A passing test suite can hide which rule you shipped.

In our worked example the generated rule passed the schema and even quoted the correct averaging footnote. It still used the latest measurement, shifted the thresholds, chose the lower category and skipped the insufficient-measurements case. The schema allowed all four.

In LLM-Vectorizer (Jubi Taneja and colleagues), model agents rewrote 149 loops into vectorized code. Checksum testing called 125 of them plausibly correct. Checking those with Alive2 verified 57, showed 37 incorrect and left 31 inconclusive. Passing tests leaves a wide margin.

In a study of 20 primary care physicians reviewing simulated portal messages, 13 to 15 of them left each of four erroneous AI drafts insufficiently corrected. Most still reported lower cognitive workload and rated the drafts safe to use. Human review needs its own evaluation.

A benefits policy says eligible after at least twelve completed months. One model writes completed_months >= 12. Another writes > 12. Both are well formed, both run, and a test suite with 11 and 13 month cases passes either one. They only disagree at exactly 12.

Sanas developers used to wait on full pipeline runs to validate one ETL step. Now every merge request gets an environment where any step can run from any point in the graph. Across the whole program the ML dev cycle came out 40% faster.

An AI can make the same mistake twice: once when it writes the rule, again when it writes the expected answers. The tests pass because the wrong rule agrees with the wrong expectations. Generating an answer key and establishing its authority are different tasks.