Working on Perch - A Semantic Linter powered by Decision One Models
Joined October 2011
- Tweets546
- Following371
- Followers1.8K
- Likes1.8K
Pinned Tweet
@typesafeai Loving behavior-driven development with Jev.
github.com/lakeday-org/perch
Working on a new Fuzzy Unit test coverage tool, that uses SystemOne to weed out defects from noise.
Anyone interested?
@StefanoErmon Nice work!
Last week, we launched Mercury Decide, a Jev-beating decision model that quickly rose to #2 on OpenRouter by number of users: openrouter.ai/inception/merc…
This week, we’re updating Mercury Decide with substantially better benchmark performance, longer context (33k → 66k), General Availability with Zero Data Retention (ZDR), and more.
All that with lower pricing than Jev (ZDR for $0.02/$0 per 1M in/out, discounted at 50% from the list price of $0.04/$0, plus 100M free tokens), alongside an unlimited free tier.
What have we improved? Read on to find out! [🧵1/4]
Perch lint tests SystemOne models over a Tree Sitter, assessing each models ability to find regressions against a behavior rule written in plain english.
Learn more at
github.com/lakeday-org/perch
Josh Brown retweeted
How's Perch working for you? Tell us what it caught, what it got wrong, and what you want it to do next.
Thread's here: github.com/lakeday-org/perch…
Nearing 5k downloads on npm! Still cant believe it.
@OpenAI support for Decisions API coming today
Over 2k downloads and counting. Ability to pin rules at repo, file, and even method leve
github.com/lakeday-org/perch
Released! + a bonus feature in early makings. Hope to share more soon
github.com/lakeday-org/perch…
The perch lint benchmark, checks various decision models against a semantic lint (or rule), aimed at preventing a regression in that code. Methods are fed to models as callstack trees, and answers are (reversed) before and after pairs from github bug and security database.
Full results:
huggingface.co/datasets/perc…
Josh Brown retweeted
Scan your code for defects, security issues, and anything you want. Get started today at perchscan.com
Perch has over 9k downloads, hoping to get over 10k this week.
Its free (MIT) licensed, and it will scan your code for bugs, security (top 25 CWE), and anything else you want it to!
youtu.be/Ybp8bIVjT4Q?is=SzXb…
Updated our benchmark! @liquidai is a great model at evaluating developer changes
See the full benchmark here:
huggingface.co/datasets/perc…
Announcing d1, our first decision model.
It's the first model to outperform Jev on @huggingface's Decision Index.
> wins on multilingual evals
> more robust against prompt injection
> handles longer inputs more effectively
> built for fast, structured decision-making in software environments
> Liquid API: console.liquid.ai
> Available on OpenRouter soon
On standard bug detection, we saw DiffusionGemma as Jev (DJEV) slightly beat out jev
Can Jev find bugs and security vulnerabilities in real code?
We tested Jev against four other decision models on code before and after real fixes. We asked broad bug and security questions, then tried a short rule describing what each method must do.
The rule-based checks gave us the strongest results. At Jev’s price, semantic tests can live alongside the code and run again on every change.
Full benchmark: perchscan.com/blog/benchmark…
Can Jev find bugs and security vulnerabilities in real code?
We tested Jev against four other decision models on code before and after real fixes. We asked broad bug and security questions, then tried a short rule describing what each method must do.
The rule-based checks gave us the strongest results. At Jev’s price, semantic tests can live alongside the code and run again on every change.
Full benchmark: perchscan.com/blog/benchmark…
Perch is an open-source semantic code linter powered by SystemOne models. It supports custom lint rules at the project, file, and method levels.
Try it out at:
github.com/lakeday-org/perch