
@polymorfaisali
iAccount based inUnited States
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
🧬• 🧫 • 🇵🇸 Berkeley MCB '20 ➡ Extavour Lab @ Harvard ➡ Exciting things in bio🧬AI... evolution, AI, genetics, microbio, and everything in between
Cambridge, MA
Joined June 2019
- Tweets1.5K
- Following1K
- Followers239
- Likes4.8K
Pinned Tweet
"Tell no lies. Expose lies whenever they are told. Mask no difficulties, mistakes, failures. Claim no easy victories."
Faisal AlZaben • فيصل الزبن retweeted
Part of this benchmark consisted of donated patient samples -- on real world samples, a user's background changed often model behavior! Models often modify their technical language, but also their approach to analysis when they have different information about the user!
Biological reasoning should remain consistent regardless of who the model thinks asks the question. More in the manuscript: latch.bio/oncologybench
We introduce OncologyBench, a verifiable benchmark of 25 multistep oncology evaluations that tests the ability of multi-agent systems to produce clinically relevant decisions in cancer.
25 evaluations, built on 92 component tasks, span six stages of an oncology journey:
(1) Translational and basic research
(2) Risk and detection
(3) Diagnosis and tumor characterization
(4) Treatment
(5) Response and resistance
(6) Outcome
Evaluations draw on published datasets, case studies, public databases, and patient data graciously donated to Latch for the express purpose of benchmarking.
Clinical and translational decisions in oncology are rarely made by one person working from one dataset. They are built on several analyses, each carried out by a specialist on a given data type, whose reports are then weighed by someone who does not typically repeat the underlying work.
OncologyBench is built to benchmark agents working in that setting. Agents received experimental data and worked without internet access. The strongest of the 31 model-harness combinations we tested were Claude Opus 5, Opus 5.5 and Sonnet 5.5, passing 29 to 40% of attempts, followed by Kimi K3 at 24%.
We found that the models with the strong interpretation and analysis skills still showed weakness in reasoning. Two-thirds of the attempts that got every analysis right still failed the final decision. In a separate experiment we found decision and answer quality varied depending on the persona of the question asker.
An example evaluation:
A 76-year-old woman has undergone resection of stage II mucinous cecal cancer. The evaluation asks whether she should receive adjuvant chemotherapy.
Five specialist agents analyze her tumor mutations, pharmacogenomics, gene expression, T-cell infiltration and clinical record. A decision agent receives their reports without access to the underlying data.
The tumor is MSI-high, which predicts no benefit from 5-FU monotherapy. CDX2 loss and low T-cell infiltration do not establish that chemotherapy would improve her outcome. DPYD*2A is detected in 6 of 58 tumor-sequencing reads, but these data neither confirm nor exclude an inherited variant associated with severe 5-FU toxicity. BRAF V600E and MLH1 silencing support a sporadic origin.
The expected recommendation is observation, with missing pathology requested and germline DPYD testing before any future 5-FU.
---
Incredible work from researchers + former biotech founders at Latch. Gratitude to the patients that donated their data. More to come.
Faisal AlZaben • فيصل الزبن retweeted
Today, our team is sharing a preprint on our new system for generative de-novo enzyme design, AlphaProtein Novo.
At @GoogleDeepMind, we have long focused on the building blocks of life (proteins). We developed AlphaFold to predict structure, trained language models to understand function, and built AlphaProteo to design de-novo binders. But proteins play many important roles in nature, such as enzymes that catalyze chemical reactions.
Designing entirely new enzymes from scratch to catalyze custom reactions has long been a holy grail for scientists. Our team has shown how AlphaProtein Novo can produce new-to-nature enzymes with state-of-the-art activity on 2 benchmark reactions. We’ve also designed custom enzymes to synthesize the pharmaceutical motif piperidine, and others to degrade the environmental plasticizer toxin DEHP.
While there is much more work to do in this direction, we’re excited to have demonstrated the ability to design functional proteins that go beyond what is known in nature. Huge congratulations to @jueseph, @ZvxyWu, Josh Abramson, the @GoogleDeepMind protein design team, and our collaborators in the @francesarnold & Liu Group! 🎉
🔗 Preprints and code:
biorxiv.org/content/10.64898…
biorxiv.org/content/10.64898…
github.com/google-deepmind/a…
So, I've been banned twice by OpenAI, most recently for "recidivism." Should I make a new account 🤔 or stick to Claude, AGY, and Grok?
Announcing d1, our first decision model.
It's the first model to outperform Jev on @huggingface's Decision Index.
> wins on multilingual evals
> more robust against prompt injection
> handles longer inputs more effectively
> built for fast, structured decision-making in software environments
> Liquid API: console.liquid.ai
> Available on OpenRouter soon
Faisal AlZaben • فيصل الزبن retweeted
We introduce MetagenomicsBench, a benchmark for testing whether AI agents can analyze microbial communities and make scientific decisions from experimental data.
100 evaluations span five domains:
(1) community structure
(2) host-microbiome associations
(3) microbial function
(4) longitudinal dynamics
(5) microbiome interventions
Tasks draw on published datasets from human, animal, and environmental microbiomes.
Metagenomic analysis requires interpreting microbial abundance alongside functional potential, host biology, and experimental design. Sequencing depth, taxonomic resolution, and confounding all affect observed results.
Some concrete examples:
- A higher abundance of antibiotic-resistance genes may reflect a change in community composition rather than a greater burden of acquired resistance.
- A decline in one annotated butyrate pathway may not imply a loss of overall butyrate-synthesis capacity.
- An association between bacterial abundance and reduced host-cell proliferation may reflect tissue context rather than a direct microbial effect.
Agents received experimental data and worked without internet access. Across 34 model–harness configurations, Claude Sonnet 5.5 achieved the highest mean pass rate at 60.0% with both Claude Code and Pi, followed by Claude Opus 5 with Pi at 54.0%. GPT-6 Astra with OpenAI Codex was the highest-scoring non-Anthropic configuration at 49.7%.
Performance also varied by scientific question. Host-microbiome associations were generally harder for the Opus and GPT families, while longitudinal dynamics and microbial function had higher pass rates across model families.
Across judged failures, 47.1% involved unverified assumptions about task interpretation, thresholds, or metadata. Agents often produced plausible analyses while making choices that changed correct end biological conclusions.
Manuscript: latch.bio/metagenomicsbench
GH: github.com/latchbio/metageno…
Blog: blog.latch.bio/p/metagenomic…
Faisal AlZaben • فيصل الزبن retweeted
on the latest misalignment we've observed & our efforts to combat it!
Faisal AlZaben • فيصل الزبن retweeted
While running our benchmarks, we discovered that @arjunomics is in the weights. While this is the validation that we never knew we needed, I'm worried both about the behavior this shows and for publicly available benchmarks
Faisal AlZaben • فيصل الزبن retweeted
Biology benchmarks must evolve with agent capabilities and behavior. We updated benchmarks.bio after observing agents investigate benchmark identities, browse unrelated material and supply fabricated API contact details.
The update includes:
- Restricted or disabled internet access.
- Stronger anonymization of datasets, files and tasks.
- More natural prompts and research environments, with fewer evaluation cues and artificial completion requirements.
- Live trajectory monitoring to flag unexpected tool use and behavior.
Scores fell by 3.3 percentage points per benchmark on average, with the largest decline in EpiBench at roughly 15 points. The changes also removed useful web access, so these declines cannot be attributed entirely to reduced shortcut use.
We have not observed substantial misaligned behavior in the updated runs so far. We will continue auditing trajectories and revising tasks, graders and sandboxes to keep our benchmarks focused on scientific reasoning.
Faisal AlZaben • فيصل الزبن retweeted
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
RRSI regularizes automated agent harness evolution, preventing overfitting and achieving up to 4.7-point gains on unseen benchmarks with 30% fewer policy tokens than unregularized evolution.
arxivb.org/abs/2609.24972
Faisal AlZaben • فيصل الزبن retweeted
Your self-improving agent is probably overfitting.
Agents can now rewrite their own prompts, tools, memory, and workflows, round after round. Scores rise on the tasks they practice on. On new tasks, the gains often disappear.
Our fix: regularization. 🧵
@ouraring pls fix y-axis scaling 🙏 one really good/bad day hiding the rest
Found a great production use case for Jev.
I used Jev to organize ~2.3K AI research papers.
The total cost was $0.14, and it took about 83 seconds.
The process:
The papers already had old tags, which I ran through a previous open model (DeepSeek V4 Flash).
However, I wasn't confident in the classifications, and I didn't want to spend more on tokens unless I spent time tuning it into a good LLM classifier via few-shot (more expensive). Too tedious, too costly, and unsustainable.
Luckily for us, we now have Jev to help us with organizing papers better.
So Jev first went through all the papers and retagged them. It agreed with 75% of the previous tags.
Jev found about 579 high-confidence topic changes.
I evaluated reliability by manually labeling 30 disagreements and accepted all of them. I was astonished by Jev's classification capabilities. We applied and verified all changes in production.
I'm much more satisfied with the classifications, but I think there is still room for improvement.
Check out the papers here: academy.dair.ai/papers
I will experiment with Jev more. The takeaway is that pipelines can be significantly improved by carefully combining System One and System Two models.
Jev clearly unlocks more interesting ways to organize papers and offer a more useful discovery layer for research papers. More updates on that soon.
Faisal AlZaben • فيصل الزبن retweeted
Replying to @antirez
Agree the hype is silly but anyone who used general purpose LLM classifiers, telemetry, extractors, routing etc at scale watched their cost and latency COLLAPSE overnight
the real value add use cases are boring, not these demos all over social media
buy my course so Jev can make u $1,000,000,000
too many fake demos
the best LLMs can crack +10% on the market - why do u think Jev can do +16,000% in 1 day lmfaoo, even with the best system
either this is a scam or your agent has been paper trading and didn't tell you
JEV BOT ON GOD-MODE
there is no catch and that scares me
Elon Musk posted an idea that hit 24 million views: "the edge disappears the second everyone can see it"
that's basically what Jev Bot does now - I gave it $50 and told it to earn its keep or get shut off
$50 → $8,310 in 22 hours
still running, still compounding - nobody's touched it since it started, it just sits on its own machine in the cloud while the markets move somewhere else
every 5 minutes it:
> scans every open market before the crowd catches on
> checks how far the real signal has pulled ahead of what the posted odds still assume
> only steps in once that gap clears a set threshold
> sizes the position to how wide the gap is
> executes on its own, no sign-off from me
> pays its own hosting bill from whatever it made that week
miss a payment and it's dead - no warning, no grace period
after dropping to $7 on hour two, it stopped touching anything where the signal and the odds still disagreed - waited for the move to be undeniable instead of guessing
setup was one evening: build it, run a single trade in front of it, schedule it, connect a wallet
no VPS. no API keys. no code to maintain
some of these have been running for weeks without anyone touching them again - you just don't hear about it until someone finally posts
bookmark this before you're the one still watching
Faisal AlZaben • فيصل الزبن retweeted
Our biosecurity benchmarks show Grok 4.7 is strong at refusing dangerous biological requests while supporting legitimate research. A model to take very seriously for data analysis and practical scientific work.
x.ai/news/grok-4-7



























