Pinned Tweet
We started @sphinx_bio to empower scientists, and today I'm excited to announce that we'll be continuing that mission as part of @Benchling!
There’s never been a better time to build AI tools to help scientists and there's no better place to build those tools than Benchling.
Benchling is already used by hundreds of thousands of scientists across the world and we are hard at work building AI agents into the platform to help accelerate research.
Keep an eye out for more updates soon!
Day 12 of “An agent a day keeps Phase 2 failures away” — automate CRO data ingestion
Everyone is buzzing about one person biotechs who are heavily leveraging CROs. But basically every biotech and pharma company uses CROs. And scientists waste hours each week monitoring their email and reformatting data. What if the data just showed up how you needed it, when you needed it, in the place you’re already doing your daily work?
Sample prompt (feel free to tweak): Create a CRO Agent which takes any new files in our CRO project and automatically runs the correct pipeline to parse and structure the data. Then, take the outputs of the pipeline and create a notebook entry that includes all the structured data we extracted as well as a summary slide deck.
s/o Clare Young for the guest video
I hate to rain on everyone's parade, but unfortunately LLMs are still not that good at biology :(. Even basic molecular cloning tasks are beyond the reach of frontier models
Full post here: benchling.com/blog/how-good-…
Note: better harnesses can help boost this performance quite a bit
I call this one the “did you” agent.
Day 11 of “An agent a day keeps Phase 2 failures away” — data QC
Doing rigorous QC of your data requires answering a lot of questions. Did you check for transcription errors? Did you check if two samples got swapped? Did you check for outliers? Did you check for edge effects? Did you check for contamination? Did you check if a control is drifting? Did you check for batch effects? Did you check if different operators have different distributions of results? Let’s make an agent to help us automatically answer this question.
Sample prompt (feel free to tweak): Create a QC Agent which runs every day and looks at all of the new data for our program. It should do rigorous QC over the data and look for the following: transcription errors, sample swapping, outliers, edge effects, contamination, control drift, batch effects, operator effects, and any other potential data issues. Give me a full report on any issues and suggest next steps to remediate them or say that there are no issues.
Whoa people are starting to use over refusals of biorisk as an LLM obfuscation technique (potentially increasing cyber risk)…
nullpt.rs/fingerprinting-bio…
Day 10 of “An agent a day keeps Phase 2 failures away” — inventory
There’s no worse feeling than starting an experiment and realizing that you’ve run out of a key reagent. Or that something is past its expiration date. But keeping track of every reagent or component of an experiment is a lot of work. So most of us don’t bother to keep our digital inventory up to date. What if we had an agent that could keep it up to date without doing any additional work (other than keeping a good notebook entry)?
Sample prompt (feel free to tweak): Create an Inventory Agent which runs every day at all of the experiments which were run and updates our inventory (plates, reagents, stocks, etc) based on what was actually used. Also mention if anything is out of date or expired. If anything in our inventory seems to be running low, queue up an order for my approval.
Nicholas Larus-Stone retweeted
black pill: enterprises are putting their AI transformation (and the most existential business risk/opportunity of a lifetime) in the hands of their IT department
And some people don't believe that AI can speed up drug discovery and development... anthropic.com/research/early…
Still early, but lots of opportunity here
Day 9 of “An agent a day keeps Phase 2 failures away” — patents
IP is the lifeblood of biotech, but most data is hidden away and never examined after the initial experiment. However, I’ve been seeing more legal teams use our AI tools to look for novel research that can be patented. But many companies don’t have dedicated patent attorneys or a large enough legal team to handle all of the research that is being produced. Luckily, you can just set up an agent to do that search on your behalf.
Sample prompt (feel free to tweak): Create a monthly Patentable Discoveries Agent which runs every month and looks at all of the research that has been done and identifies up to 3 pieces of work that could be patented. If there are no discoveries that rise to the level of a patentable piece of work, it says so and suggests how existing work would need to be extended to be a novel, patentable discovery.
Kind of surprising they doubled down on the “repeat” discovery given the backlash from the scientific community
Anthropic's S-1 is out
Its first highlighted potential use of AI: "profoundly accelerate the rate of progress in biology and medicine, potentially curing diseases that have plagued humanity for millennia ..."
sec.gov/Archives/edgar/data/…
Day 8 of “An agent a day keeps Phase 2 failures away” — higher quality notebook entries
Notebook entries are the foundation of a biotech’s scientific record. If they’re not accurate, high quality, and up to date, anything you try to do downstream will be more challenging. Most scientists I know see recording their work as a tax, not something they look forward to. But what if there was an agent that could help keep the quality of notebook entries high without requiring more work from already busy scientists…
Sample prompt (feel free to tweak): Create a Notebook Review Agent which runs when a notebook entry is sent for review. The agent should look over the entry and check for any data issues (e.g. transcription errors), the quality and clarity of the prose, as well as the overall completeness of the entry. If there are attached data files that haven’t been put into a structured form, extract the data and put it into structured tables. After reviewing the full entry, the agent should make a few recommendations of what the author should do based on the entry.
This is the level of biorisk discourse on this app
A biological attack is planned for Saturday
600 million pounds of sugar laced with synthetic chemicals will enter the bodies of our children.
Wreaking metabolic havoc, injuring vascular architecture, acidifying oral lining, releasing bacterial toxins, and blunting neural reward pathways.
Instructing that gluttonous consumption of poison is celebratory.
Unfortunately, no
If you had an exponential amount of compute you could solve simulate biology exactly and wouldn't need to run any experiments.
Some of you will think "of course, but that's an unrealistic assumption" and others will think "no, you need experiments even with that much compute. That's just How It's Done".
You would of course need evaluations once you have a solution, but you don't in principle need experiments to find it: Experiments are a way to save compute, dating from a time when it was much more expensive.
And I think the presentation is exponential but real systems aren't irreducibly exponential. AlphaFold worked, after all.
This one is for all my LabOps folks!
Day 7 of “An agent a day keeps Phase 2 failures away” — equipment
Most lab equipment costs hundreds of thousands (or sometimes millions) of dollars. So why wouldn’t you have an agent making sure there’s contamination or miscalibration on any of your machines? Maintaining a lot of equipment is challenging, but knowing when something needs your attention is half the battle: and an agent can help with that.
Sample prompt (feel free to tweak): Create an Equipment Monitor that runs every week and looks at all experiments run on a piece of equipment to determine if the machine might be miscalibrated, needs to be maintained, or is contaminated. Make sure it reports it as a full canvas depiction of the current state of our equipment.
This is a surprisingly reasonable description of why the AI and biology communities keep talking past each other
There’s a really pronounced divide between people who are on the AI side of things and people who are on the biology side of things. Even among people working in AI, the divide persists based on where they’re focused.
The AI side, adherents of scaling laws and the bitter lesson, believe that AI getting better means AI being more capable of novel discoveries in medicine (that go beyond discovering new drugs in well understood targets or speeding up dry lab work etc).
The biology people think the idea of a bitter lesson in biology is kind of ridiculous and that regardless of AI’s capabilities it’s going to be at least 5-10 years until you could hope to reliably simulate human toxicology in silico (right now AI/ML can’t even reliably simulate in vivo studies like binding affinities or functional assays).
Even the most AI pilled of them, who will list all the areas that AI can reduce the time spent on various areas, don’t think the concept of AI discovering novel targets or generating new understanding of causal biology is realistic in the near term. Most, in fact, will readily characterize the assumption that even an “ASI” level model could find cures hidden in our current understanding (without expanding that understanding with significantly more biological data) as patently false.
This was a big part of why we wanted to experience it ourselves by using AI for a (very) small portion of what it can do in biology. It was both mind blowing and revealed some poignant insights into how different the problem of making AI better at something like coding is to the problems of biology.
Even if it unlocked the ability to generate new molecules at a stunning pace, its simulation (in silico) of binding only had 49% accuracy. And if we wanted to improve that, we’d need to feed it back the data from the lab - data that took about 30x as long to get as it took to generate the molecules. And these were tests in a test tube (turn of phrase), imagine how long it would take if we wanted to improve the model with data that required recruiting humans and monitoring their reactions.
Human biology is very complex, and adding a drug - especially a novel drug and doubly so for a novel drug acting on a novel mechanism - means mapping interactions that have nth-order-effects that we don’t have the data to predict. Virtual cells have made advancements enabled by AI, but the perturbations equate to a small fraction of the possible.
This is before you even get into curing disease, especially uniquely human diseases. Brain function involves cellular identities that are unique to humans, meaning animal studies are of little use.
How long does it take to understand what the side effects of chronic administration of a drug in humans are? AI can speed up analyzing the data, or designing the trial or even getting it approved…but it can’t make a 6 month trial complete in 3 months.
I think its important to point out that significant advancements can be made in AI that contribute to biology, while also being realistic about how much it speeds up something as complex as curing a specific cancer or diseases of the brain.
Nicholas Larus-Stone retweeted
The exploits of the last few days are a good reminder of why we use SaaS. Our company pays for ~20 SaaS services. If we self-hosted them, we’d now be responsible for all of the security patches this week. Instead, our small monthly bill pays for their teams to handle everything.