🛠️ Building data infra for AI/ML. Ex-Data Scientist @Microsoft. Created DVC, now DataChain. PhD in CS. Serious about data. Less serious about everything else.
San Francisco, CA
Joined October 2011
- Tweets1.3K
- Following571
- Followers2.3K
- Likes4.3K
Physical AI data breaks the usual agent loop.
Tomorrow: Claude Code over raw video, metadata, and expensive compute in S3.
Live demo: 8am San Francisco / 4pm London / 5pm CET
Join us: luma.com/3jjb56bx
Claude Code and Codex were built for weightless data.
Physical AI data has mass.
My talk on the other physics of agents:
youtube.com/watch?v=bUJgirn4…
Going deeper this Thursday - with a live Claude Code demo over raw Physical AI data in S3.
8am San Francisco / 11am NY / 4pm London / 5pm CET
luma.com/3jjb56bx
This made me check my own citations. Turns out DVC has 40+ citations which is more than my PhD research
Apparently, open source is a better academic strategy than academia 😄
@shcheklein @rkuprieiev @isidentical congrats, we’re academics now. Who knew?
You don’t have to publish papers to have academic citations or even an h-index.
I dropped out of uni years ago 🤦♂️, and always wanted to contribute in different ways: open-source libraries, datasets, benchmarks, and overly long blog posts. I finally created a Google Scholar profile and started linking my GitHub repositories and articles.
It turns out some academics are kind enough to cite good ideas regardless of where they come from 🫶
Scholar’s attribution machinery is hilariously basic: it split citations between GitHub and Zenodo versions, missed entire projects, and failed to connect several papers to the software or articles they explicitly reference. Nevertheless, I have already amassed a monumental h-index of 3 😂
Open-source devs, your software may already be part of the academic literature. You might even have an h-index without knowing it. Go check Google Scholar, you may be pleasantly surprised!
P.S. Geoffrey Hinton’s h-index is 192. Only 189 to go.
Coding-agent intuition breaks down on physical-world data
That’s my AI Engineer World’s Fair Online Track talk this week.
See you there! I’ll be the guy in the Villain-Con shirt 😄
Call for Proposals for AGNTCon + MCPCon North America 2026 is open on Sessionize and I've just submitted a session! sessionize.com/agntcon-mcpco…
Talked about data agents beyond SQL at @pydatalondon today.
Files, video, sensors, metadata, Python workflows - all the messy stuff where real data work happens.
Thanks everyone who came by 🙌
Excited to speak at @PyData London this Sunday!
I’ll be talking about data context for agents beyond SQL: video, sensors, files, metadata, and Python workflows.
10:15 · Hardwick Hub Conference Room
Come say hi!
Tomorrow, PyData London 2026 begins. If you have your ticket, we'll see you there. If you don't have yours yet — there's still time: hubs.la/Q04jpJ3R0
- - -
PyData London 2026:
Convene Sancroft, St. Paul's
Tutorials: Friday, June 5-7
Talks: Saturday and Sunday, June 6-7
Keynote Speakers all 3 days
Tickets: hubs.la/Q04jpJ3R0
Full side-by-side comparison table with more details: datachain.ai/blog/openai-ant…
Opus 4.8 is really smart!
It spent 38 minutes implementing a pretty complex feature, proved it worked, then reverted it with a detailed explanation.
So, like humans 🤷♂️
3am thought: pi.dev tonight. Pretty solid, and you pick your model GPT/Opus/DeepSeek.
The harness category is maturing into "bring your own model" mode. The harness IS the product now.
Every dev-experience problem ends-up generating and executing some code, and all code is moving under a harness (Claude Code, Cursor, Codex), leaving people with text.
Data platforms are next! And that's what we need to build.
🚀Code as Agent Harness: A survey work from UIUC, Stanford, and Meta.
📄arxiv.org/abs/2605.18747
Code is no longer just the output of AI.
It is becoming the executable, inspectable, and stateful substrate through which AI agents reason, act, verify, remember, and self-correct over long horizons.
In our new survey, we examine this shift through the lens of Code as Agent Harness, focusing on how code serves as:
• 🧠 Harness Interface: coding for reasoning, acting, and environment modeling
• ⚙️ Harness Mechanisms: planning, memory, tool use, feedback, and optimization
• 🤝 Multi-Agent Harnesses: collaboration through shared code, tests, and execution traces
We review applications spanning:
💻 Coding Agents
🖥️ GUI/OS Agents
🤖 Embodied Agents
🔬 Scientific Discovery
🏢 Enterprise Workflows
If you find this survey helpful, feel free to explore our resource collection below.
🤗 Hugging Face Daily: huggingface.co/papers/2605.1…
💻 GitHub: github.com/YennNing/Awesome-…
🌍 Website: code-as-harness.github.io/co…
Feedback, suggestions, and community contributions are warmly welcome!
#AI #Agents #LLM #Coding #AgenticAI #SoftwareEngineering
I just started a Substack! You can subscribe to it here dmpetrov.substack.com/?r=6ix…
Turns out "Claude Code over files in S3" quickly becomes "rebuild half the data warehouse stack" 🫠
Schemas, datasets, lineage, file refs, etc.
OpenAI's Data Agent post made us feel slightly less insane 😄
Read more: datachain.ai/blog/openai-dat…
Milla Jovovich releasing a agent memory framework wasn’t on my 2026 bingo card
the Fifth Element but it stores everything instead of summarizing
