Jack Diederich (@jackdied) header
Jack Diederich (@jackdied) profile picture

python-dev, PSF fellow ermeritus, bowler, con speaker, stories, zone-6 farmer. Sailing around the world in a dirty gondola.

Joined April 2009
Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)Photo by Jack Diederich (@jackdied)
Jack Diederich retweeted
today, i'm proud to unveil some of archil's deep research results: tam-bench based on what our researchers are telling me, we're seeing a doubling of companies building "file systems for AI agents" every 3.3 months. at this rate, we project that we will hit "sandbox-scale" (300 companies working on file systems) by just february 2028. please join me in calling for a pacing of the frontier, while we assess how safe and powerful these file systems truly are
10
3
57
3,101
Jack Diederich retweeted
I had Opus 5.5 formalize CPython in Lean: 613,318 lines of Lean that copy Python's internals line by line, tested against the real interpreter and checked by Lean. On the way it found real bugs in Python, including crashes from plain Python code. 🧵
64
230
34
2,494
164,040
Jack Diederich retweeted
i'm sorry to do this again. tl;dr -- archil hosts spark for 3x less and 3x faster than databricks to reiterate: the problem with commercial open source is that organizations did this because they *assumed* it would be much much harder to self-host than run on their cloud. why was it so much harder? the *data infrastructure* required to run this kind of cluster was extremely difficult to build. archil has made it very very simple to get high-performance shared storage across lots of nodes, therefore making data analytics cheaper and simpler than before. our seed pitch deck 2 years ago specifically called out spark as one of the workloads that we wanted to optimize for, and it turns out that we've done that now! check out the guide to run serverless spark on archil here: docs.archil.com/guides/analy…
tl;dr -- archil hosts clickhouse for 50% less than clickhouse. traditionally, "commercial open source" meant that the provider open-sourced a worse version of the product, to disincentivize people from self hosting. frequently (as with clickhouse here), the OSS version requires local disks and doesn't work with s3. unfortunately for these folks, archil has solved the problem of using s3 as a local disk. so, you can use open-source clickhouse with s3-backed data and save 50% of your clickhouse costs DM me if you want some credits to move your clickhouse workload to Archil guide here: docs.archil.com/guides/analy…
6
3
102
7,918
Jack Diederich retweeted
armin is on austrian tv. dying.
42
6
2
660
27,975
Jack Diederich retweeted
An example in the hour since I tweeted that: I told it to fix some DATABASE_URL management. It added `if`s directly inside NPM scripts, and a conditional node invocation in CI running an inline JS script. About 20 hunks in the diff. After I corrected it: +0 lines, +1 word.
3
2
1
86
4,290
Jack Diederich retweeted
Most adults remember where they were on 9/11, how they felt, and how it changed their lives. I was a Captain in the National Guard, and it refocused the whole Army. I have two adult children, one of whom wasn't even born by then.
1
1
1
1
321
Jack Diederich retweeted
Those of us who do remember have an obligation to not just remember, but to pass on what we learned. There's a new documentary, 911remembered.com, that released yesterday. The website has showtimes for stations it's on this weekend, starting yesterday.
1
1
64
Jack Diederich retweeted
Super stoked to announce @pycon videos are out! Including mine! Wherein I speedrun all the things @claudeai got wrong about Python in the browser.
2
3
13
626
Jack Diederich retweeted
He could be talking about bowling also!!
1
3
578
Jack Diederich retweeted
It’s pretty clear nobody in @awscloud leadership has ever seen a customer’s bill. If they had, it’d have ads on it by now.
10
4
215
14,169
Jack Diederich retweeted
At Archil, we have the privilege of working with nearly all of the teams building frontier agents at scale. One thing that we heard over and over again: is grep really the right tool to search over my file system data as it grows? Today, we're answering that question with a resounding yes by releasing Archil grep. Archil grep makes it simple to take advantage of massive map-reduce compute to search your data, without needing to configure external systems, indexes, ETL, etc. Give your agents `disk.grep` to 10x their search performance by only changing a single line of code.
Today, we're speeding up ripgrep by 10x+ for agents built using Archil disks with our launch of "Archil grep". We're exposing Archil grep as a new function in our Typescript SDK that you can hand to your agents. Under the hood, it's a powerful map-reduce system that allows Archil to partition your data sets, and spin up concurrent sandboxes to perform massive, parallel search. This launch finally alleviates the concern that many agent builders have: how should my agent performantly search my data as it continues to grow in size? Watch the demo below to watch us run Archil grep on 100,000 files 10x faster than ripgrep can. Give it to your agents today with the latest release of the "disk" SDK.
1
1
32
8,387
Jack Diederich retweeted
LLMs ruined Chesterton's Fence before agents: q: 'why is this random fence with no obvious purpose here?' a: 'someone had problem, this was solution' after agents: q: 'why is this random fence with no obvious purpose here?' a: 'agent put it here randomly for no purpose'
1
9
20
3,063
Jack Diederich retweeted
It's not rocket science, but command line interfaces matter. I'm looking forward to presenting "How to Write a Great Command Line Application" at #pycon on Friday, 12:30p. us.pycon.org/2026/schedule/p…
1
2
141
Jack Diederich retweeted
today, I'm so excited for the entire Archil team to turn the cloud on its head. developers and agents don't care about CPUs, they care about (a) where their data is stored and (b) how they can do something with it. with Serverless Execution, we're enabling users to work with their data directly where it's stored to radically simplify how they build systems.
Introducing serverless execution. You can now give agents a bash tool that supports running bash directly on its context, without needing to manage a separate lifecycle for sandboxes. Generally available now to all file systems in AWS regions. docs.archil.com/concepts/ser…
4
1
52
8,615
Jack Diederich retweeted
I put the slides from our @aiDotEngineer talk up.
8
12
3
208
15,251
Jack Diederich retweeted
shout it from the rooftops! the cloud sold us all a lie: that networked block storage (~EBS) is the easiest way to run stateful software like postgres. this is the worst of all worlds! if you care deeply about latency, you should use local NVMe for everything (like PlanetScale metal). if you care deeply about flexibility, well, guess what. block storage doesn't easily scale up or down. worse: block storage doesn't know ANYTHING about what you're storing, so you get into a situation (like liz describes) where you're paying a distributed consensus cost for something that might not matter. we actually founded Archil on the thesis that networked block storage was the enemy. why is file storage better? you can bake elasticity into the protocol, like S3, and only charge people for what their using. you can get semantic information from the customers about their goals and optimize storage on the backend. you can provide services on top of the storage (ripgrep on the server-side?) to make writing applications easier. distributed file storage is the clear winner. why hasn't every cloud embraced distributed file storage? simple. it's INCREDIBLY HARD to build. there are basically two *real* implementations of elastic file storage in the cloud. there's Amazon EFS (where I worked for a long time), and there's Oracle FSS (built by the original architect of Amazon EFS). that's it. neither of these really managed to capture the magic of what makes distributed file storage special, so we're trying to change that. networked block storage is simple to build, easy to reason about, and high-margin, so cloud companies are incentivized to stick to this local maxima (or, for reliability purposes, just keep using it -- looking at you, Fly.io sprites). there is so much more that storage in the cloud can do for us, and people aren't demanding it because they just don't know what's possible.
i think its sad how “the cloud” has made large portions of the internet run on horrifically slow single-tenant annoying to scale block storage if you pay the (very high) cost of disaggregating storage and compute, you should at least get a distributed file system out of it!
7
6
4
101
23,115
Jack Diederich retweeted
no time to read the supabase post-mortem? let's walk through it together. i've picked apart hundreds of post-mortems over the past decade. some surprising learnings here, and some things i would have clicked on internally. this is the kind of outage that could definitely hit anyone. my big takeaway: i would have expected AWS to make this super dangerous button harder to click, and have more alerting when a large customer like supabase accidentally clicked it #hugops to @supabase
Replying to @jhleath
Definitely some outages that are “ok these people don’t know dist systems, auth, etc” Idk the details about this one tho
1
1
1
31
6,023
Jack Diederich retweeted
this platform is… something. but many of my important friends are still here, so please see (good! not bad!): pyfound.blogspot.com/2026/01… bsky.app/profile/durbin.ee/p… mastodon.social/@EWDurbin/11…
2
1
7
613
Jack Diederich retweeted
In a full circle moment from my time at AWS, I got a chance to sit down with @QuinnyPig and talk through the problems that @archildata is solving for customers. We get to talk in detail about what this means for AWS customers, and (if you need a reprieve) I promise that we hardly even talk about AI. Check it out here: youtube.com/watch?v=rMcZ0KtR…
4
14
6,438