tinker interested in distributed systems, compilers, and sometimes philosophy. eng @datadoghq, previously @awscloud @winglangio. he/him
New York, NY
Joined September 2015
- Tweets2K
- Following1.2K
- Followers433
- Likes17.7K
Chris Rybicki retweeted
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
As a toy weekend side project, I built a small interactive website that visualizes the famous three-body problem in physics. It lets you play with a number of initial configurations, and even edit the initial settings if you want!
rybicki.io/three-body-lab/
Enjoy :-)
"Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth, author of The Art of Computer Programming
Chris Rybicki retweeted
Stoked to be presenting to #rustnyc in two weeks to talk about Rust in the LLM age. Signups open -- with thanks to @datadoghq for hosting!
meetup.com/rust-nyc/events/3…
Chris Rybicki retweeted
People assume that in an AI doing formal verification world, they can take their existing codebase, ask AI to prove it correct, and get either a proof or a bug.
Sadly reality has a few bones to pick with this, and one of them is the "existing codebase" part. And that's due to the good ol' halting problem.
The short of it is that there is no general purpose algorithm that takes any program and input and correctly determines whether it will terminate or run forever. With a few mathematical flourishes we can extend that to "Rice's theorem" and prove this is true for any nontrivial semantic property, meaning we can't automatically prove arbitrary things about arbitrary programs.
Now obviously we can prove some things about some programs, or else formal verification wouldn't exist as a field. The trick is that we write the code in a style suitable for verification. One example of this is totality: in Lean, all functions must be guaranteed to return a value for all inputs. If we want some code that runs forever (like f.ex a server), instead of `while true` we do `while (i < 10^60)` so that it's guaranteed to terminate and return a value (after the end of the universe).
(Totality is also why, in Lean, `1/0 == 0`. If you convert your code to Lean without accounting for this, you could have correct lean but incorrect code!)
Another "style" we use is state machines. Even if your code doesn't quite fit an SM architecture, if you want to prove it at scale, you're probably going to go with a state machine. State machines are the carcinisation of provable code.
Hopefully by now the problem is clearer: even if your codebase is objectively correct, it might not be in the right shape to be proven correct!
Okay, just have the AI rewrite your codebase into another form, right? Two problems. One: you don't know if you've introduced errors in the rewrite because you can't prove them equivalent (since one of them is not in a provable shape). Two: making code more verifiable can make it worse on other metrics, like modifiability, performance, and complexity. And now you've got the old bugbear of software engineers, the dreaded tradeoff.
Now, don't get me wrong, formal verification has a huge role in the future of software, but it's not going to be a breezy "prove my Ruby thx" Claude prompt. It's still going to take work and expertise on our parts to prove our code, and to strike the right balance between formal verification and other forms of correctness.
This is one reason I'm so excited to be working at @AntithesisHQ , which tackles a lot of the same properties as proof methods, but verifies them with "deterministic simulation testing" and fuzzing instead of with a proofs. So it's not as thorough and doesn't give you confidence in all inputs, but it automatically works on pretty much any code shape. I've literally grabbed open source projects off Github, told Claude "check properties XYZ in Antithesis thx", and gotten bug reports back. No state machine rewrites needed.
Chris Rybicki retweeted
Introducing gdp-ts: Ghosts of Departed Proofs for TypeScript.
gdp-ts is a library, linter and AI skill for safer API design. Under this contract, sensitive functions require 'proofs' that the caller performed an authorization check.
The typechecker verifies these proofs at compile time, preventing your team and agents from shipping catastrophic security (and other kinds of) bugs.
While these patterns have existed for quite some time, especially in ecosystems like Haskell, ① human code review and ② cognitive and syntactic overhead made these solutions niche.
The situation is now inverted. Agents are writing more code than we can review, and they *thrive* in tight loops with hard constraints that would frustrate us. We see this with the rise of Rust, borrow checker, code aesthetics debate and all.
The README and examples model a real-world Vercel API product constraint: changing the password on a Project requires a proof of a certain role + a certain entitlement. Thanks to Matt Noonan and Ollie Charles for their research in this space.
github.com/rauchg/gdp-ts
Nice!
I believe there are many useful tools left to build for software development in this direction.
Mixing between deterministic algorithms and using agents in between the gaps - agents making lots of small decisions, leveraging how cheap they are to run at scale.
Introducing e2e
The agentic testing framework for any app
~/ npx e2e init
→ Fully Open Source
→ Mix deterministic and agentic APIs
→ Supports web, mobile, and more
→ Bring your own agent and infrastructure
→ Run locally or in CI
tester.army/e2e
Chris Rybicki retweeted
Overdue career update time! I’ve moved to Toronto and will be starting graduate school (MS/PhD) @UofT & @VectorInst, advised by @florian_shkurti! I will remain @ Stack AV until the end of the year to wrap up some projects, after which I’m open for internships!👇
Chris Rybicki retweeted
You guys think they’re ever gonna put up a statue of a dev writing code by hand?
Phenomenal stuff! It really is true: simple, clean API design goes a long way.
It's nice when demos show off use cases, but it's EVEN better when they show off building blocks and primitives. It's like they're showing you a brand new box of LEGO pieces and how they can connect!
New week new demo! Today I want to showcase the CLI to control the @superlogical multiplexer. The CLI is an extremely powerful tool for automation and hooking into other tools like editors, agentic coding tools, and more. And, it's super fun!
Everything you can do in the GUI, you can do via the CLI. Actually, you can do more in the CLI. You can create sessions, run commands, make splits, move things around, change focus, simulate any input device (key/mouse/etc.), wait on scripts, and more.
There are a couple unique parts of Rex I'm showing off here too: our streaming event system that any client can hook into (or script). And the terminal-specific API to retrieve information like a JSON object of the running process.
I hope this demo lets your imagination run free of where you might slot something like this into your existing tools.
this poem captures
the quiet dignity of
on-call engineers
ALT vertical card with a painting of cherry blossom tree branches, with a poem overlayed: "It's not DNS / There's no way it's DNS / It was DNS" attributed to SSBroski
Chris Rybicki retweeted
I conjecture there will be fewer conjectures going forward. Why would you tell the world about an interesting problem you didn’t know how to solve?
data science with blocks (pythonly.org)
ALT screenshot of a code example from Pythonly.org featuring some code expressed in graphical blocks for loading an Iris dataset, creating a scatter plot, and buliding a classifier for it
Chris Rybicki retweeted
Back when I wrote technical articles for the Confluent blog, I had this constant argument with the editorial team:
They hated that I used the same words to describe the same thing. Especially when it meant saying "low latency" 20 times in a 500 word post. They suggested I mix things up with "performant", "fast", "real time", etc.
My opinion is that using different words will be imprecise at best and confusing at worst. Using the same words to describe a concept whenever it comes up, helps readers know that I am still talking about the same thing and not introducing new ideas. For me, this was good writing.
Fast forward 6 years, and I'm using LLM that was clearly trained by the same editors who drove me nuts. And I think the harnesses also penalize models for repeating the same words in the output.
So every single time I need to ask: "You described this part of the design as 'retry path' and the other part as 'drive forward after failure' - do you mean retry in both cases? or is 'drive forward' somehow different from retry?
And no matter how often I say "please use the same terms when describing the same idea", it doesn't help - the LLM remains bound by its training and penalties, and I get to try and decypher what it means each time.
Chris Rybicki retweeted
"The pursuit of excellence does not need justification."
Mind boggling to me that I can make a thing faster and there's always people that ask "but why?" What kind of mentality is that? The pursuit of excellence does not need justification. Also, I find in so many cases, we can't know the impact of an improvement until we do it.
For example, one I've talked about before: Ghostty's high IO throughput has enabled terminal program (emulator and TUI) fuzzing at a speed thats incomparably fast to prior solutions. This has resulted in upstream patches to resolve issues in popular projects like btop, tmux, and more.
Speed enabled that anecdotally example that lifted the tides of adjacent communities that don't rely on Ghostty technology at all. I didn't predict this.
Make things better because they can be better and let the results naturally play out.
