Luis Capelo (@luiscape) header
Luis Capelo (@luiscape) profile picture

Interested in deep learning research and applied AI systems. Building serverless at @modal

New York, NY
Joined May 2009
Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)Photo by Luis Capelo (@luiscape)
We're launching our new Sandbox Events UI to make it easier to see when commands were exec’d, how long they ran, were they successful, etc. And it’s not just execs - we’re showing the same info for TCP connections over Tunnels and HTTP requests with Sandbox Connect Tokens. Here’s a timeline of execs and HTTP requests to a Sandbox running an RSI eval:
6
13
4
78
5,712
Luis Capelo retweeted
A typical sandbox spends more than half its life using less than 6% of its peak CPU. At Modal, we see that pattern for coding agents, RL rollouts, and evals. Reserve for the peak, and most of what you pay for sits idle. Modal Sandboxes burst instead, and you pay for what they use.
11
5
8
200
18,501
Luis Capelo retweeted
Can a satellite search forever for an island that never existed? What does an image model imagine that words can’t describe? What does a creek 150 miles away feel like? What happens when 1,000 agents share a world with no instructions? 11 artists set out to find answers to these questions and more for Scaling Intelligence, an exhibition with @grayareaorg with work built on Modal.
2
5
1
25
2,914
Luis Capelo retweeted
Modal Endpoints are now available in @NousResearch Hermes Agent hermes plugins install modal
11
11
4
130
26,303
Luis Capelo retweeted
We’re joining the SGLang consortium as a founding member! @sgl_project has been the foundation for some of our most demanding inference workloads, helping us to serve trillions of tokens. Along the way, we’ve shared the improvements from our own deployments for everyone to build on. We’ve bet on open source since the beginning, and we want more engineers to be able to do the same. We’re excited to bring what we’ve learned in production to the consortium.
Introducing the SGLang Consortium at SGLang Summit. @nvidia , @AMD, @Google, @Meta, @intel, @Cloudflare, @modal, and @radixark are joining SGLang’s maintainers to help shape the project’s roadmap and advance open-source AI inference. @sgl_project runs on hundreds of thousands of GPUs and powers some of the largest production inference deployments in the world. The consortium will turn production needs into shared engineering work: supporting frontier models, improving reasoning and multimodal workloads, scaling distributed inference, and optimizing across hardware platforms.
3
8
98
4,103
Luis Capelo retweeted
Super excited about this! I spent 6+ years on gVisor at Google, then recently joined Modal. It’s been an adventure pushing the frontier of gVisor’s capabilities and performance here. @modal is one of the most advanced gVisor users, running 40M+ containers a day, so we care a lot about where the project goes. Glad to see it move to CNCF, and happy to be back as a maintainer.
gVisor is joining the @CloudNativeFdn, and Modal is becoming one of its maintainers. We’ve run gVisor on Modal since day one. Along the way we've upstreamed: GPU support, multi-node training, snapshots, and more. All of it open-source. We’re incredibly proud to grow from being a team that depends on gVisor to a team that also gets to help steer it.
12
16
3
237
30,488
Luis Capelo retweeted
Let Gemini drive a @modal sandbox, or a million. Great working with the Deepmind team on this.
building with Managed Agents on the Gemini API? now you can bring your own sandbox with self-hosted environments start building today: ai.studio
3
34
1,375
Luis Capelo retweeted
ever wanted to hear what an LLM sounds like? turns out, they’re pretty good at controlling the synth models manually! here’s Ave Maria in 4-part harmony built with Kimi K3.1 Pro on @modal
my vocal synth models the vocal tract as a network of branching acoustic tubes, unlike vocaloid which stitches together and pitch-shifts phoneme recordings. still working on improving naturalness by tuning it against recordings of real singers, but I'm already impressed at how cool it sounds!
9
7
1
51
3,311
Luis Capelo retweeted
Compute shouldn’t be the bottleneck for your next big idea. We're launching Modal for Builders, a program that provides compute credits to builders working on ambitious AI projects. Have a demo you’ve been wanting to build? A research question you want to explore? Or another idea that needs compute? We’d like to hear about it. Apply with your idea and an estimate of the compute you’ll need. Projects may also be featured on Modal’s social channels. Some projects we’ve backed in the past:
9
41
16
378
49,878
Luis Capelo retweeted
I’m back at @modal Working at @thinkymachines was really something. I’ll miss my coworkers, and a lot has happened since I joined last year - all sorts of ML and production systems, witnessing what goes into a model, throwing new kernels at training and inference, scaling research infra, making interaction models fast and good, wrangling RL dynamics. It was also my second job, so perhaps a formative experience. When I joined, I discovered I was the youngest at the company (like 60 people) and felt pretty out-of-place, surrounded by ex-OpenAI/Meta/GDM/Mistral folks. Anyway, the company reinvented itself many times, and I think with the recent model releases we can see it’s on a good path. Thanks to @cHHillee for originally recruiting me to Thinky and my boss and various coworkers (whom I won’t dox) for teaching me very much I saw this sunset rainbow the other day in NYC, and it’s the only time aside from July 4 I’ve seen everyone on the streets stop and stare at the sky for a bit, together. It was brief. This won’t last forever. Anyway, hope to have more for you soon… much to build! The Modal team is very strong. I’m glad to be here.
68
24
13
1,775
124,173
Luis Capelo retweeted
0-100k Minecraft agents in 60s! DM me if you want your limits raised to build something like this.
you called? @antonosika 100K agents playing Minecraft in 60 seconds on @modal
3
8
1
116
13,842
Luis Capelo retweeted
Kimi K3.1 Pro from @Kimi_Moonshot is now available on Modal. A major post-training update to Kimi K3, with expanded reinforcement learning for stronger coding, financial research, and long-horizon agentic work. Kimi K3.1 Pro supports text and images with 1M context.
9
18
5
98
10,687
I'm joining @modal! Software is undergoing a revolution; we need new tools and infrastructure for this new era. @modal is the platform I want in these times, so I decided to help build it. If you want to come build it with me: modal.jobs and my DMs are open!
📢 ADSP Episode 310 is out! 📢 In this episode, @blelbach and @code_report chat about Bryce leaving @nvidia, where he is headed next and more! 🥳 adspthepodcast.com/2026/10/3…
56
22
4
615
30,572
Luis Capelo retweeted
Alright I finally have more to share on my personal project! Resin is a runtime for building fast LLM inference engines. The idea is to make all the generic parts of inference reusable, and leave only the model-specific forward pass + kernels to be written for each architecture / hardware target. To build an engine: 1. Import Resin's core. 2. Have an agent write a forward pass for the model you’re interested in, for your specific hardware (GPU or TPU or AMD), any kernels you want. Then, boom, you have a standalone inference engine. — I've been hacking over the past month (early post: nitter.cf/ekzhang1/status/208950…), and recently this week I got a lot of features + parallelisms working, inspired by DeepInfer. For DeepSeek-V4.1-Flash, it's nearly as fast as DeepInfer (within 5-10%), while being self-contained and supporting other models, plus metrics + HTTP server + prefix cache. Yesterday I wrote a skill for authoring models and put Codex to work on it overnight. The agent built a GLM-5.3-Flash engine with MTP, reaching up to 60,000 input tok/s and 16,000 output tok/s on B200 (EP4). So far inference "vibe coding" has not been very reliable or successful. I think it's because people start from scratch each time and reinvent lots of tricky things on the scheduler and server layers. But with a shared core, we can automate lots of inference work. The tricky bit is writing a fast forward pass. But Resin turns that into a standalone engine by just importing a library, and it becomes a verifiable goal for the AI. Code: github.com/ekzhang/resin
So… I have it working now. The core of an inference engine in Rust is about 3k lines (scheduling, kv/swa/recurrent state, prefix cache, multimodal, dynamic batching) I then asked Codex to implement (with MTP, cudagraphs, TP on Blackwell): - DeepSeekV4 Flash - Gemma 3.8 27B - GPT-OSS - Inkling It turns out that most models take a lot of code to run with fast, persistent graphable kernels and so on… on the order of 2.5k for a simple model (gpt-oss) and up to 7k for a monster like DeepSeekV4 Flash that has mHC, CSA, HCA and DSpark Not sure if I’ll continue with this, but it was fun! The simple core is reusable, and startup times are crazy fast for an inference engine with no tech debt and not having to serve a ton of dynamic configurations - just one model forward pass in nvfp4 :’) - maybe one day SGLang/vLLM will be modular like this. See interactivity-throughput below for Inkling-Small TP=1 after a few sconv kernel fusions. (This is without overlap scheduling btw! Don’t even really need it when your core + scheduler is small and Rust.)
115
105
33
1,515
105,555
Luis Capelo retweeted
We worked with @SakanaAILabs to optimize their Sakana Namazu API down to the kernel level: → Selective NVFP4 quantization (23k+ experts) → DFlash draft model trained on raw token streams (+25% acceptance) → In-house prefix KV routing + serverless autoscaling
10
19
6
119
16,210
Luis Capelo retweeted
Congrats to @GradiumAI on launching Phonon! We built a voice agent with their new TTS models and Pipecat on Modal to put them to work. Read more about Phonon and check out the demo: gradium.ai/phonon
A new frontier for on-device speech. We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date. Phonon-31m is available under Apache 2.0. gradium.ai/phonon🧵
2
5
42
2,598
Luis Capelo retweeted
We got our hands on @nvidia's new Vera CPU and put it to the test. We built a benchmark using real-world agentic workloads to see how it performs under contention. In early tests, Vera delivered up to 2x the throughput of comparable x86 hardware, maintaining low latency even as hundreds of concurrent workloads competed for resources. For Modal Sandboxes, that could mean running more agents in parallel without sacrificing latency.
29
26
18
222
27,957
Luis Capelo retweeted
The secret to Modal's speed is our custom lazily loaded container filesystem, ImageFS. ImageFS is built on a massive tiered distributed filesystem, layering local NVMe disks, regional and p2p caches, and global blob stores. When we launched VM Sandboxes this month, we wanted to make sure they could take advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.
14
30
5
542
53,134
Luis Capelo retweeted
Nemotron 3.5 Super VL from @nvidia is live on Modal today with 120B params for visual reasoning and agentic work across images, docs, and UIs. Hear more about the thinking behind the Nemotron family: At Runtime, @ctnzr explained why “efficiency is also intelligence,” and how NVIDIA pre-trained Nemotron 3 Ultra entirely in FP4.
1
11
36
2,926