Pinned Tweet
Introducing Crab: a serverless Git remote storage solution for teams working with large files.
Git is still the best collaboration model we have for software: branches, commits, review, rollback, CI, and history.
But modern repositories no longer contain only code.
They contain model weights, datasets, media assets, simulation outputs, build artifacts, checkpoints, design files, and other large binary payloads that can easily reach gigabytes or terabytes.
That creates a painful tradeoff.
Plain Git bloats history.
Git LFS adds another server, endpoint, quota model, and operational surface.
Manual object-storage buckets keep bytes somewhere durable, but they separate the data from Git history and make collaboration harder to reason about.
Crab is built around a simpler idea:
Keep Git as the interface.
Use your cloud object storage as the backend.
Remove the server layer in between.
With Crab, developers keep using familiar Git workflows. Commits, branches, reviews, worktrees, CI, and rollback still behave like Git. Large-file bytes are routed into customer-owned object storage such as S3, GCS, or Azure Blob.
The repository stays lightweight because Git carries metadata and tiny pointer files, while the real payloads live durably in your bucket.
A normal workflow looks like:
crab init
crab add models/
git commit
crab push
Or, where configured, teams can keep using familiar Git commands like `git add` and `git push` through Crab’s Git integration.
Under the hood, Crab uses Git remote-helper and filter-process concepts to make large-file handling feel native. The developer works in Git, while Crab manages chunking, deduplication, upload, checkout, hydration, and storage layout.
The technical model is designed for large binary data.
Large files are split into content-defined chunks, so small edits do not necessarily rewrite the entire file. Crab can identify which chunks are already known and which chunks are new.
Known chunks skip upload.
New chunks upload once.
Chunks are packed into xorbs to reduce duplicate storage, bandwidth, and object request volume.
That matters when teams are moving ML checkpoints, datasets, media, or build artifacts repeatedly across branches and CI jobs.
Clone and checkout are also designed for large repos.
Instead of forcing every user or CI job to materialize terabytes immediately, Crab can clone metadata first. Teams can inspect branches, review code, and start work quickly with lightweight pointer checkout, then hydrate real bytes only when needed.
Access modes are flexible:
crab hydrate downloads selected content.
crab dehydrate frees local disk by returning files to pointer form.
crab mount supports on-demand reads through a virtual filesystem.
CI and desktop workflows can hydrate only the files they actually need.
For enterprise teams, the important part is ownership and control.
Crab does not require a central LFS server or a new database service. Storage stays in your cloud account. Your buckets, IAM, lifecycle rules, encryption posture, and identity model remain the control plane.
Crab is designed to fit enterprise auth and operational requirements: customer-owned buckets, IAM, OIDC, Entra ID, Workload Identity, encrypted local token cache, structured CLI output, and operational commands such as doctor, status, fsck, and gc.
This gives infra teams fewer services to run and gives developers a workflow that still feels like Git.
The business benefits are straightforward:
Less Git bloat.
No separate LFS server to operate.
Large files stay versioned with the repo.
Object storage remains customer-owned.
CI can avoid unnecessary downloads.
Teams can adopt gradually instead of rewriting everything at once.
Costs can be managed with lifecycle policies and storage tiering.
Existing Git habits remain intact.
Crab is especially useful for teams building with:
ML models and checkpoints
Training and evaluation datasets
Media and creative assets
Large generated artifacts
Scientific or simulation outputs
Game assets
Build cache outputs
Enterprise repos that have outgrown normal Git storage
The goal is not to replace Git.
The goal is to let Git keep doing what it is excellent at: collaboration, review, history, and coordination.
Crab turns object storage into an enterprise Git remote for large files.
Git-native large files.
Your cloud.
No server layer.
DM open for early users ;)
Powered by @huggingface xet protocol github.com/huggingface/xet-c…, @criccomini 's SlateDB github.com/slatedb/slatedb, and @theelbasian 's Gitoxide github.com/gitoxidelabs/gito…
Haiping retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
Haiping retweeted
my lighting talk from cloudflare connect on building artifacts in 3 weeks is up!
youtu.be/D_tnYlZe5t8?is=LDkb…
I've been spending a lot of time on Crab v2 and getting pretty excited about the outcome.
Crab v2 allows team to self-host its own Git platform without managing a bunch of infrastructure, just a binary and an object-store bucket.
Under the hood, Crab server runs on Cellar, a customized celld core. Object storage is its only infrastructure dependency. Everything lives there: Git repositories, Capsules (our custom Git pack objects), LTX, Xet xorbs/shards, and LFS objects.
Teams get:
1. Git-compatible remote for any files in large repositories (tested at the scale of the Kubernetes repo, which has a ~1.3GB pack file).
2. GitHub-like UI with issues and pull requests, metadata stored in SQlite per repository cell.
3. Xet-powered chunk dedup for versioned large files, or simply store large files in Git LFS-compatible storage backed by the same bucket.
4. S3 Gateway that exposes Git-tracked files through the standard S3 API. This opens up the entire S3 tooling ecosystem to Git repositories. Think lakeFS-like capabilities, but Git-native.
No external database. No complex infrastructure to manage.
I'm aiming to release Crab v2 in the next few weeks, fully open source under Apache 2.0.
3
8
409
Haiping retweeted
Today we're launching an exciting new project: Highlights
Highlights is a syntax highlighter that outputs HTML. It improves on both speed and correctness compared to the existing industry standard libraries.
It's an optional drop-in replacement in Diffs, and it's ready to use today. In fact, it's already powering diffshub․com!
Check it out at diffs.com/highlights
OR `pnpm install @pierre/highlights`
Haiping retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Haiping retweeted
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
I believe in Cloudflare. Building with Durable Objects and Artifacts has been a lot of fun, almost ready to ship.
Haiping retweeted
celld v0.6.2 released with bug fixes.
We are considering celld to be in beta release now. Please report issues if you encounter them!
github.com/denoland/celld/re…
It's a GitHub implementation on Durable Objects, still a baby but most of the functionalities are already working, powered by Cloudflare (Artifacts/D1/R2/Worker/Container/Agents etc), it's a great DX to build this prototype in a weekend, should I continue polishing and submit to Git competition cloudflare.com/git-competiti…? @CloudflareDev
Haiping retweeted
Over the next 28 days, each day we’ll either ship one thing that is a clear improvement and relevant for most codex/work users or ship a full reset. Let the improvements begin.
Haiping retweeted
i feel like people do not grok how important celld is.
it's scalable applications of arbitrary complexity. the only dependency is s3.
once you see it, you'll realize there's no point to traditional javascript runtimes in serving applications - only for build processes (eg bundling) and scripting.
the networking and scaleability are deeply rooted not only in the distributed runtime (celld), but in the programming model itself (largely kenton's design)
I wasn't quite sure when I started celld how well this could be made to work, but we've been running it in bigger capacities and getting user reports over the last months. Actually handles load surprisingly well. (any experience in the comments to share?)
because the programming model does not expose the underlying linux system, you can actually think of celld as a distributed operating system - running in user space of linux. it only runs one program / doesn't support multi-tenant, but it has system services like an operating system. it's all about networking - i like to think of it as the practical modern version of plan9 or even erlang beam.
I became a big fan of cell-based architecture while working on S3 at AWS.
At S3 scale, you can’t treat the entire service as one giant system. You partition it into cells, give each cell its own capacity(data plane) and clear ownership(control plane), and contain failures so a problem in one cell doesn’t bring down everything else.
Cellularization brings several important benefits:
1. Scale by adding cells
2. Recover and operate cells independently
3. Limit the blast radius of failures
4. Make capacity and ownership more explicit
But it also introduces real engineering challenges: routing requests to the right cell, balancing load across cells, moving or rebalancing data, and scaling cells up and down safely.
Now I’m building my own cell-based runtime Cellule - github.com/crabbuild/cellule - around the same ideas, heavily inspired by celld.dev and rivet.dev, but with a much smaller scope.
Cellule is a Rust library that can be embedded directly into a cell-based application with minimal dependencies—ideally, just an object store. It provides the basic building blocks for:
- Key-value storage
- SQL
- Blob
- Queues
- Workflows
- Schedulers
- Effects for cross-cell coordination
Applications built on top of it should be deployable as a single binary, keeping operations and deployment as simple as possible.
I’ve already built a few very different systems on top of Cellule, including a DynamoDB-compatible database server and a Git hosting server. crab-http-server is being built on it too.
I’ll write about these examples, and what worked and didn’t, in a follow-up blog post.
I’ve been reading both Rivet and Deno’s celld while rolling out my own cell-based runtime for crab-http-server.
They share the same basic idea:
Disposable compute + single owner + SQLite + Object Store
But the storage model is quite different.
Rivet rivet.dev/
Actor → SQLite → custom VFS → hot storage → S3
Celld celld.dev/
Cell → local SQLite → LTX → followers → object storage
Rivet virtualizes SQLite storage.
Celld keeps SQLite local and replicates the changes.
Same goal, very different implementation.
🤖 Made with AI
Some feedback for @XiaomiMiMo on MiMo-V2.6-Pro:
It doesn't seem usable for long-running coding tasks yet.
I burned through ~85% of a Standard Monthly Plan ($16/month) using Codex across just 3 tasks.
The biggest issue wasn't the cost — it was that on longer tasks, the model eventually fell into loops and kept generating essentially the same sentences/actions repeatedly, consuming a huge amount of the quota without making meaningful progress.
For a model positioned for long-horizon agentic/coding workloads, this failure mode is particularly painful because one bad loop can burn a significant portion of the monthly plan.
Would love to see:
1. stronger loop/repetition detection
2. automatic termination when progress stalls
3. better context management for long-running tasks
4. safeguards against burning large amounts of quota while stuck
Happy to provide logs/traces if they would help debug this.
2
2
432
Haiping retweeted
GPT-6 Sol and Luna are out. Not only are they a very significant improvement across the board, but also in writing and general "you know when you try it" quality.
We are also permanently reducing the API price by 50% making both of them viable for a ton of new usecases and making your usage go further too, even on the subscriptions.
And one more thing. We are loading a banked reset into all accounts of our Plus, Pro and Business users. Let's go!
openai.com/index/introducing…
Haiping retweeted
Grok 4.7 is here.
It's a notable improvement over Grok 4.6 at the same price and speed.
Compass 0.3.28 is out.
We tightened agent-facing graph queries, kept usage evidence tied to source, improved CompassQL ordering, and made path guidance clearer.
Installers for macOS, Linux, and Windows:
github.com/crabbuild/compass…
1
1
280
























