
AI Education | Developer Advocate @huggingface
EU
Joined November 2022
- Tweets1.2K
- Following588
- Followers4.6K
- Likes3.7K
here's the video summary of this video on DSH. it compresses the full lecture to 4 minutes.
i still recommend watching the full lecture, but this ai video might help you get started.
ofc, i made sure all the contents of the summary represent the lecture faithfully.
Alejandro AO 🤗 retweeted
You asked for dsh, I added it
DeepSeek Harness is the open-source coding agent by @deepseek_ai. It's now on the @huggingface Harness Leaderboard, which compares ten coding agents on the same open models.
DSH is statistically tied for first in all five evaluations: DeepSeek-V4.1-Flash and GLM-5.3-Flash at low and high effort, and Flint.
It shines on DeepSWE, where agents change code in open-source repositories: 2nd of 10 averaged over the three models, 0.5 points behind Hermes Agent by @NousResearch. On Terminal-Bench's command-line tasks it ranks 6th.
One habit explains much of that gap: when DSH finishes a task, it shuts down every background program it started.
Three Terminal-Bench tasks ask the agent to start a server, for example a model inference API. The grader tests that server after the agent has finished. With DSH, the server is gone by then.
On DeepSeek-V4.1-Flash and GLM-5.3-Flash, DSH passed 2 of 36 attempts on these tasks. @opencode, @pidotdev and Hermes Agent passed all 36.
Those three tasks cost DSH 1.5 to 1.7 points in every evaluation. On DeepSeek-V4.1-Flash at high effort, that is more than its 1.3-point gap to the leader.
Developer preview: 0.2.2-rc.1. Web search is off because it needs a DeepSeek account key.
It costs $0.53 per solved trial, averaged over the three models.
Leaderboard: huggingface.co/spaces/joelni…
Alejandro AO 🤗 retweeted
Everything that you need to know about decision models, in one place.
> what are decision models
> transformers integration
> trl support
> llama cpp support
huggingface.co/blog/decision
DeepSeek's DSH harness is built on a simple idea: everything is a plugin. Including the agent loop itself.
In this tutorial I break down cordis, the plugin system behind it:
> the architecture of dsh
> services, dependency injection, and events
> building a real plugin live
Thread 👇
The cordis.patch.yml file is where plugins get wired in.
Patch layers build on top of each other (like .gitignore files):
1. base bundle (cordis.yml)
2. your active profile, e.g. web or desktop
3. your user config at ~/.dsh/web (or desktop)/cordis.patch.yml.
The last layer always wins, and config properties flow into the plugins they target.
1
3
391
Thanks @joelniklaus for burning this compute for all of us and sharing the learnings openly. Cost per task is a critical metric for our team so your results make us feel warm and fuzzy
I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard.
Which harness should you run your open model in? I ran 28,849 trials to find out.
The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model, provider and reasoning effort stay fixed, and every task gets three attempts.
The trials ran GLM-5.3-Flash, DeepSeek-V4.1-Flash and Flint on Terminal-Bench 2.1 and DeepSWE, for $4,746 of inference.
Averaged over the three models, mini-swe-agent (@KLieret), a bash-only agent, solves 51.4% of tasks and @openclaw 27.3%. Four others are statistically tied with mini-swe-agent.
One of them is Pi (@pidotdev), at 49.4% for $0.29 per solved task, about 45% less than mini-swe-agent's $0.53.
The best harness depends on the model: mini-swe-agent on GLM-5.3-Flash, Hermes Agent (@NousResearch) on DeepSeek-V4.1-Flash and Kimi Code (@Kimi_Moonshot) on Flint.
@opencode costs $3.35 per solved task, 11.7 times as much as Pi.
See Harness Hub's Harness Lens for what the nine harnesses do differently. It documents the prompts, tools and compaction of 34 coding-agent harnesses and links every claim to its source.
Leaderboard: huggingface.co/spaces/joelni…
Harness Lens: huggingface.co/spaces/harnes…
Crazy that this is free. I come from a SWE background. HF's educational material has always been a blessing and helped me get closer to the intersection of SWE and ML.
In the handbook:
> Train your first MoE
> Tune routing & load balancing
> Debug training & GPU bottlenecks
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
Alejandro AO 🤗 retweeted
frens at Hugging Face tell me llama app (llama.cpp) has a new home: @Llama_app
Follow & spread the word!! 🤗🦙
i had a blast at the NEXXT AI Summit in Bucharest last week.
i was at a panel interview about how to delegate work to agents. interesting because half of the audience were non-technical, business users of claude code.
i also had a booth there. i was able to talk to a lot of interesting people about open models, local AI, and Reachy Mini, who accompanied me during the event.
lots of fun 🤗
8
19
705
Alejandro AO 🤗 retweeted
Just a reminder that DeepSeek V4.1 Flash 1017, GLM 5.3 Flash, Qwen 3.8 Next Flash, and even Qwen 3.8 27B are all outperforming (in both intelligence and capabilities) every model that was considered "frontier intelligence" in Xmas 2025 (just 11 months ago)
Opensource AI is on fire
I don't think people realize just how much of an engineering marble DeepSeek V4.1 Flash is, because guess how much RAM you need for 1M context window? For reference Llama 3.3 70B would need around 320gb of RAM.
Qwen 3 8B around 144gb of RAM.
DSV4.1F? Less than 0.9gb.













