
ML researcher. Multimodal, foundations @cohere
Joined May 2014
- Tweets272
- Following645
- Followers2.3K
- Likes689
Pinned Tweet
Happy to share what I've been working on recently: today we release Cohere Transcribe, a state-of-the-art speech recognition model that beats both commercial and open-source models to land at #1 on the Open ASR Leaderboard!
Julian Mack retweeted
Your speculative decoder uses one drafter for everything. But the best drafter depends on the content and can change every step.
Hybrid-Spark switches between a model-based (DSpark) and a model-free (n-gram) drafter per request, per step.
Up to 1.62× over DSpark on code edits.
Julian Mack retweeted
New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s!
This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with incredibly clever engineering and ML judgement. If a flop is low value on a particular step, skip it.
Specifically:
-(~8s) Sampled softmax. If a token doesn’t appear in a batch, skip its lm_head fwd/bwd some fraction of the time.
-Sparse values. Only run an optimizer step for ngram embeddings that occurred in the batch. Set beta1 to zero to enable this. Beta2 is applied retroactively when the row is later used.
-Sparse updates. Only update ngram and value embeddings once every 4 steps instead of once every 2.
-Sparse communication. Shard the n-gram table across GPUs, and only pass the rows receiving updates on each step.
-Sparse optimizer states. For the n-gram table, reduce from 2 floats in Adam optimizer per param, to 1 float per 768 params.
-Hand-rolled flash attention for 64 dim heads.
There are several additions that add accuracy too:
-(~4s) EMA during last 300 steps, combined with lifting final_lr to 0.3 instead of 0.15.
-(~1s) A new optimizer, Anvil2, which expands muon via a second tracked momentum buffer, improves the ortho coefficients, and modifies the cautious weight decay application.
-A couple additional dynamic skip connections in the network.
The most striking consequence of the ‘flop aware paradigm’ is you can grow parameters arbitrarily large,
only limited by the available memory, since you can selectively choose how to expend flops on those parameters on each step. NanoGPT has kept active parameters below 124M, but total is unbounded, and has grown to 640M through embedding sparsity over the last year. This PR takes that to its logical conclusion on the 8xH100, scaling up to 65B sparse embedding parameters, which accounts for 25% of the PR’s gains. At frontier scale, where one is not bounded by an 8xH100, one could imagine where this paradigm could lead.
github.com/KellerJordan/modd…
As this was a very notable PR, I spoke with Deven for an hour to learn how he did it. Here’s his story on the changes: hyperstition.cc/training-nan…
Julian Mack retweeted
wow so much drama about navier stokes. had no idea things could blow up so infinitely in such a finite time.
Julian Mack retweeted
نطلق اليوم نموذج @cohere Transcribe العربي، أفضل نموذج مفتوح المصدر لتحويل الكلام العربي من الصوت إلى نص، مع مراعاة اللهجات العربية المختلفة. يتصدر معيار Open Universal Arabic ASR Leaderboard.
صُمم النموذج لدعم حالات تحويل الكلام بين العربية والانجليزية، والتعرف على اللهجات المختلفة، بالإضافة إلى اللغة الانجليزية باللكنات العربية. هذا يمكّن المطورين والشركات من بناء تجارب صوتية مذهلة لعملائهم.
رابط التحميل في التعليقات. نتطلع إلى سماع تجاربكم له!
Julian Mack retweeted
DeepSeek just released DSpark for V4 Flash & Pro, a new speculative decoding method boosting throughput by 51% to 400%!
DS also showed DSpark works well for other models like Gemma & Qwen
Github: github.com/deepseek-ai/DeepS…
Paper: github.com/deepseek-ai/DeepS…
HF: huggingface.co/deepseek-ai/D…
Julian Mack retweeted
> if you have a model that can come up with novel ideas, it must have high loss during sft or rl
pov: your training loss looks like this and you try to find a way to rationalize it
i’m really surprised that people don’t see this.
It’s mathematically true that llms can’t come up with novel ideas, because the whole point of training is to reduce loss, gain rewards so that the model adhere to rules and ground truth.
if you have a model that can come up with novel ideas, it must have high loss during sft or rl.
Julian Mack retweeted
Moshi is great but sometimes not fully exploiting its full-duplex abilities. @atsumoto_ohashi applied carefully crafted RL rewards on Moshi's output given some real inputs to improve interactivity on all axis. Works great on Moshi and the derived PersonaPlex by @nvidia.
Julian Mack retweeted
New leaderboard for audio transcription just launched and our apache 2.0 Cohere-Transcribe is at the top. This eval didn't exist when we trained the model, so its nice to see us do so well on it.
huggingface.co/spaces/treble…
Looking forward to speaking at this webinar next week. We'll be discussing a new ASR leaderboard from @huggingface and Treble.
The other speakers are 🔥. Researchers from @nvidia, @huggingface, Treble, @IBM and @WavLab (CMU)
More info and sign-up details below!
Want to hear about where ASR is headed? Check out @HuggingFace's webinar about their new Far-Field ASR Leaderboard with @shinjiw_at_cmu and our own @Julianfmack on Thursday, June 11th.
We'll be chatting about Cohere Transcribe and how to evaluate ASR systems in the real world. Sign up here: treble.tech/insights/treble-…
We have an all-star line-up:
• Nithin Rao Koluguri, @nvidia (Parakeet, FastConformer)
• George Saon, @IBM (Granite Speech)
• @EricBezzam, @huggingface (Pyroomacoustics, Open ASR Leaderboard)
• Shinji Watanabe, @shinjiw_at_cmu (CHiME, OWSM, and so many more...)
1
6
153
You have to sign-up in advance!
treble.tech/insights/treble-…
54
Come build something local with cohere-transcribe or TinyAya at this huggingface hackathon
We're sponsoring a hackathon to scale down.
Hosted by our friends @huggingface and @Gradio, we want working with models to feel like yours again. Small enough that it's inexpensive to run, big enough that it can change the life of someone you care about.
Build something delightfully weird that only AI could help you create, or pick a problem for someone you actually know and build a solution. The choice is yours 🧙
Command A+ sets a new high for Cohere's machine translation capabilities.
Opening a clear gap over open source peers Mistral Medium 3.5, DeepSeek, & OpenAI's gpt-oss, as well as Claude Opus 4.6. A+ also outperforms specialist systems like Google Translate.
RWS is better... but we built that with them too
Julian Mack retweeted
Announcing the Artificial Analysis Coding Agent Index! Our new coding agent benchmarks measure how combinations of agent harnesses and models perform on 3 leading benchmarks, token usage, cost and more
When developers use AI to code they’re choosing a model, but also pairing it with a specific harness. It makes sense to benchmark that combination to understand and compare performance.
The Artificial Analysis Coding Agent Index includes 3 leading benchmarks that represent a broad spectrum of coding agent use:
➤ SWE-Bench-Pro-Hard-AA, 150 realistic coding tasks that frontier models struggle with, sampled from Scale AI’s SWE-Bench Pro
➤ Terminal-Bench v2, 84 agentic terminal tasks from the Laude Institute and that range from system administration and cryptography to machine learning. 5 tasks were filtered due to environment incompatibility
➤ SWE-Atlas-QnA, 124 technical questions developed by Scale AI about how code behaves, root causes of issues, and more, requiring agents to explore codebases and give text answers
Analysis of results:
➤ Opus 4.7 and GPT-5.5 lead the Index: Opus 4.7 in Cursor CLI scores 61, followed closely by GPT-5.5 in Codex and Opus 4.7 in Claude Code at 60. GPT-5.5 in Cursor CLI follows at 58.
➤ Open weights models are competitive, but still trail the leaders: GLM-5.1 in Claude Code is the top open-weight result at 53, followed by Kimi K2.6 and DeepSeek V4 Pro in Claude Code at 50. These are strong results, but still meaningfully behind the top proprietary models.
➤ Gemini 3.1 Pro in Gemini CLI underperforms: Gemini 3.1 Pro in Gemini CLI scores 43, well below where Gemini 3.1 Pro sits on our Intelligence Index, highlighting that Gemini’s performance in Gemini CLI remains a relative weak spot for Google’s offering.
➤ Cost per task (API token pricing) varies >30x: Composer 2 in Cursor CLI is cheapest at $0.07/task, followed by DeepSeek V4 Pro in Claude Code at $0.35/task and Kimi K2.6 in Claude Code at $0.76/task. At the high end, GPT-5.5 in Codex costs $2.21/task, while GLM-5.1 in Claude Code costs $2.26/task. For both models this was contributed to by high token usage, and in GPT-5.5’s case by a relatively higher per token cost.
➤ Token usage varies >3x: GLM-5.1 in Claude Code uses the most tokens at 4.8M/task, followed by Kimi K2.6 at 3.7M/task and DeepSeek V4 Pro at 3.5M/task. GPT-5.5 in Codex uses 2.8M tokens/task, substantially more than Opus 4.7 in Claude Code at 1.7M/task. In GLM-5.1’s case, higher token usage, cost and execution time were partly driven by the model entering loops on some tasks.
➤ Cache hit rates remain high but vary materially: Cache hit rates range from 80% to 96% across combinations. Provider routing, harness prompt structure and cache behavior can materially change the economics of running the same model given cached inputs are typically <50% the API price of regular input tokens.
➤ Time per task varies >7x: Opus 4.7 in Claude Code is fastest at ~6 minutes/task, while Kimi K2.6 in Claude Code is slowest at ~40 minutes/task. This is contributed to by differences in average turns per task, token usage and API serving speed. Opus 4.7 had materially lower amount of turns to complete a task than all other models while Kimi K2.6 had the most.
➤ Cursor made real progress with Composer 2: Composer 2 in Cursor CLI scores 48, near the leading open-weight model results, while being the cheapest combination measured at $0.07/task. Cursor has stated Composer 2 is built from Kimi K2.5, showcasing they have made substantial post-training gains.
This is just the start. We are planning to add additional agents (both harnesses and models). Let us know what you would like to see added next.
Julian Mack retweeted
It's been 3 months since the 100x vibers started 100x vibin'! So, post your 25-years-of-work-equivalent project here, so we can signal boost and everyone can celebrate the Life's Work that you did in 3 months. Looking forward to it, Let's Go!!!
A new faster implementation of cohere-transcribe
It gives a 1.5x speedup on short-form and a 3.2x speedup on long-form audio over the native transformers (which was already pretty speedy). Awesome @Tu7uruu!
Cohere transcribe deserves some more love, so I made a pure-torch port with blazing fast long form transcription!
> 14 languages
> energy-based chunking + cross-file chunk batch packing
> pushes the full earnings21 long-form set (39.3 h of audio) through a single A100 in 3.7 min at RTFx 632×.
you can try it now, just pip install nano-cohere-transcribe !
Julian Mack retweeted
Ever wondered how Speculative Decoding interacts with production MoE models?
Conventional wisdom: MoE + speculative decoding = too many experts to load, gains disappear.
Reality: MoE amplifies speculative decoding.
Checkout Cohere Blogpost: cohere.com/blog/mixture-of-e…
This is cool. Our model running in the browser via transformers.js!

























