
Post-Training Data Research Lab Encoding Human Expertise into Machines. backed by @ycombinator, @drivecapital, @alumniventures
San Francisco
Joined March 2024
- Tweets48
- Following21
- Followers151
- Likes6
Pinned Tweet
Ocular AI (YC W24) evaluated @NVIDIAAI's latest Nemotron 3.5 Transcribe 30B-A4B ahead of today's release, working with their research team!
It ranks #1 among open-weight transcription models and #4 overall across 21 SOTA models on our benchmark, with a 2.95% word error rate.
A standout: preserving repetitions, short acknowledgments, and fragmented speech, with the second-lowest deletion count.
One scoring detail matters: its spelling of “hm” added 57 insertion errors under our benchmark’s normalization rules.
Changing that spelling alone would lower WER to 2.50%—second overall!
Congratulations to the @nvidia team!
#NVIDIA #Nemotron #SpeechAI #OcularAI
Full evaluation and error analysis: useocular.com/blog/evaluatin…
17
🚀 New Research: Post-Training Open TTS Models with Human Data.
We fine-tuned 8 models, including NVIDIA Audex-2B & Qwen3-TTS 1.7B, using LoRA & full-weight SFT.
Just 1hr 22 minutes of studio-grade, TTS Dataset improved 5 models by up to +13.6 points on alphanumeric reading.
@NVIDIAAI @nvidia @Alibaba_Qwen @sesame
More here: useocular.com/blog/improving…
14
1/ Results on our held-out alphanumeric TTS benchmark:
→ @nvidia @NVIDIAAI Audex-2B: 23.6% → 37.2%
→ Llasa 1B: 14.5% → 28.1%
→ Spark-TTS: 19.6% → 31.7%
→ Orpheus 3B: 32.0% → 39.3%
→ @Alibaba_Qwen Qwen3-TTS: 23.6% → 30.5%
Full research, training configurations & evaluation results:
2
47
𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗧𝗵𝗲 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗲-𝗗𝗶𝗮𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗕𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸 🎙️
Getting the words right is only part of understanding a conversation. 𝗞𝗻𝗼𝘄𝗶𝗻𝗴 𝘄𝗵𝗼 𝘀𝗮𝗶𝗱 𝘁𝗵𝗲𝗺 𝗺𝗮𝘁𝘁𝗲𝗿𝘀 𝘁𝗼𝗼.
11 models. Real-world, two-person American English conversations. Interruptions and overlapping speech included.
Models hear a single mix. Separate speaker tracks and time-aligned transcripts provide the reference.
Vendor-by-vendor results below ↓
Scores are pooled diarization error rate (DER) on this sample: no collar, overlap included. Lower is better.
Leaderboard, report, dataset sample & evaluation harness:
useocular.com/converse/diari…
#VoiceAI #SpeakerDiarization
9/10 • @Meta
Muse Voice Transcribe 1.0: 𝟯𝟱.𝟴𝟰% DER.
Eleventh overall on this four-conversation sample.
1
23
10/10 • @SpaceXSI (xAI)
Two Grok systems evaluated:
• Grok Voice Transcribe 2.0: 𝟯𝟭.𝟯𝟯% DER
• Grok STT 1.0: 𝟯𝟰.𝟮𝟴% DER
Ranked 6th and 10th respectively on this sample.
15
🚀 New Product: we're introducing Default Voices for Text-to-Speech (TTS) models.
A catalog of professionally performed, studio-recorded voices that can be licensed for commercial TTS default voice cloning.
Each voice comes with:
1. A professional voice actor
2. Studio-recorded performance
3. Distinct and consistent voice identity
4. Commercial synthesis and voice-cloning rights
Listen to the voices: useocular.com/human-data/tts…
If the voices fit what you're building, send me a message and we’ll share the licensing details.
#TextoSpeech #VoiceAI #AI #TTS #ASR
Ocular AI (YC W24) retweeted
Replying to @SonofMushabati @OcularHQ
More about this round: useocular.com/blog/pre-seed
