jan (@janpf95) profile picture
Joined October 2011
Photo by jan (@janpf95)Photo by jan (@janpf95)Photo by jan (@janpf95)Photo by jan (@janpf95)
📄 Releasing the Soofi S pretraining tech report: a sovereign, open foundation model for German and English Today we’re publishing the full pretraining tech report and project page for Soofi S 30B-A3B — a Mixture-of-Experts hybrid Mamba model trained on ~27 trillion tokens with deliberately up-weighted German. What’s in the report: 🏆 Strongest fully open model in our evaluations on BOTH the English and German aggregates — ahead of Olmo 3 32B and Apertus 70B (full methodology in the report) 📋 Radical transparency: complete per-source data accounting, all hyperparameters, training + eval code, checkpoints — everything under permissive licenses 🇩🇪 Trained end-to-end on Deutsche Telekom’s Industrial AI Cloud in Munich — sovereign AI infrastructure on German soil Soofi S combines frontier-level capability with the highest measured aggregate long-context decode TPS, and unlike full-attention dense baselines maintains high throughput as context grows. The figure plots Capability Index versus measured aggregate decode TPS/GPU at 40K context and batch 32. The Capability Index averages five benchmark groups, i.e., Code, GSM8K, GPQA-Diamond, English aggregate, and German aggregate, after normalizing each group to the best plotted model. Aggregate decode TPS/GPU is measured with a TP=1, one-B200 vLLM latency-subtraction protocol.
88
174
52
1,133
390,017
gpt-oss 120B is very blatantly incapable of producing linguistically correct german text. 🧵
73
29
27
702
135,531
We’re returning to Würzburg from the largest ACL ever 🌸. We presented 4 papers, including #LLäMmlein 🐑, a natively German decoder model family, a new SotA on ABSA, won another SemEval Task and best paper award. We had a fantastic time and drew inspiration for future research 🚀
1
3
5
324
Exciting news from #CAIDAS @Uni_WUE ! We're proud to announce the launch of "#LLäMmlein," the first LLM exclusively trained in German with 1B parameters! Check out the details in our press release go.uniwue.de/llammleinnews & project page go.uniwue.de/llammlein!
1
6
18
693
And that's a wrap from the #NAACL2024 in 🇲🇽 Mexico City! We showcased "SuperGLEBer" - the first comprehensive German language benchmark, earned a best paper honorable mention with our hierarchical classifier, and even presented the Roman Empire's take on model hallucinations!
1
1
1
14
962
Live poster session for our @SemEvalWorkshop shared task @shroom2024 ! Thanks a lot to all the participants who presented their systems, in person and remotely!
1
5
16
1,293
All good things come in threes: We will present our last paper at #NAACL2024 "The Roman Empire Strikes Back" at the @SemEvalWorkshop poster session at 14:00. Meet us there! It is a joint work with @konstantinkobs on detecting hallucination detection in text generation models.
1
1
6
425
We are thrilled to announce that we won the "Best System Paper Honorable Mention Award" at @semevalworkshop (#NAACL2024) for our work "Developing a Hierarchical Multi-Label Classification Head for Large Language Models". See you at our talk (now) and our poster (15:30)!🦦
1
13
567
Our paper "SuperGLEBer: German Language Understanding Evaluation Benchmark" was accepted at the NAACL 2024: In our paper, we assemble a broad Natural Language Understanding benchmark suite for the German language and consequently evaluate a wide array of… dlvr.it/T43gJQ
1
7
237