
AI and Quantum. currently @ZyphraAI. ex-Cisco, ex-Microsoft. Former postdoc at Harvard, MIT. Former student at Princeton, UIUC, and U.Chicago.
Joined October 2012
- Tweets238
- Following573
- Followers473
- Likes1.3K
Pinned Tweet
this one is special.
it has been quite a ride. we built the entire training pipeline and data stack from scratch.
today we shipped ZAYA1-VL-8B: a strong compact MoE VLM that punches above its weight class.
so proud of this team. and we're just getting started🚀
Today we're releasing ZAYA1-VL-8B, our first vision-language model.
ZAYA1-VL-8B is a 700M active / 8B total MoE built on our ZAYA1-8B base trained on @AMD. We achieve strong performance for our size resulting in leading intelligence density and inference efficiency.
@ZyphraAI is on the @ArtificialAnlys leaderboard for GLM-5.3-Flash throughput. Optimized on @amd MI300X with Zyphra’s Optimization Engine (ZOE).
Try it on Zyphra Inference: cloud.zyphra.com
$/1M tokens:
- Input: $0.15
- Cached: $0.03
- Output: $0.25
Meet ZONOS3-Mini: open-source text-to-speech with voice cloning and word-level control. All on-device and super fast.
Every voice in this video is from a TTS model that can run real-time on a phone.
Comment “ZONOS” within 2 days for 20 hours of free audio.
Sound on 🔊
🤖 Made with AI
Our mission at Zyphra is to make open superintelligence accessible to everyone.
Today, we're excited to beta release MAIA, the application layer of our fullstack.
Meet your new colleague. A proactive, multiplayer AI agent for teams.
Free during beta: zyphra.com/maia
Introducing Zyphra FlashAttention (ZYFA): faster, more accurate 8-bit floating-point (FP8) attention on AMD MI355X.
Our prefill kernels deliver 1.16–1.76x the throughput of AMD’s inference kernel library, AITER, on real model inputs at 16K tokens under sustained load. 🧵
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
Zyphra presents the second wave of inference optimizations for @AMD Instinct MI355X GPUs, powered by Zyphra Optimization Engine (ZOE).
Our latest engineering blog details the kernel work behind faster long-context serving for frontier open models. 🧵
In Mixture of Expert (MoE) models the cost of moving tokens to their experts can dominate total runtime.
At Zyphra research, we use patterns in how tokens are routed to experts to make that communication faster by up to 2.63x on @AMD MI300X GPUs, with the model unchanged.
Hassan Shapourian retweeted
This contains a shocking list of problems in quantum information, many-body physics and quantum computing, the field that are dear to my heart. There are too many, but let me pick the following.
1. Proof of area law in 2D.
2. Spin-one Haldane gap
3. Parity is not in QAC^0
4. Constant-error Aaronson-Kuperberg conjecture
5. Unitary VOAs generating conformal nets.
Things are changing fast. I cannot even imagine what will happen over the next few months, let alone a year.
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Hassan Shapourian retweeted
Open models were already winning on token share. Now they’re generating more inference spend than OpenAI (on Vercel)
More pressure on frontier model margins
Looks like today may be a record day for token volume % of open models on Vercel AI Gateway:
🟦 Open 78.4% 🟨 Closed 21.6%
While spend 💲 usually tells a different story, #3 and #4 today are Moonshot AI & DeepSeek. Adding Z.ai, their combined spend surpasses OpenAI (#2).
(Do note that's the spend for inference of the model across providers (mostly in the US), not revenue going directly to the open weight labs.)
Zyphra is at AI Infra Summit in Santa Clara this week:
• Tue 2:20 PM: @QuentinAnthon15 moderates a panel on scale-out networking
• Wed 3:00 PM: @jjubran in a fireside chat on standing up AI capacity
• Wed 3:30 PM: @QuentinAnthon15 on serving MoEs on @AMD MI355X
Zyphra Research releases PUFFER, a novel incremental fuzzy deduplication system for LLM-scale datasets.
PUFFER runs entirely on CPU and achieves 11-35x speedup over existing methods.
We use PUFFER internally for our own training dataset and release it under Apache 2.0 license.
Hassan Shapourian retweeted
Everyone is busy telling their LLMs to prove or disprove well-known conjectures. Is there anyone trying to find fatal flaws in equally well-known results whose proofs have only been studied by a couple of reviewers, besides their authors?
Introducing ZUNA1.1, a far more flexible version of our open EEG foundation model and a further advancement toward noninvasive thought-to-text.
It reconstructs, denoises, and upsamples messy real-world EEG. Apache 2.0, available free in the Zyphra Cloud EEG Playground🧵
Hassan Shapourian retweeted
Introducing Flash-MSA, the world’s first open source sparse attention training kernels optimized for extreme context lengths. 4 simple kernels, 400%+ speedup over dense flash attention at long context ⚡
Zyphra is sharing our first work in continual learning where we study: Can LLMs learn forever from new data?
Many see continual learning as a path to AGI through recursive self-improvement (RSI).
The first obstacle is plasticity loss. We derive a scaling law for its onset 🧵
Zyphra Research is releasing Norm-AGnostic residual networks (NAG) - a new architecture that mitigates the diminishing returns of deeper residual models by controlling the residual stream geometry.
NAG makes Mixture-of-Depths practical for pretraining.
Vision workloads are notoriously demanding on memory and compute. Exploring SSMs in vision-language models opens up a different part of the design space—one where efficiency and capability can scale together.
Zyphra Research continues to explore architecture innovations beyond standard transformers.
Today we’re releasing Zamba2-VL, extending our prior Zamba2 hybrid SSM-Transformer work into vision-language modeling. 🧵
3
246
It all started with a basic question: what happens when you bring hybrid SSM-Transformer architectures to multimodal intelligence?
Zamba2-VL is an early step toward that future. Huge credit to our vision team for making it happen.
Replying to @ZyphraAI
Zamba2-VL is competitive with the leading open Transformer vision-language models of comparable scale, including Qwen3-VL, InternVL3.5, Molmo2, and PerceptionLM, across image understanding, reasoning, OCR, grounding, and counting benchmarks.
3
181























