Follow for Actionable tips, Best Resources and Daily content on Python, Data Science & Machine Learning. 🚀
Joined July 2022
- Tweets1.4K
- Following743
- Followers12.8K
- Likes5.2K
Machine Learning Community ⭐️ retweeted
UC Berkeley open-sourced FreeToken!
FreeToken is an edge-native inference engine for running frontier-scale MoE models on consumer hardware. GPU, CPU, host memory, and PCIe interconnects treated as one unified inference platform.
The reason this works comes from how MoE models are structured. A model like DeepSeek-V4-Flash has 284B total parameters but only activates 13B per token - 6 out of 256 routed experts across each layer. The computation per token is feasible. The problem is the full expert pool still needs to be accessible in memory, which far exceeds GPU VRAM.
FreeToken handles this with a two-level hierarchy. Non-expert weights stay resident on GPU. All expert weights live in host RAM. Only the experts needed for each token get fetched over PCIe. A bandwidth-adaptive policy continuously decides whether to fetch experts to GPU or compute them on CPU based on the machine's actual measured PCIe bandwidth.
The result: a laptop with 8GB VRAM runs Qwen3.6-35B. A single RTX 5090 runs DeepSeek-V4-Flash at 284B. A workstation GPU runs GLM-5.2 at 753B.
No GGUF conversion needed. Loads HuggingFace safetensors directly. OpenAI and Anthropic API compatible on localhost. Native GUI with one-click install on Windows and Linux, with agent harnesses built in.
Key capabilities:
• Two-level expert hierarchy: GPU for non-expert weights, host RAM for expert pool
• Bandwidth-adaptive CPU-GPU co-execution calibrated to your machine
• Global LRU expert caching across all MoE layers
• Semantic-aware KV caching for agentic workflows
• Loads HuggingFace safetensors directly, no GGUF conversion
• OpenAI and Anthropic API compatible on localhost
• Native GUI with built-in agent harnesses
100% open source.
I've shared the link in the replies!
Machine Learning Community ⭐️ retweeted
MLCommons recently introduced DLRMv4, the next-generation MLPerf Training benchmark for recommendation models.
And it’s built on Yambda-5B, the open dataset from Yandex I covered a while ago.
Developed by a task force from AMD, Meta, NVIDIA, and ByteDance, DLRMv4 measures how efficiently modern recommendation models can be trained at scale.
Why this matters:
Production recommenders have moved from aggregated features to sequence models that learn from each user’s interaction history. The previous benchmark had no per-user timeline, so it couldn’t reflect how these models actually work.
Yambda-5B fills that gap with 4.79 billion real interactions across 1 million users, recorded in chronological order. Combined with HSTU sequence modeling and a 560 GB embedding footprint, it gives the industry a realistic, standardized way to benchmark production-scale recommendation training.
Recommendation models are only as good as the behavior they learn from, and now the benchmark reflects that too.
I've added the link to my post on Yambda below if you want the dataset details.
Yandex’s Yambda-5B is the primary dataset for DLRMv4, @MLCommons’ next-generation benchmark for training recommendation models.
Developed by researchers at @AMD, @Meta, @nvidia, and ByteDance, it measures how efficiently modern recommenders are trained.
mlcommons.org/2026/10/dlrmv4…
Machine Learning Community ⭐️ retweeted
I built a document agent with a live web-grounded decision engine!
Decision models like Jev are powerful, but they don’t have live context by default. They can only reason over what they learned during training and the context you give them at inference time.
That becomes a problem when you’re working with documents containing information that can change over time.
An API referenced in a document could have been deprecated, a package or model recommendation could be outdated, or an instruction that was correct when the document was written may no longer work today.
So I built an agent that checks these time-sensitive claims against the live web.
I first use Agentic Document Extraction to parse the uploaded document while preserving its structure and content.
From that extracted content, the agent identifies claims that are actually freshness-sensitive rather than sending every sentence for verification.
For each claim, it keeps the original statement and the surrounding document context, then turns the verification into a decision task for Parallel’s Decisions API.
Parallel searches the live web for evidence relevant to that specific claim and returns a grounded decision with supporting sources.
The agent then maps the result into one of three states:
- VALID: current evidence still supports the claim
- OUTDATED: newer evidence contradicts or supersedes it
- NEEDS REVIEW: the available evidence is not strong enough to make a reliable call
For example, if an older document says:
“Use API v1 for this integration.”
the verification request includes that claim and its document context. Parallel can search the current API docs, check whether v1 is still supported, and ground the decision in the latest sources it finds.
The final output keeps both sides together: the claim extracted from the original document and the current web evidence used to evaluate it.
So instead of treating a document as static context, the agent can reason about which parts of it may have drifted since it was written.
Github Repo: github.com/Sumanth077/Hands-…
This can be useful across technical docs, research reports, internal knowledge bases, implementation guides, product documentation, policies, and other documents where some claims remain stable while others age quickly.
RAG for decision models is here!
Parallel just launched the Decisions API, which combines live web search with bounded decision-making in a single call.
Decision models like Jev are useful when the output is small and structured: yes/no, one option from a list, a category, or a routing decision.
The problem is that those decisions are only as good as the context you give them.
If the answer depends on something that changed recently, or information that only exists on the web, the normal flow becomes retrieval first and decision second.
Parallel brings those two steps together.
You provide the state and the questions you want answered. The API searches the web, grounds the decision in what it finds, and returns the typed answer along with probability, confidence, and sources.
That makes it much more useful for agent workflows where a decision depends on current external information, research tasks where the model needs to verify something before choosing an outcome, or eval pipelines where the judgment needs grounding from the web.
It can also answer multiple questions against the same retrieved context in one request, instead of repeating the search for every decision.
So instead of:
Search → retrieve context → decision model → structured answer
you can make the grounded decision directly through one API.
I think this is a pretty natural extension of where decision models are going: bounded outputs are useful, but giving that decision layer access to the live web makes the set of problems it can handle much larger.
Machine Learning Community ⭐️ retweeted
Baidu just open-sourced a new SOTA Base model!
ERNIE-5.5-Flash-Base is a 280B MoE model with 14B active parameters per token, pretrained from scratch on 33.5 trillion tokens.
And this is the Base checkpoint itself.
It hasn’t gone through mid-training or post-training yet, which makes the results especially interesting because they give a much cleaner view of what the architecture, data, and pretraining have learned.
In Baidu's evaluations, ERNIE-5.5-Flash-Base outperforms similarly sized Base models on most benchmarks across knowledge, reasoning, mathematics, coding, multilingual understanding, and long-context tasks.
A few parts of the architecture stood out to me.
For long-context efficiency, the model uses Latent-Gated Sparse Attention, which combines MLA, heavily compressed attention, and sparse attention.
Instead of keeping every layer operating over the full token-level history, most layers work over a heavily compressed representation, while selected layers retain access to the detailed context.
The sparse attention component then focuses computation on the most relevant positions rather than attending densely across the entire sequence.
This reduces the compute and memory cost of long contexts while still supporting up to 128K tokens during pretraining and up to 1M tokens during fine-tuning and inference.
The MoE routing is also different.
Most MoE models rely on a single router to decide which experts should process each token.
ERNIE-5.5-Flash-Base uses a Multi-Router Latent MoE, where multiple routers contribute to expert selection without increasing the number of activated experts or the communication cost.
Baidu also introduced ECHO for multi-token prediction.
Instead of adding a separate MTP module, ECHO reuses the model’s final backbone layer to predict future tokens.
This lets the future-token prediction objective directly train the same backbone used for normal next-token prediction, and Baidu reports improvements across mathematics, reasoning, and coding.
The bigger point here is that this is not an instruction-tuned assistant being benchmarked after a long post-training pipeline.
It is the pretrained foundation itself.
That makes the release useful for researchers and AI teams that want a strong Base model they can further train, adapt, or customize for their own workloads.
I’ve shared the model in the comments!
Excited to introduce ERNIE 5.5 Flash Base, my first release since joining @Baidu_Inc !
An open-sourced 280B MoE model with just 14B active parameters, pretrained on 33.5T tokens. It delivers strong performance across reasoning, coding, and long-context benchmarks, with support for up to 1M context.
Huge respect and thanks to the incredible team behind this — outstanding work!
Post-training is already underway, and I can't wait to bring it to you soon.
And this is just the beginning — more exciting technical developments are on the way. Stay tuned!
ernie.baidu.com/release/erni…
Machine Learning Community ⭐️ retweeted
The next frontier for AI agents is scientific research!
Science’s Last Exam is a new benchmark designed to test whether an AI agent can carry out an entire research cycle on its own, not just solve isolated scientific problems.
Instead of giving the model a question with a known path to the answer, SLE gives it a research objective, the required tools and materials, and a limited computational budget.
From there, the agent has to decide what to do next.
It needs to formulate hypotheses, choose experiments, run computations, interpret the results, react when something fails, and decide whether the evidence is strong enough to support a final conclusion.
One of the tasks makes this especially clear.
The agent has to identify the lowest dose of an antithrombotic drug that can suppress dangerous blood clots without disrupting normal hemostasis.
That means it cannot simply find one dose that works once.
It has to test different dosing intervals, vessel geometries, blood-flow conditions, and pharmacological response profiles, then use each result to decide what experiment is worth running next.
Even after finding a workable regimen, it still has to ask whether the dose can be lowered further, whether the result holds under different conditions, and where the evidence stops being reliable.
Once the agent submits its final result, the answer is frozen and evaluated against hidden checks defined in advance by domain experts.
SLE also breaks the research process into individual capabilities such as hypothesis discovery, protocol planning, experiment observation, scientific computing, and scientific software operation, so it is possible to see not just whether an agent failed, but where it got stuck.
The initial release includes 50+ selected tasks across nine scientific disciplines, with researchers from 38 institutions contributing to task design and review in their personal capacities.
And the next step goes beyond simulation.
SLE is also building wet-lab tasks where AI-designed experimental protocols will be executed in real laboratories, with the resulting data sent back to the agent for another round of reasoning and experimentation.
That is where agent evaluation starts looking much closer to actual scientific research.
I’m incredibly proud to share what we’ve been building at the AItonomy Foundation: Science’s Last Exam (SLE). 🧪✨
Can AI do science? Instead of Q&A, we give agents actual research goals and real-world tools (lab data, software), then grade them against hidden expert checks.
Deeply grateful to the brilliant minds at 34 institutions (Harvard, MIT, Stanford, etc.) who helped craft 113 tasks across 9 disciplines. ❤️
The results? Science is hard. Of 8 frontier models, the best scored 37.5/100. On true end-to-end research? All scored 0. We have an exciting journey ahead! 🚀
(Claude Opus 5.5 & GPT-6 Astra coming soon—just clearing bio safeguards).
Everything opens Nov 6. 🗓️
🌐 sciences-last-exam.org
💻 github.com/Gen-Verse/Science…
🤗 huggingface.co/datasets/Gen-…
Thread below on how the exam works and where models break. 👇 (1/14)
#AGI #AIforScience #AIAgents #SciencesLastExam #OpenAI #Anthropic
Machine Learning Community ⭐️ retweeted
Modest intelligence is arriving faster than most people realize.
Today, we're open-sourcing TwIL-8B.
Just 8.8 billion parameters, yet approaching frontier-model scores on several difficult benchmarks:
• 93% of frontier performance on coding
• 92% on instruction following
• 90% on contest mathematics
All from a model small enough to run on your own hardware.
But the benchmarks aren't the most important part of this announcement.
For years, the AI industry has focused on building the most capable model, regardless of the cost required to run it.
I believe that equation is changing.
The next era of AI will be defined by the cost of completing work. Not simply the intelligence of the model, but how efficiently that intelligence can accomplish the task.
If a model with 8 billion parameters can reliably complete the same work as one with hundreds of billions, why pay for the larger model?
As smaller models become more capable, intelligence becomes something we can distribute, specialize, and run across the hardware we already own.
That's the future we're building at @thewebAI
TwIL-8B is available today. Open weights.
And this is just the first of several announcements we're making today.
HF Link: huggingface.co/webAI-Officia…
Machine Learning Community ⭐️ retweeted
NVIDIA Open-Sourced Auto Ontology, a Semantic Layer for AI Agents!
Auto Ontology is a Python framework for building semantic, graph-based reasoning layers over data.
AI agents can already query databases and retrieve documents.
The harder problem is making sure they understand what the data actually means.
A metric like “revenue” might be defined differently across two systems. The same customer can appear under different identifiers. A generated SQL query can be syntactically correct and still use the wrong business definition.
Auto Ontology is built for that layer in between.
It can build a semantic catalog over enterprise metadata so agents can reason about entities, metrics, relationships, lineage, and where trusted definitions come from.
The workflow is roughly:
Enterprise data + metadata
→ Auto Ontology
→ Semantic catalog / knowledge graph
→ AI agent
→ Grounded query + provenance
It can generate ontologies from existing metadata, connect to semantic models you already have, reconcile inconsistent definitions across systems, and expose that shared context through MCP, CLI, or API.
For structured data, it can also translate natural-language questions into SQL while validating the query against the ontology and certified definitions before returning the result.
So instead of an agent simply asking:
“How much revenue did we generate last month?”
it can understand which revenue definition to use, which tables contain the right data, how those entities relate, and where the final answer came from.
Auto Ontology currently supports relational sources including Databricks, Postgres, Snowflake, and DuckDB.
It also exposes an MCP server, which means agents can query the same semantic layer directly.
For enterprise agents, retrieval alone is not enough.
They also need a shared understanding of what the underlying data actually means.
100% open source.
I’ve shared the GitHub repo in the comments!
NVIDIA Open-Sources Auto Ontology, a Semantic Layer for AI Agents
Your boss asks the AI agent: "What was revenue last quarter?"
It answers in 3 seconds. Clean number. Nice chart.
But it used the wrong "revenue" column. 𝗡𝗼𝗯𝗼𝗱𝘆 𝗻𝗼𝘁𝗶𝗰𝗲𝘀.
That's the problem NVIDIA's new open-source library, Auto Ontology, goes after. Its README puts it plainly:
"…𝘢 𝘳𝘦𝘷𝘦𝘯𝘶𝘦 𝘮𝘦𝘵𝘳𝘪𝘤 𝘢𝘯𝘴𝘸𝘦𝘳𝘦𝘥 𝘧𝘢𝘴𝘵𝘦𝘳 𝘪𝘴 𝘸𝘰𝘳𝘵𝘩 𝘯𝘰𝘵𝘩𝘪𝘯𝘨 𝘪𝘧 𝘵𝘩𝘦 𝘯𝘶𝘮𝘣𝘦𝘳 𝘪𝘴 𝘸𝘳𝘰𝘯𝘨."
𝗪𝗵𝗮𝘁 𝗶𝘁 𝗶𝘀:
A "context layer" that sits between an AI agent and your database. It tells the agent where the data lives, what each field means, and whether to use it.
𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀, 𝘀𝘁𝗲𝗽 𝗯𝘆 𝘀𝘁𝗲𝗽:
• 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿: it reads your database metadata and drafts an ontology, a map of business terms like "active customer"
• 𝗚𝗿𝗼𝘂𝗻𝗱: agents use those terms instead of guessing from raw table names
• 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗲: it checks generated SQL against certified definitions
• 𝗘𝘅𝗽𝗼𝘀𝗲: agents reach it through MCP, a CLI or an API
• 𝗟𝗲𝗮𝗿𝗻: certified corrections go back into the map
𝗙𝗼𝗿 𝘂𝘀𝗲𝗿𝘀:
Ask a question, get 𝘁𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿, 𝘁𝗵𝗲 𝗦𝗤𝗟 𝗶𝘁 𝗿𝗮𝗻, 𝗮𝗻𝗱 𝘁𝗵𝗲 𝗿𝗼𝘄𝘀. You can check the work, not just trust it.
𝗙𝗼𝗿 𝗯𝘂𝗶𝗹𝗱𝗲𝗿𝘀:
• Apache 2.0, runs with Docker Compose or Helm
• Works with Databricks, Postgres, Snowflake and DuckDB
• Defaults to NVIDIA's Nemotron models, but 𝗮𝗻𝘆 𝗢𝗽𝗲𝗻𝗔𝗜-𝗰𝗼𝗺𝗽𝗮𝘁𝗶𝗯𝗹𝗲 𝗲𝗻𝗱𝗽𝗼𝗶𝗻𝘁 𝘄𝗼𝗿𝗸𝘀
• The MCP server has 𝟵 𝗿𝗲𝗮𝗱-𝗼𝗻𝗹𝘆 𝘁𝗼𝗼𝗹𝘀. There is deliberately no tool to browse raw tables, so agents must go through the business terms
𝗜𝘁 𝗯𝘂𝗶𝗹𝗱𝘀 𝗼𝗻 𝗲𝗮𝗿𝗹𝗶𝗲𝗿 𝘄𝗼𝗿𝗸:
• Cube and dbt MetricFlow: semantic layers you write by hand in code
• Wren AI: an open semantic layer for AI agents, with a context layer built partly by agents
• Snowflake's Semantic View Autopilot: also drafts semantic models with AI, but only for Snowflake
What's different here: 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲, 𝘄𝗼𝗿𝗸𝘀 𝗮𝗰𝗿𝗼𝘀𝘀 𝗳𝗼𝘂𝗿 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲𝘀, 𝗮𝗻𝗱 𝗯𝘂𝗶𝗹𝘁 𝗳𝗼𝗿 𝗮𝗴𝗲𝗻𝘁𝘀 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝗠𝗖𝗣.
𝗧𝗵𝗲 𝗵𝗼𝗻𝗲𝘀𝘁 𝗽𝗮𝗿𝘁:
• 𝗧𝗵𝗲 "𝟵𝟬% 𝗳𝗮𝘀𝘁𝗲𝗿" 𝗰𝗹𝗮𝗶𝗺 𝗶𝘀 𝗡𝗩𝗜𝗗𝗜𝗔'𝘀 𝗼𝘄𝗻 𝘁𝗲𝘀𝘁𝗶𝗻𝗴, with no baseline published
• 𝗡𝗼 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝘀𝗰𝗼𝗿𝗲𝘀 𝗮𝗿𝗲 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱. NVIDIA ships an eval harness (BIRD, 1,534 questions) but no results
• That harness was tested with 𝗖𝗹𝗮𝘂𝗱𝗲 𝗢𝗽𝘂𝘀 𝟰.𝟴, 𝗻𝗼𝘁 𝗡𝗲𝗺𝗼𝘁𝗿𝗼𝗻
• 𝗪𝗶𝘁𝗵𝗼𝘂𝘁 𝗛𝗮𝘀𝗵𝗶𝗖𝗼𝗿𝗽 𝗩𝗮𝘂𝗹𝘁, 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗽𝗮𝘀𝘀𝘄𝗼𝗿𝗱𝘀 𝗮𝗿𝗲 𝘀𝘁𝗼𝗿𝗲𝗱 𝘂𝗻𝗲𝗻𝗰𝗿𝘆𝗽𝘁𝗲𝗱 in Postgres (per the README)
• It's young: 𝗮𝗯𝗼𝘂𝘁 𝟳𝟱𝟬 𝗚𝗶𝘁𝗛𝘂𝗯 𝘀𝘁𝗮𝗿𝘀, and NVIDIA isn't accepting outside contributions
𝗧𝗵𝗲 𝗯𝗶𝗴𝗴𝗲𝗿 𝗽𝗼𝗶𝗻𝘁: the hard part of AI analytics isn't writing SQL. 𝗜𝘁'𝘀 𝗮𝗴𝗿𝗲𝗲𝗶𝗻𝗴 𝗼𝗻 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲 𝘄𝗼𝗿𝗱𝘀 𝗺𝗲𝗮𝗻.
GitHub: github.com/NVIDIA/auto-ontol…
Links in reply 👇
Machine Learning Community ⭐️ retweeted
Software correctness needs more than tests!
Tests can cover thousands or even millions of cases, but they still only tell you what happened for the executions you actually tested.
Formal verification goes further by defining properties the software must satisfy and mathematically proving that those properties hold across the program states covered by the proof.
GenProof is applying this to real-world software using AI-assisted proof generation with machine-checked formal verification.
They've found 300+ bugs across open-source software including Linux, OpenSSL, curl, and Redis. Some of those bugs had been there for decades. Some were even missed by Anthropic in their formal verification release.
The defensive side needs to get stronger as AI gets better at finding errors and exploring software at scale. Formal methods let us ask whether a specific property of the code can actually be proven correct.
GenProof is releasing QuickProof for verifying properties of C code: you write executable specs, AI agents generate proofs, and Coq checks them. The guarantee depends on the stated property, assumptions and proof boundary.
Worth digging into if you work on software reliability or security.
Today, we're launching @genproof: a research lab making formal methods accessible.
We found bugs in 100+ OSS projects, including 6 libraries “formally verified” by Anthropic. We’re also top-1 on the lean-eval leaderboard.
With our tools, you too can become a verifier: 🧵
Machine Learning Community ⭐️ retweeted
🚨 Huge news 🚨 today we are launching SIE Serverless - a managed inference product focused on small models. It’s useful for spiky workloads on 20+ task specific workloads across Search, OCR and Agentic execution.
Sign up for a $500 credits launch grant 👇
It’s powered by our open source inference engine that has received thousands of github stars in the last few months, battle-tested across finance, legal & tech. We believe that self-hosting is going to be critical given all the trust & security issues in AI, but practically there are always workloads where you’d like somebody else to deal with the GPU quota like synthetic data generation, evals, public research and beyond.
The team at Superlinked has been burning the midnight oil for months and I’m so proud of every single one of you - what a privilege to call you my team mates y’all!
We built it, now help us make some noise 🙌
A like, a comment or a repost would mean a lot! ❤️
Machine Learning Community ⭐️ retweeted
Inference speed is becoming a model benchmark of its own!
For a long time, most model comparisons focused on intelligence, context length, and price.
But as more workloads become agentic and iterative, generation speed starts to matter a lot more. A coding agent, research agent, or RL rollout may make many sequential model calls, so reducing the time for each generation can change the wall-clock time of the entire workflow.
That is why we are starting to see more teams push inference speed much harder.
Looking across the models in this chart, what stands out to me is how differently teams are approaching the same problem.
Some are pushing Flash-class models for low-latency workloads, like DeepSeek V4.1 Flash running on Fireworks at around 1,500 tokens/s. Others are getting much more aggressive with serving around much larger models, like Kimi K3 reaching around 850 tokens/s on NVIDIA Rubin NVL72.
Naive takes a different route. They’re claiming up to 2,000 tokens/s on a single request with Naive-N0.5-Flash in Ultrafast mode.
Behind that is NaiveRT, an inference runtime built specifically for the model, combining speculative decoding, fused kernels, and GPU-driven execution to push single-stream generation speed much further.
What makes this trend interesting is that tokens per second is no longer just an infrastructure number.
Once models sit inside coding agents, research agents, RL rollouts, and other iterative systems, generation latency directly affects how quickly the entire workflow finishes.
Model quality still matters first, but inference speed is becoming another dimension that is increasingly hard to ignore.
Introducing Naive-N0.5-Flash: Building Frontier AI with AI
🔹 309B MoE, 15.5B active: top-tier in coding, leading in AI R&D.
🔹 Native 1M context, no full-attention layers (hybrid SWA + DSA).
🔹 Inference runtime built by AI: up to 2,000 tok/s in Ultrafast mode.
Weights are open today under MIT license.
🔗 Tech blog: naive.ai/en/research/
🤗 Hugging Face: huggingface.co/NaiveAI/Naive…
💻 GitHub: github.com/NaiveAI-Labs/Naiv…
🌐 naive.ai/en/
Machine Learning Community ⭐️ retweeted
RAG system that skips HTML parsing entirely!
PixelRAG is an open-source visual RAG framework that renders documents as screenshots instead of parsing them into text.
Most RAG pipelines start by converting HTML to text. Tables flatten into unstructured rows. Charts disappear. Layout context is gone before the LLM ever sees it. The paper measured this directly: HTML-to-text conversion accounts for 36.6% of retrieval failures on SimpleQA.
PixelRAG skips that step entirely. It renders pages as screenshot tiles using Playwright, embeds those tiles with a fine-tuned Qwen3-VL-Embedding model, builds a FAISS index, and passes retrieved images directly to a VLM reader. No text abstraction in between.
Benchmarked across six datasets against the strongest text-based baselines:
- SimpleQA: 78.8% vs 71.6% (+7.1 points)
- NQ-Tables: 48.8% vs 42.5% (+6.3 points)
- EVQA: +15.5 points
- LiveVQA: +11.3 points
One honest caveat from the authors: this requires Qwen3-VL-4B class models or larger to see the benefit. Smaller models trail text retrieval. The authors also recommend using PixelRAG as an enhancement layer alongside existing text systems rather than a full replacement.
Ships with a pre-built Wikipedia index covering 8.28M articles across 28.1M screenshot tiles. A Claude Code plugin lets Claude take screenshots of any URL and reason over the visual content directly.
Key capabilities:
• Renders web pages, PDFs, and images as screenshot tiles via Playwright
• Fine-tuned Qwen3-VL-Embedding model for visual retrieval
• FAISS index for fast vector search
• Pre-built Wikipedia index: 8.28M articles, 28.1M tiles
• 3x token cost reduction via image compression
• Claude Code plugin for direct URL screenshot and visual reasoning
• LoRA fine-tuning support via pixelrag-train
100% open source.
I've shared the link in the replies!
Machine Learning Community ⭐️ retweeted
Your AI agent may not need a complex memory system!
A new paper called RAW-RAG tests something much simpler: keep the raw conversation history, retrieve the relevant parts, rerank them, and let the LLM reason over the original evidence.
No memory summarization, no graph construction, and no LLM-based memory extraction during ingestion.
The pipeline is basically:
Raw history → Dense retrieval → Reranking → Reader LLM
And surprisingly, that baseline ranks first among the compared memory systems in 6 out of 7 settings across LoCoMo, LongMemEval, MEME, and MemoryAgentBench.
In some settings, RAW-RAG beats the strongest compared agentic memory system by more than 5 points.
The interesting part is why.
Agentic memory systems usually transform conversations into cleaner memories: extract facts, update old information, summarize events, or organize them into graphs.
But every transformation creates another opportunity to lose information.
The paper finds that retrieving the right answer value alone is not always enough. The model may also need the temporal relationships, previous events, and dependency paths around that information to work out what is currently true.
Raw history preserves those relationships.
They even test giving the model complete access to transformed memory stores, and there is still a gap compared with the original raw evidence.
Another interesting result comes from memory mutation. In their tested Mem0 setup, disabling UPDATE and DELETE actually improved accuracy, suggesting that capable reader models can sometimes resolve changing state directly from the underlying history instead of relying on the memory system to maintain the “correct” state for them.
This doesn’t mean agentic memory is useless.
But if a complex memory architecture can’t beat retrieval + reranking over the raw history, that should probably be the baseline you beat before adding more memory machinery.
I’ve shared link to the paper in comments!
Machine Learning Community ⭐️ retweeted
Jev as a reviewer for OCR!
OpenBMB and ModelBest just released MiniCPM-OCR 1.0 for document parsing and MiniCPM-OCR Jev for reviewing the reliability of those outputs.
That second part is what I find more interesting.
OCR errors are often hard to catch once the output looks plausible. A number can be slightly wrong, a formula can be misread, or a table cell can shift, and the result can still look completely reasonable when it moves into the next step of the workflow.
MiniCPM-OCR Jev sits after the parser and evaluates how reliable those outputs are across text, formulas, and tables.
It does not rewrite or correct the OCR result.
Its job is to score the output and help decide which blocks can move forward and which ones should be sent for human review.
So the workflow becomes:
Document → Layout detection → OCR parser → Jev review → Human review where needed
On the parsing side, MiniCPM-OCR 1.0 is a 1.3B model that supports scanned documents, photographed pages, curved inputs, tilted documents, tables, formulas, and key information extraction.
OpenBMB reports 93.68 on Real5-OmniDocBench and 54.7 on OCRBench v2, while DSpark improves end-to-end throughput by 62% in their benchmark setup.
On the review side, MiniCPM-OCR Jev reaches 93.5 AUROC on their JevOCRBench, close to Qwen3.5-397B-A17B at 93.8 and above JEV-35B at 90.1 in the official evaluation.
The useful part here is that you no longer have to treat the whole OCR output as equally reliable.
You can review it block by block and route only the uncertain parts for another look.
I tested both models on my own documents to see how well Jev catches OCR outputs that look right but are actually wrong.
I’ve shared the GitHub repo below!
Machine Learning Community ⭐️ retweeted
RAG for decision models is here!
Parallel just launched the Decisions API, which combines live web search with bounded decision-making in a single call.
Decision models like Jev are useful when the output is small and structured: yes/no, one option from a list, a category, or a routing decision.
The problem is that those decisions are only as good as the context you give them.
If the answer depends on something that changed recently, or information that only exists on the web, the normal flow becomes retrieval first and decision second.
Parallel brings those two steps together.
You provide the state and the questions you want answered. The API searches the web, grounds the decision in what it finds, and returns the typed answer along with probability, confidence, and sources.
That makes it much more useful for agent workflows where a decision depends on current external information, research tasks where the model needs to verify something before choosing an outcome, or eval pipelines where the judgment needs grounding from the web.
It can also answer multiple questions against the same retrieved context in one request, instead of repeating the search for every decision.
So instead of:
Search → retrieve context → decision model → structured answer
you can make the grounded decision directly through one API.
I think this is a pretty natural extension of where decision models are going: bounded outputs are useful, but giving that decision layer access to the live web makes the set of problems it can handle much larger.
Introducing the Parallel Decisions API: the first web-grounded decisions engine.
Decision models like Jev can answer in an instant, but they lack the recency of current news. Now you can get both in a single call.
Decisions API is live today at $0.001 per request.
Try it with 5,000 free requests: platform.parallel.ai/play/de…
Here are some examples we've been experimenting with:
- Fact-check content (@Grok is this true?)
- Refresh stale banking records
- Classify companies and leads with fresh signals
- Backtest trading strategies
- Track competitor e-commerce inventory
Machine Learning Community ⭐️ retweeted
Long-document OCR has an attention problem!
Baidu just released Unlimited OCR 2, an open-source OCR model built around a new Sparse Reference Sliding Window Attention mechanism for parsing extremely long documents efficiently.
The problem is that normal attention gets increasingly expensive as the document grows. Every new page adds more visual tokens, more text tokens, and more history that the decoder may need to attend over.
Unlimited OCR 2 changes that with SR-SWA.
Instead of keeping all text-visual interactions active across the full document, SR-SWA compresses them into a fixed-size set of working references. In Baidu’s setup, that can be as small as 512 tokens.
Two parts make that possible.
Page Gate first decides which pages are actually relevant to the current decoding step.
Token Sieve then filters the visual tokens inside those pages, keeping only the references the decoder needs instead of repeatedly attending over everything it has already seen.
That keeps the working attention state bounded even when the document itself keeps growing.
So the document can move from dozens of pages to hundreds or thousands of pages without the per-step decoding cost growing at the same rate.
Baidu reports up to 7× faster per-step decoding than Full Attention and 5.6× faster than the previous R-SWA approach.
Its page-level chunking and KV-cache reuse also reduce peak KV-cache memory from 47 GB to 1.77 GB in the B=1 setup.
It also reaches 84.6 on MPDocBench and 77.1 on MarathonDocBench.
And this is not just a benchmark setup. They demonstrate parsing all seven Harry Potter books, more than 3,600 pages, and the entire Three-Body Problem trilogy in a single run.
The architecture is the interesting part here.
Instead of solving long-document OCR by simply increasing the context window, Unlimited OCR 2 changes what the decoder keeps paying attention to as the document grows.
100% open source.
I’ve shared the GitHub repo in the comments!
Machine Learning Community ⭐️ retweeted
Claude Code, Codex and Hermes can now work as a team!
BAND lets your local coding agents work together: they share context, hand off work, and unblock one another.
Normally, running multiple coding agents means running multiple separate sessions. Claude Code might handle the backend, Codex the frontend, and another agent reviews the work, but you still end up being the person carrying context between them.
BAND gives those agents a shared room around the task.
One agent can create the plan, split the work into tasks, and hand pieces off to other agents. Those agents can message each other directly, pass the context with the work, and keep the current state outside any single coding session.
Your coding sessions still run locally in the tools you already use. BAND sits around them as the coordination layer, so you can see what each agent is working on, what is blocked, and where a human decision is actually needed.
That also means the planner or reviewer does not have to disappear once implementation starts. They can stay in the same room while the work progresses and step back in when something changes.
Key capabilities:
- Shared context across coding agents
- Direct agent-to-agent messaging and handoffs
- Shared plans, tasks, and work ownership
- Local coding sessions with centralized visibility
- Human decisions and approvals inside the workflow
The interesting part here is that adding more coding agents is easy. Getting Claude Code, Codex, Hermes, or other agents to actually coordinate around the same task is the harder problem.
BAND is building that shared layer around them.
I’ve shared the link in the comments!
Machine Learning Community ⭐️ retweeted
Jev as a decision layer for LLM agents!
LLM agents make a lot of small decisions between the bigger reasoning steps: which tool to call, which argument to use, whether the task is done, or what should happen next.
JevAgentHarness explores what happens when those bounded decisions are shared with a much smaller decision model instead of sending every step back to the LLM.
The decision model returns probabilities over the available options. If its confidence is above a threshold learned from the agent’s own trajectories, the harness uses that decision directly. If confidence is too low, control goes back to the LLM.
But the paper found that standalone decision accuracy is not enough.
The model that performed best on isolated decision questions completed only 69.5% of full agent tasks. And when tool choices were delegated without fitting the confidence threshold, task success dropped by 19 points.
The reason is that agent decisions are sequential. One wrong tool call changes the state the agent sees next, so later decisions happen in situations the decision model may never have seen before.
To address that, the researchers trained AgentDecision-4B on the states agents actually reach, including states that appear after earlier mistakes, with Jev providing the decision targets.
That changes the harness from simply replacing LLM calls with a smaller model into a confidence-based control system where both models handle the decisions they are better suited for.
On tasks similar to its training distribution, the resulting 4B decision models reduced agent cost by 81.1–89% and made execution up to 2.5× faster without losing 3 or more points of task success.
And on shifted tasks, AgentDecision-4B reached 91.5% task success compared with 82% using Jev.
The broader takeaway is useful: if you want decision models inside agents, benchmark accuracy alone is not enough. You need to evaluate them inside the actual agent loop, on the states their own decisions create.
The harness, benchmark, training data, and models are open source.
I’ve shared the paper in the comments!
Machine Learning Community ⭐️ retweeted
Google open-sourced Harness²!
Harness² is a framework for recursively improving the harness around an agent while keeping the underlying model weights fixed.
The interesting part is that it does this at two different levels.
For the current task, Harness² uses a Propose → Probe → Compose loop. It proposes changes to the harness, runs those variants on the same task, compares how the executions behave, and then composes a better harness from that evidence.
That is the task-level recursion.
But the process does not reset after every task.
Harness² also keeps the harnesses it adopted and the lessons supported by execution evidence, then uses those to guide how future harnesses should be improved on new tasks.
That is the harnessing-level recursion.
So one loop improves the harness for the task in front of the agent.
The second loop improves how the system learns to build better harnesses over time.
The editable harness itself can include the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins.
You can also run the recursion in two ways:
Parallel recursion explores more candidate harnesses.
Sequential recursion carries the refined harness and accumulated evidence through multiple Propose → Probe → Compose steps.
Google reports that even a single Harness² step improved GLM-5 and Gemini 3.5 Flash across JobBench, LAB, and WorkBuddy-Bench, with larger task-level recursion budgets bringing further gains.
That is what makes Harness² more interesting than just another agent framework.
It is treating the harness itself as something that can be iteratively improved from execution evidence, both within a task and across tasks.
100% open source.
I’ve shared the GitHub repo in the comments!
Machine Learning Community ⭐️ retweeted
Reverse engineer any app with an AI agent!
REA gives coding agents the tools to inspect shipped software, understand how a feature works, and recreate that behavior in your own project even when you don’t have the original source code.
You can point your agent at an app or a specific feature and ask it to investigate how that behavior is implemented.
REA handles the reverse-engineering layer underneath. It can inspect native binaries, JavaScript and Electron apps, .NET applications, websites, and other packaged software, then return the evidence behind what it found.
The agent can follow that evidence across functions, imports, strings, modules, runtime behavior, and other parts of the application until it has enough context to explain how the feature actually works.
The useful part is what happens after that.
Your coding agent can take those findings and build a version of the same behavior for your own project, adapted to your stack and requirements.
So if you like how search, offline mode, sync, or another feature works inside an app, you can ask the agent to investigate that specific behavior instead of trying to infer everything from the UI.
REA also keeps the investigation evidence alongside its conclusions, so the agent can show why it believes something works a certain way instead of simply guessing.
And it doesn’t claim to magically recover the original source code. The goal is to understand the shipped software well enough to reconstruct the behavior.
Key capabilities:
- Inspect apps and features without access to the original source
- Reverse engineer native, JavaScript, Electron, .NET, and browser targets
- Trace how a feature works across functions, modules, imports, and runtime behavior
- Return evidence and limitations behind each conclusion
- Recreate the learned behavior inside your own project with a coding agent
REA works through both MCP and CLI, so tools like Claude Code, Codex, Cursor, and other coding agents can use the same reverse-engineering workflow directly.
100% open source.
