Redefining RAG for AI products and agents. Open-source data parsing and retrieval infrastructure by Ontos.
Joined September 2025
- Tweets56
- Following25
- Followers7
- Likes76
Converting PDF to Markdown is solved. Giving AI agents the structure, tables, and page citations to verify answers is the hard part.
We compared Marker v2, MarkItDown, MinerU 4.0, and Knowhere for agentic RAG—including when NOT to use our tool 👇
With Knowhere 2.0, we made a technical bet: parsing and retrieval shouldn’t be separate pipelines.
Vision + text now converge into one memory schema, and agents decide how to search, traverse, read, and cite it.
Fewer brittle chunks. Less wasted context. Better evidence.
Really excited to finally share this: Knowhere 2.0 is live! 🎉
Over the past few months, we kept coming back to two problems.
First, real-world documents are messy. Scans, dense tables, slide decks, engineering drawings—they don’t always survive being flattened into Markdown.
Second, once the knowledge base gets large, agents can easily get lost. They need to know which file to open first, which section to read next, and when they have enough evidence to stop searching.
So in Knowhere 2.0, we made two big changes:
👁️ Dual-track parsing brings VISION-MAP and text parsing together, helping Knowhere understand a much wider range of complex documents.
🧭 Agent-native retrieval lets the agent choose how to search based on the task—whether that means a quick lookup or a deeper journey across files, chapters, and versions.
The result: better retrieval, fewer wasted steps, and lower token usage.
In the article below, I walk through two practical examples: an engineering knowledge base and a due-diligence review.
Would love to hear what you think—and what kinds of difficult documents you’d like us to try next.
👇 Read the full story here:
medium.com/@13512395620/know…
Knowhere 2.0 is live. 🚀
Real-world documents don’t fit neatly into Markdown—and agents often get lost as the knowledge base grows.
2.0 adds dual-track parsing and agent-native retrieval, helping agents understand more files and decide what to read next.
Read more 👇
medium.com/@13512395620/know…
3,000 GitHub stars in just 3 months. ⭐
We’re genuinely blown away.🤗
We built Knowhere because turning documents into Markdown isn’t enough. AI agents need structured memory they can navigate, retrieve, cite, and reason over.
Here’s what we built beyond MinerU—and why it resonated 👇
medium.com/@13512395620/how-…
Knowhere retweeted
Everyone in AI is obsessing over memory right now, mostly because our models are still pretty dumb. You tell it something, open a new chat, and poof—it’s completely gone.
Funny thing is, brain surgeons saw this exact same thing 70 years ago.
Back in 1953, a patient had his hippocampus removed during a rare surgery. After that, he literally couldn't form new memories at all. He was just stuck in 1953 forever.
That was when people finally realized: human memory isn't just one big thing. It has layers. Different parts handle different jobs, and if you take one piece out, you lose that whole ability and the rest of the brain can't work together anymore.
The AI we have today is basically that patient.
Everything we're doing right now is just copying biology—trying to give AI those missing layers of memory so it can actually remember things.
Put together a quick piece on this 👇：
medium.com/@13512395620/ais-…
Current AI has the exact same brain damage as a famous medical patient from 1953.
Think about it: you tell an LLM your entire life story, start a new chat, and poof—it doesn't even know your name.
Most people think this is a compute problem. It’s not.
70 years ago, a neurosurgeon removed a patient’s hippocampus. The result? The man literally couldn't form new memories. He was trapped in 1953 forever.
That single surgery proved memory isn’t one big storage drive. It’s 7 distinct biological layers working together. Knock one out, and everything breaks.
Today, we're essentially trying to cure AI of that exact condition—by copying biology to patch in those missing layers.
Broke down the full story and what it means for the next generation of agents👇：
medium.com/@13512395620/ais-…
Lately, our team rolled out a new visual parsing route for Knowhere: VISION-MAP.
We built it because traditional text chunking completely ruins complex tables, stamps, and engineering drawings. Once you flatten a layout into Markdown, the relations between data points just disappear.
With this pipeline, the AI navigates the document tree to find the right candidate pages, then reads the raw page visually—keeping every answer tied to the exact source page.
It's mainly for finance, engineering, and compliance tasks where guessing isn't an option and you need proof.
Put together a write-up on the architecture and how to use it over on Medium. Give it a read if you're curious:
medium.com/@13512395620/how-…
Knowhere retweeted
Hot take:
Most AI teams don't have a model problem.
They have a context problem.
And many context problems are actually document problems in disguise.
The industry is busy talking about Context Engineering, Agentic RAG, MCP, and multi-agent systems. But if your PDFs are parsed poorly, you're building on a shaky foundation.
A practical comparison of Reducto, LlamaIndex, Unstructured, MinerU, and Knowhere for anyone building production AI systems.
medium.com/@13512395620/how-…
🤖 Made with AI
A lot of AI agent problems start before the LLM ever sees the data.
Bad parsing → bad context → bad answers.❌️
PDF parsing isn’t just plumbing anymore. It’s part of the AI stack.
I put together a practical look at Reducto, LlamaIndex, Unstructured, MinerU, and Knowhere — and when each one actually makes sense.👍
Hope this helps others make more informed technical decisions and choose the right tools for their use case.
medium.com/@13512395620/how-…
🤖 Made with AI
Knowhere retweeted
Honestly, this started from a pretty simple ask in the comments:
“Can I just hook Knowhere into my own agent?”
Fair question. We were already parsing messy docs well — but if that knowledge only lived in the browser, it felt half-finished.
So we shipped Notebook + MCP.🚀
Now the docs you keep in Knowhere can show up in Cursor / Claude Code / Codex. Same account, same cloud store. Agent can search it; with Full access it can also parse new stuff back into the Brain.
Still early, but this is the direction that makes sense to me.
medium.com/p/knowhere-can-no…
We kept seeing the same question from users:
“Can I connect Knowhere to my own agent?”
So we built that.
Knowhere Notebook and Knowhere MCP are live.
Manage your docs in Notebook.
Mount the same library into Cursor, Claude Code, Codex, and other MCP clients.
Same account. Same cloud Brain. Search from your agent — or, with Full access, parse new docs straight back into Knowhere.
If you’ve been waiting for this, go try it and tell us what breaks / what’s missing.
medium.com/p/knowhere-can-no…
Knowhere retweeted
After building RAG systems for the past year, I've come to a somewhat controversial conclusion:
90% of the work has nothing to do with the model.
If I had to break down where engineering time actually goes, it would look something like this:
📍50% Evaluation
📍40% Data curation
📍8% Business integration
📍2% Model training
The biggest failures I've seen weren't caused by weak LLMs. They came from outdated documents, missing metadata, broken chunking, poor retrieval, and a lack of clear evaluation criteria.
Swapping models is easy.
Building a system that knows which document is valid, who it applies to, and when it became effective is the hard part.
This realization has fundamentally changed how we build AI infrastructure.
The goal isn't just to retrieve text—it's to preserve the structure, context, and provenance that make knowledge reliable in the first place.
Curious whether others building production RAG systems have reached the same conclusion.
medium.com/p/does-90-of-a-ra…
The biggest lesson we've learned from building RAG systems:
Better models don't automatically create better products.
In practice, most of the work goes into evaluation, data quality, and preserving the context behind enterprise knowledge—not model training.
Reliable AI starts with reliable information.
That's the philosophy behind everything we build at Knowhere.
medium.com/p/does-90-of-a-ra…
👏👏
Knowhere just crossed 1,500 GitHub stars about a month after we open-sourced it. 🚀
One question I keep getting is:
“How is it different from MinerU?”
My answer: MinerU solves parsing. We're solving what comes after parsing.
Converting a PDF into Markdown is only the beginning. Once you chunk it for RAG, you lose hierarchy, context, table relationships, image semantics, and cross-document links.
That's exactly where many AI Agents start to fail.
Knowhere rebuilds document structure into an AI-native memory layer, preserving navigation paths, multimodal relationships, and semantic context so agents can retrieve and reason instead of stitching together random chunks.
We're not trying to replace great parsers like MinerU.
We're trying to bridge the gap between "document parsing" and "documents that AI can actually use."
The response from the open-source community has been amazing. 1,500 ⭐ is just the start.
The bottleneck for Agentic RAG isn't always the model. Sometimes it's how we prepare memory.
🔧GitHub：github.com/Ontos-AI/knowhere
🌐Knowhere：knowhereto.ai/?utm_source=x
Know more about Knowhere👇
medium.com/p/1-500-stars-in-…
🎉 1 month. 1,500+ GitHub Stars.
Thank you to everyone who starred, contributed, and tried Knowhere. ❤️
Most document tools stop at parsing PDFs into Markdown.
Knowhere goes one step further.
We turn documents into AI-ready memory by preserving hierarchy, linking tables and images, and organizing knowledge for Agentic RAG and AI Agents.
✨ Better retrieval
✨ Richer context
✨ Structured memory instead of fragmented chunks
If you're building RAG, enterprise knowledge bases, or AI Agents, give Knowhere a try and see the difference.
🔧GitHub：github.com/Ontos-AI/knowhere
🌐Knowhere：knowhereto.ai/?utm_source=x
⭐ Star the repo. Test it with your own documents. Tell us what you'd build.
#OpenSource #RAG #AIAgents #LLM #GenAI
Read more about Knowhere👇
medium.com/p/1-500-stars-in-…
Welcome everyone to use our project! You can directly use knowhere on our official website. Welcome everyone to give it a try! We hope our work can be of great help to developers and users!👏👏👏
给搞 RAG 和 AI Agent 的兄弟安利一下 Knowhere：它专门解决非结构化文档进大模型前的处理问题。不管是 PDF、Office、图片还是 Markdown，它能自动解析出带层级结构的语义块，而不是简单暴力分片。每个 Chunk 还保留原文出处，检索时直接定位证据。
核心用法：1）把脏乱文档丢进去，吐出 Agent 直接能用的记忆层；2）做向量 RAG 或 Agentic RAG 的预处理；3）构建知识图谱前的文档结构化。文档支持多模态，输出带导航树和摘要。
有云端 API 可直接跑，不想自托管可以试一下，输入写了给 $5 免费额度。5 月刚开源，支持自部署整套流程。适合谁：我在整理内部知识库的时候最需要这种，想做私有知识库或文档流水线的可以先看看 README 了解parsing逻辑再决定。
🔗github.com/Ontos-AI/knowhere
Thank you very much for using our project. We also noticed some problems with RAG, which led to the development of Knowhere. We welcome you to use it frequently. If you have any suggestions, please feel free to put them forward at any time!
下班前看到这个项目，有点震惊——Knowhere直接把你手里乱七八糟的文本、PDF、网页自动提取内容，解析成结构化块，AI Agent 和 RAG 直接能用。
本质上就是把非结构化数据的“最后一公里”干了，而且干得很彻底。平时要么丢全文，要么手动切块，它直接抽成干净的结构化数据块。
离谱的是输出直接为 Agent 准备，不需要再手写一堆解析逻辑。说白了，这才是 RAG 系统真正缺的那个环节。
github.com/Ontos-AI/knowhere
Thank you for sharing our project. We hope the content we have created can help more users and developers and improve their work efficiency!🥳🥳
Your RAG pipeline gets worse when documents become flat chunks.
Knowhere is an open-source document memory layer for AI agents that turns messy unstructured files into structured, navigable context.
It helps you prepare documents for agentic RAG by parsing files, rebuilding document hierarchy, linking chunks into a lightweight memory graph, and returning traceable evidence instead of isolated snippets.
Key features:
• Multi-modal parsing – extracts from PDFs, Office files, images, Markdown, text, JSON, and CSV
• Hierarchy reconstruction – rebuilds section trees so chunks keep semantic context
• Memory graph – stores chunks, navigation trees, summaries, and graph links for agent-ready retrieval
• Agentic retrieval – combines keyword, path, content, and semantic signals with agent navigation
• Evidence-based citations – returns source document, section, chunk, and linked asset references
It’s open-source (Apache License 2.0).
Link in the reply 👇
