Benoît Quartier

The State of Generative AI in the Enterprise and what i says to me! Last week, @MenloVentures released its 2024 report, which surveyed 600 U.S. IT decision-makers. Here is what it says to me: 👀 Open-source AI is slowly taking over. Enterprises want to build their own AI solutions. Infrastructure spending has increased 8x, deployment investment has increased 3.8x, and build vs. buy strategies have shifted from 20% to 53%. The complexity of deploying your own AI models takes more time compared to the faster, simpler option of using APIs. Combining this with open models approaching closed performance and @OpenAI losing its market share, the direction seems clear. I was expecting fine-tuning to decrease, not because it is not needed. More because, with better models, it moves to a later stage in the lifecycle of AI applications. Thats why see an increase in RAG. As a company, you should always try to make things work and then optimize. Focusing on UX/DX, reliability (evaluation, tracing, logging) is more important for most companies in the current state. Fine-tuning will help these companies drive down costs significantly later. Workflow automation will be the biggest growth area in the next two years in terms of use case adoption. Vector databases likely won’t win long-term in enterprises. PostgreSQL and MongoDB's vector capabilities are good enough and already present in most companies. The real challenge isn't about model capabilities anymore. We're in a similar situation as with cloud adoption in 2010. My advice is simple: build, learn, and iterate. Enterprises that invest in understanding these technologies will have significant advantages in the future.

OCR-2.0 is coming, and Generative AI and multimodal LLMs will power it! 🔍 GOT (General OCR Theory) is a 580M end-to-end OCR-2.0 model that outperforms all existing methods. GOT consists of a Vision-Encoder to convert images into transformers images into tokens and a decoder for generating OCR outputs in various formats (e.g., plain text, markdown, Mathpix). GOT is designed to handle complex tasks like sheets, formulas, and geometric shapes Implementation 1️⃣Vision Encoder Pre-training: The encoder (VitDet) trained using scene text and document OCR data to recognize both slice and whole-page inputs. 2️⃣ Joint-Training: The encoder is connected to the decoder (Qwen-0.5B) and both are trained on more general OCR tasks (e.g., formulas, sheet music, geometry). 3️⃣ Post-Training: Fine-tuned the model with specific tasks, such as fine-grained OCR, multi-page PDFs, and dynamic resolution, using new synthetic datasets. Insights 🧠 Encoder-Decoder with 80M (VitDet) Encoder and 500M (Qwen2) Decoder with 8k context 🥇 Achieves a 0.035 edit distance and 0.972 BLEU score on plain OCR 📊 Outperforms LLaVA-NeXT and Qwen-VL-Max in document and scene text OCR 🧮 Can extract LaTeX formulas from Arxiv and convert them to Mathpix format 📃 Supports dynamic resolution and multi-page OCR 🖼️ Input resolutions up to 1024x1024 Paper: huggingface.co/papers/2409.0… Github: github.com/Ucas-HaoranWei/GO…