Vik | Practical AI & Tech (@uctive) header
Vik | Practical AI & Tech (@uctive) profile picture

Private local AI server for small businesses: your data stays inside your company. https://nitter.cf/t.co/dnhFJ2Yoqh

Como, Italy
Joined December 2011
🚀 Strata vs optimized llama.cpp — RTX 5090, 24GB VRAM. Same 54.5GB AI model. Same hardware. ⚡ Prefill: up to ~14× faster 🧠 8 agents: ~17× higher throughput 🔋 ~4× better GPU energy efficiency We tested the latest Strata against a tuned llama.cpp. Results & caveats 👇
4
3
2
45
3,168
🚨 Strata vs Basalt — benchmark COMPLETE. Same Qwen3.8-Flash-Next 54.5GB IQ3_S MoE. Same RTX 5090 Laptop (24GB). ⚡ Strata: multi-agent throughput & efficiency 🚀 Basalt: single-stream decode 🔥 Cached prefill: up to 275× faster llama.cpp excluded: our agent misconfigured its MoE GPU cache and context slots. Retesting. 🧵👇
5
4
1
33
4,579
Replying to @uctive @since1795
Удивительные вещи происходят - Гугл сообщил и оно сразу в приложении 🤣
1
23
7/7 — Why I’m not waiting for perfect automation Perhaps the most important point in this entire story is that I don’t need to build a perfect system before I can benefit from it. I still perform plenty of manual operations. Sometimes I simply copy a message from WhatsApp or Telegram and paste it into ChatGPT, Grok, or another AI. 📋 Good old copy-paste. Of course, these workflows can also be automated. Most of the necessary tools already exist. And I’m gradually connecting systems, adding integrations, expanding agent capabilities, and reducing the number of operations that require my direct involvement. But automation for its own sake doesn’t interest me. If something takes a few seconds and doesn’t create a meaningful inconvenience, I’m perfectly happy to keep doing it manually for now. Meanwhile, agents can work on programming, testing, infrastructure monitoring, document analysis, and other tasks. That’s why I don’t see local and cloud AI as competitors. What matters to me is: 🧠 Capability and intelligence 🎯 Output quality ⚡ Speed and performance 📚 Access to the right context 🔐 Privacy and security 🤝 Convenience and practical usefulness 🏠 One agent works locally. ☁️ Another analyzes its activity from the cloud. 🔍 A third helps investigate a technical issue. Sometimes I step in and make a decision. Sometimes a task is completed without my involvement. Meanwhile, I can run cables, connect equipment, deal with business matters, spend time with my children, or prepare chicken for smoking. November 18 was an exceptionally busy day, and my current system is still far from perfect. But I can already see its greatest advantage: AI is gradually becoming a natural part of my workflow and everyday life, rather than a separate application I have to sit in front of for hours. To me, that’s far more interesting than endless comparisons of models based on impressive-looking performance numbers. Although I’m certainly not giving up measuring tokens per second. 😁
1
37
6/7 — How local and cloud AIs interact with each other I currently use several cloud AI systems: Grok, ChatGPT, Gemini, Claude, and additional models through APIs. Locally, I run Hermes and other agents and models. But this isn’t simply a collection of independent chatbots that each require me to explain everything from scratch. There are already several mechanisms connecting them. 🔧 For example, Grok can manage Hermes, inspect its activity, and help troubleshoot technical problems when Hermes encounters failures. 💻 ChatGPT can examine not only public repositories and published technical results, but also my local computing environment through coding and computer-access tools such as Codex. 📂 Agents can use shared information in OpenProject and Git, while also communicating through APIs and other integrations. But I have another particularly interesting mechanism. Hermes has its own desktop environment running on a local server. That desktop includes a browser with access to the web versions of the cloud AI systems I use. When a direct integration is missing or temporarily stops working, Hermes can interact with a cloud AI through its regular web interface — opening the page, asking questions, and receiving answers. In other words, a local artificial intelligence can literally use another artificial intelligence through a browser, just like a human would. It sounds slightly ridiculous, but in practice it makes perfect sense. 😁 🔐 Naturally, information sent to cloud services still needs to be controlled. One of the central goals of my architecture is to keep confidential information local and only send data outside the private environment when appropriate. And I’m not claiming that every system mentioned here is already fully interconnected and automated. ✅ Some connections work directly. 🔨 Others still need to be implemented. 🚀 Much of the larger-scale automation is still ahead of me.
1
60
5/7 — How news becomes tasks, and how different AIs share context Suppose I come across a post on X about a new language model, inference engine, reverse-engineering technique, interesting hardware, or an economic development. I find it interesting, but I’m not planning to investigate it immediately. Instead of saving dozens of bookmarks that nobody will ever open again, I simply tap Share and send the link to Grok. Grok already knows what to do with it. It performs an initial analysis based on my current projects and tasks: 🎯 How relevant is this information to me? 🔍 Could this technology be useful? 🚨 Does it require immediate attention? 🧪 What experiments or benchmarks should be performed? 🛠️ What would need to be changed or implemented? 📊 What priority should the task receive? It then creates a structured task in my local OpenProject instance. The entire operation usually takes just a few seconds of my time. 📋 OpenProject is one of the shared information hubs accessible to different AI systems. It stores tasks, plans, and priorities. The second major shared information layer is Git. I have more than ten public and private Git repositories containing source code, documentation, technical reports, benchmark results, change histories, logs, and other information. After experiencing GitHub reliability issues, I migrated a significant portion of these repositories to local Git infrastructure. Different agents can access this information, inspect projects, and manage them within their assigned permissions. This means project context doesn’t constantly have to be manually transferred from one conversation to another. 🤖 One agent creates a task. 📚 Another can analyze the associated documentation and code. ⚙️ A third can perform the actual technical work. This is still far from perfect automation, but it’s already a genuinely useful way for multiple AI systems to work together.
1
38
4/7 — Why three hours on X are also part of my workflow That day, X accounted for 3 hours and 13 minutes of my screen time. But X serves three very different purposes for me. 💼 1. Business and account management This is practical work connected to my projects, business, technical publications, discussions of experimental results, and communication with people working on similar problems. 📰 2. News and research I follow developments in AI, new models and inference engines, programming, reverse engineering, hardware, equipment prices, business, finance, and macroeconomics — a subject I’ve studied and followed for decades. X allows me to understand what’s happening remarkably quickly. I’ve learned to move rapidly between different information bubbles, including communities with completely opposing viewpoints. This helps me see how different groups interpret the same events. When I need a deeper analysis, I simply ask ChatGPT to gather information from different sources, examine arguments from multiple perspectives, fact-check claims, and prepare a summary. As a result, I click through to external articles and read lengthy pieces much less often. Usually, this combination is enough: rapid information discovery on X, followed by deeper investigation through AI. 🚀 3. Entertainment and curiosity Reading the news, enjoying memes, following politics and football, exploring history and science, watching Starship launches in amazement, and discovering countless other fascinating things. Sometimes I just want to admire an enormous rocket lifting off into the sky instead of reading about another inference optimization. Although, knowing myself, I’ll probably end up asking an AI to analyze the rocket engines anyway. 😁 But here’s the most interesting part: X is not just an information source for me. It’s directly connected to my agents’ task planning and execution workflow.
1
18
3/7 — Real experiments: from smart lighting to preparing chicken One of my experiments involved home automation. I noticed that the basement lights were on for no reason. I asked my local agent what was happening and started investigating the lighting controls. One stage of the experiment took about 3.5 hours. That doesn’t sound particularly impressive if you imagine an artificial intelligence spending three and a half hours trying to turn off a light bulb. 😁 But in reality, the agent spent most of that time waiting for my commands and confirmations. That’s precisely one of the issues I’m working on: increasing agent autonomy and enabling agents to make more decisions independently, within defined permissions. At the same time, I was continuing other improvements to my home infrastructure. Recently, I moved my servers from a small storage room, where the air temperature reached approximately 30°C, to the basement, where it’s now below 20°C. The servers are much more comfortable now, but almost all the heat they generate is wasted. Previously, at least some of that heat helped warm the house. Apparently, my AI now enjoys better living conditions than its owner. 😁 Beyond physical infrastructure, ChatGPT and I discussed a secure local environment for private family and business documents, hybrid AI architecture, inference engine testing, long contexts, performance, and different hardware configurations. I occasionally use Gemini to independently double-check technical conclusions. But it would be strange to use all this technology exclusively for servers. For example, ChatGPT and I also discussed how to prepare chicken legs for smoking. The same AI helps me design multi-agent systems, analyze hardware, and prepare dinner. The important thing is making sure it doesn’t confuse the smoker’s temperature with the maximum safe GPU temperature. And, of course, I have children, a family, and ordinary household responsibilities. I still prefer interacting with my children directly rather than through AI agents. 😁 This combination of technical work, business tasks, and completely ordinary human activities is where I see the real value of AI.
1
37
2/7 — How much time did I spend using AI, and what was I doing? 📱 Here’s the screen-time breakdown from my main iPhone on November 18: 🤖 ChatGPT — 5 hours 26 minutes 🔍 Gemini — 19 minutes 🏠 Hermes — 1 hour 53 minutes 💬 Telegram — 57 minutes 𝕏 X — 3 hours 13 minutes Telegram is primarily an interface to my local Hermes agent, so I combine those two: 2 hours 50 minutes. That gives us 8 hours 35 minutes of direct AI-related screen time: ☁️ 67% — Cloud AI (ChatGPT and Gemini) 🏠 33% — Local AI (Hermes and Telegram) Grok was missing that day because I had exhausted my usage limits. Normally, it also handles a substantial amount of work. But there’s an important distinction: these numbers measure app screen time, not continuous active conversations with AI. They also exclude autonomous agent activity. I rarely sit in front of a computer. Almost all project management, agent supervision, and infrastructure control happens through my phone. And November 18 was particularly busy. I was running a cable from my solar inverter down to the basement, working on heat pump hardware, connecting devices, relays, cameras, and dealing with various other physical tasks. Meanwhile, agents were handling much of the software work. I periodically checked their progress, clarified instructions, and approved actions. In other words, a significant part of my day was spent not in front of a monitor, but with tools and cables in my hands. Agents can already write code, analyze systems, and control software. But they still can’t physically connect a relay to a wire or install a camera. I don’t have robots yet. So I’m gradually delegating the intellectual work to AI, while the jobs of electrician, installer, and general handyman still belong to me. 😁
1
2
194
1/7 Today, I’ll use a real example to show how I use local AI alongside several cloud AI systems — ChatGPT, Grok, Gemini, Claude, and additional models via API. How they interact, what’s already automated, and how convenient this actually is in everyday life. On November 18, my iPhone recorded 18 hours and 27 minutes of screen time in a single day. No, I haven’t lost my mind. Although sometimes I wonder. 😁 It was an exceptionally busy day, and my phone is my primary work interface. I barely ever sit down at a computer.
1
1
4
236
A quick progress update from my local AI experiments. Earlier today, I went down to the basement and found the lights on for absolutely no reason. I simply asked my AI agent, out loud, in natural language: “What’s going on? Figure out why this happened and make sure it doesn’t happen again.” No technical instructions, no commands, no configuration details. Just a normal conversation. And here’s the agent’s report.
1
1
1
56