螺壳实验室 Luoke Labs

聊天、写文档、搜东西，还有那些打了又删、最后没发出去的话，输入法是唯一全都见过的那个。 这些字，其实是一个人最重要的积累。就像写作文，写得越多，越知道自己想说什么、习惯怎么说。 可现在，这些字一条条发出去就散了。大部分输入法开着云联想的时候，还要先送到服务器上算一遍候选。字是你打的，积累却没留在你这里。 螺壳一直在探索，本地模型怎样能更好地帮我们工作和生活。输入法是我们找到的答案之一：你写下的字离你的想法最近，也最不该离开你的电脑。 所以我们做了螺壳输入法，只属于你的 AI 输入法：你写下的字，留在你自己的 Mac 上。 它稍微有点不太一样的地方： 1. 语义选词。打「大家看了方案，没有异议就这么定」，按下空格，出来的是「没有意义就这么定」。螺壳里一个 0.8B 的本机模型会读你前面写的内容，把「异议」排到第一位。断网也照样能用。 2. 小地方替你写对。GitHub 这类名字的大小写直接替你写对；打开设置后，中英文之间自动空一格。 3. 记什么，你说了算。密码框里的字一个都不记；它可以学习你的表述方式，你随时能看、能删。 4. 双拼全拼都行，还能自定义。小鹤双拼、全拼随时换；引擎是个独立的小程序，用什么语言都能写。 同一套 600 句，苹果自带输入法首选准确率 78.8%，螺壳输入法 89.8%。 整个过程不调用云端。语音输入也很快会加上。 螺壳的本地工具俱乐部现在还很早期，特别需要大家的反馈。 想参加内测的，留言告诉我们你的 Mac 型号和内存，我们拉你进螺壳社区交流群，有问题直接在群里提，一起把本地 AI 做得更好用。

I used Divyansh's approach to build an entire Nintendo into ~1GB of Qwen3.5 weights. Qwenputer, meet Nesformer: github.com/dbreunig/nesforme…

Sound on 🔊 Listen to the difference. Our new open multilingual NemotronLabs RE-USE model from NVIDIA researchers cleans up background noise, echo and other audio distortions to make speech clearer. All with just 9.6M parameters. Try the demo and download the model on @huggingface → huggingface.co/nvidia/NVIDIA…

We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks. Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more. Run on 64GB+ RAM. GGUF: huggingface.co/unsloth/DeepS… Blog: unsloth.ai/docs/models/deeps…

sam: local inference is a terrible allocation of scarce resources “a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment somehow many people's intuition is the opposite of that”

有人开源了个 ai-phone，让 Mac 自己接打电话，全程跑本地模型，通话内容不出机器。 用到的三个模型都不大： 👂 听用 Parakeet 0.6B， 🧠 想用 Qwen3.5 4B（32GB 内存能换成 9B）， 🗣️ 说用 Kokoro 82M。 电话线路走 SIP，绑个一美元左右月租的号就行。 作者实测，开口后大约 1～1.5 秒回话，对方插话它就停下来听。目前只能单线，英文为主。 挺好，AI 打电话，AI 接电话，就让他俩聊去吧～ nitter.cf/realfxw/status/2115702…

终于有人把“本地大模型打真实电话”做出来了，而且完全开源！ 以往体验市面上的各类 AI 电话助手，最让人顾虑的就是隐私安全：通话录音、转写文本、沟通对象全都会留在第三方云端。开发者刚刚开源的 ai-phone 则彻底打通了本地闭环——让你的 Mac 直接接管真实电话线路，所有推理均在本地完成，通话数据完全不出机器。 🛠️ 核心技术栈（基于 Apple Silicon & MLX）： 语音识别（STT）： Parakeet TDT 0.6B 本地运行，极速语音转文字 语言模型（LLM）： 默认本地运行 Qwen3.5 4B（32GB 内存支持 9B 4-bit），同时也支持接入自己的 Claude 或 ChatGPT 端点 语音合成（TTS）： Kokoro 82M / Phonon，生成自然流畅的声音 通信接入： 通过 SIP 协议直连 Telnyx 或 Twilio，直接购买或绑定真实电话号码（月租约 1 美元） ✨ 核心亮点： 低延迟与实时打断： 在 Apple Silicon 芯片上实测首字响应延迟约 1~1.5 秒；支持实时插话打断（Interruption），一旦你开口，AI 会立刻停下倾听。 支持外部 Agent 联动： 提供了 call、listen、say、hangup 等标准化 CLI 接口，可以直接让 Claude Code、Codex 等编码智能体通过命令编排通话，自动完成预约餐厅、电话咨询等任务。 本地守护进程： 提供向导式 CLI 配置，一行命令拉起本地守护进程，静默接听来电或随时发起外呼。 ⚠️ 当前要求与限制： 仅支持搭载 Apple Silicon 的 macOS 系统 目前为单路通话，优先支持英文环境 项目刚刚上线，对本地 AI 语音助手与 SIP 通信落地感兴趣的朋友值得关注：github.com/oxnw/ai-phone

昨天介绍了螺壳输入法，打的字都留在自己电脑上。今天想聊聊邮件。 想让 AI 帮你理邮件，又不想把信交出去？我们自己就是这样。邮箱里有工作往来、账单、验证码，还有亲友发来的消息。想让 AI 帮忙看看，就得先把这些信传到别人的服务器上，心里总有点别扭。 所以我们在螺壳底座上做了一个邮件助手。信只在你自己的设备上处理，用的是本机模型，内容不传给云端。螺壳的应用都共用同一套引擎和模型，装过输入法的，不用再下载一遍。 你不用电脑的时候，比如晚上睡觉，它在后台一封封慢慢看。第二天打开邮箱，要回的信排在最前面，每封旁边一句话说清是什么事、为什么要回，有截止日期的也标出来了。 冒充银行要验证码的信，它会先标成可疑。我们拿 45 封测试信试过，13 封钓鱼信认出 12 封，正常的 32 封没有误报。 回信它会先写好放进草稿箱，发不发由你。删信、退订也都等你点一下，当天处理过的都能撤回。 Gmail、QQ、163/126 和其他 IMAP 邮箱都能接，16GB 内存的 Mac 也能跑。 想一起测试当小白鼠的，留言告诉我们 Mac 型号和内存，一起看看本地模型能够折腾点啥。

Qwen4, coming soon… CTO quote was “next week” Xudong Guo - Research Scientist, Qwen, Token Foundry Live from Amsterdam

DeepSeek did it again! They just open-sourced DeepInfer, the inference stack behind cache hits that return the first token up to 26x faster. When a request repeats a long prefix, like a system prompt or earlier chat turns, the server can reuse the KV cache it already computed instead of running prefill again. Ideally, you'd expect that cache to sit in GPU memory, since disk is supposed to be far too slow for anything on the request path. In practice, DeepSeek keeps it on SSDs. When DeepSeek launched this, a 128K-token prompt that hit the cache got its first token in 500ms instead of 13s. And in one day of production traffic last year, 56% of all input tokens were served straight from disk. So disk isn't the bottleneck here as long as what you store is small enough, and the storage is wide enough. Let's understand how this works. 1) Why reuse the KV cache at all: Before generating anything, an LLM reads the whole prompt and computes a key and a value for every token in every layer. This is prefill, and on long prompts it's the slow part. Many requests start the same way. For instance, a chat resends every earlier turn or an agent resends the same system prompt and tool definitions on every call. If those KV are saved, the next request with the same prefix loads them and only computes the new tokens. 2) Why GPU memory is predominantly used: Every token stores KV in every layer, so on a standard model a long prompt's KV cache runs into many GBs. Reading that much from disk can take longer than recomputing it on the GPU. So engines keep it in GPU memory. But GPU memory is small, so old prefixes get evicted fast and many repeats miss. 3) DeepSeek made the KV cache small: DeepSeek-V2 introduced Multi-head Latent Attention (MLA). Instead of storing separate KV for every attention head, MLA stores one small compressed vector per token per layer, and the keys and values are rebuilt from it during attention. In the V2 paper, this cut the KV cache by 93.3% compared to DeepSeek's earlier 67B model. And MLA shrank the KV cache enough to store it on low-cost disks. Fewer bytes per cached token implied fewer bytes to read on every hit, and it made loading from SSD faster than recomputing. 4) 3FS reads it from many SSDs at once: One SSD would still be too slow, so the cache lives on 3FS, DeepSeek's own distributed file system. 3FS splits files into chunks and spreads them across thousands of SSDs on many storage nodes. A read pulls chunks from many drives in parallel, so their bandwidth adds up. When a request arrives, the system matches its prefix against cached blocks, fetches those blocks from 3FS into host memory, and copies them to the GPU layer by layer while the model is already running. And since every prefill machine can reach every cached prefix, a request doesn't have to go back to the machine that served it last. Disk is also cheap, so the cache can stay for days. As per the DeepInfer docs, cached prefixes are retained for 3 days for agent traffic and 7 days for chat. Disk caching works when the KV cache is small, and the storage is wide. DeepSeek got the first from its attention design and the second from 3FS. This is also evident from the price. On deepseek-flash today, a cache hit costs 50x less than a miss. If you use the API, keep the stable parts of your prompt (system prompt, tools, documents) at the front, since only an exact match from the very first token counts as a hit. The visual below depicts what we discussed. That said, prefix caching is only one of the caches in LLM serving. I covered all four in an article recently, with code for each. Read it below.

Local Qwen generates 234 tok/s in JSON format. In plain text, only 70 The same local AI model: 70 tok/s in prose and 234 tok/s when generating JSON. More than 3 times faster. Without changing hardware. In Qwen3.6-35B-A3B-Splash tests on Mac mini M6: - EN Prose: 69.6 tok/s - Python Code: 135.9 tok/s - JSON Extraction: 234.1 tok/s Why? Speculative decoding works better when subsequent tokens are easier to predict. Repetitive JSON and code structures can provide much greater speedup than ordinary conversation. nitter.cf/swarogan/status/210392…

Apple M5 Ultra 512GB is live on Apple store! 27K EURO!!! with 4TB SSD in Italy 👀 Availability 23rd November!

Your local LLM can now take control of a browser and complete tasks autonomously via Unsloth! Qwen3.8-27B successfully controlled the browser to complete tasks like chess, painting, with no human intervention. GitHub: github.com/unslothai/unsloth Guide: unsloth.ai/docs/basics/brows…

Big win for local LLM inference! This open-source engine generates tokens 2.7x faster than Ollama on a MacBook: I tested it on a base M5 MacBook Pro with Qwen3.5 4B at 4-bit and the same 9.2k-token prompt. Ollama ran its MTP build, so both had speculative decoding. - Prefill: Ollama 464 tok/s, Magnitude 755 tok/s (1.6x) - Decode: Ollama 22.6 tok/s, Magnitude 60.0 tok/s (2.7x) - End to end, Magnitude finished in half the time The video below shows both runs side by side. The Ollama numbers don’t mean it’s badly built. Its kernels were simply tuned in advance. A kernel is the small GPU program behind every matrix multiply and attention step. Its speed depends on choices like how work is split across GPU cores and how data moves through memory. The best choices change with every chip. Apple Silicon alone spans 8 to 80 GPU cores and under 70 to over 800 GB/s of bandwidth, and every model has its own layer sizes. A prebuilt engine has to pick one set of settings for all of them, so most local setups run on a compromise. Magnitude replaces that compromise with tuning on your machine: ↳ When you download a model, it compiles that model’s kernels locally. ↳ It tries different kernel configurations and keeps whichever runs fastest on your chip. ↳ It pairs each model with the fastest speculative decoding drafter it has found, so each pass of the model produces more than one token. GitHub repo: github.com/magnitudedev/magn… (don’t forget to star it 🌟) This matters most for agents. Every file and tool output has to be processed before the model writes anything, and that prefill step is where kernel efficiency shows up. Concurrent sessions also share prefix caches, so parallel agents don’t slow each other down. Magnitude is an open-source (Apache 2.0) inference engine written in Rust. It plugs into Claude Code, Codex, OpenCode, Pi and other agents with one click. I shared the GitHub repo above.

𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱 are decision models that turn documents, images and audio into structured answers your applications can act on. 𝚍𝟷-𝟸𝙱 accepts text and images. 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱 adds audio. Both answer multiple questions about the same input in a single forward pass, with calibrated probabilities. New typed outputs extend their capabilities beyond yes/no answers, choices and scores: > Locate objects with bounding boxes, or find and count every match. > Extract text, numbers, and dates with their source spans, returning “not found” when a field is absent. > Estimate numerical values with a likely range. > Select every applicable option, each with its own probability. Route support requests, extract invoice fields or inspect camera frames locally, without a round trip to the cloud. On our index covering 117 public and private benchmarks, d1-2B scores 63.2 versus 58.8 for d1-3B, with a third fewer parameters. > 𝚍𝟷-𝟸𝙱 model card: huggingface.co/LiquidAI/d1-2… > 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱 model card: huggingface.co/LiquidAI/d1-O… 8/