AI Architect & Builder - Inference, LLMs, SLMs GSoC Org Admin - Jenkins | Docker Captain | CNCF Ambassador
New Delhi
Joined December 2018
- Tweets7.3K
- Following1.3K
- Followers8.4K
- Likes38.1K
Shivay Lamba retweeted
Build a tiny vLLM by yourself
no better way to learn than this
github.com/skyzh/tiny-llm
Shivay Lamba retweeted
Agentic kernel optimizations with Josh Lochner (@xenovacom)
#WebAI Summit
ALT Billions of devices on the web
Shivay Lamba retweeted
Introducing agent-desktop
agent-browser, for your whole computer
→ Computer Use for any coding agent
→ macOS, Windows and Linux
→ Runs in bg while you keep working
→ Agents use your computer in parallel, each with its own cursor
Shivay Lamba retweeted
Everyone who comes to the Cloud Native Hackathon in Salt Lake City will get a Keychron keyboard.
The winner will take home an NVIDIA DGX Spark supercomputer, and 4 more attendees will get $16,000 in cash.
Kelsey Hightower will be there as well!
I've been heading to KubeCon since 2019 and wanted to do this hackathon for a really long time. Over 100 people have registered since yesterday alone.
👉 Spots are limited. Join us: luma.com/cnh
Headed to KubeCon?
Win an NVIDIA DGX Spark AI & $20,000 in cash prizes! Kelsey Hightower is Judging!
Everyone who comes gets a FREE Keychron mechanical keyboard!
11/9 @ The Shop Workspace
Register Now: luma.com/cnh
Find More Events: KubeConParties.com
Shivay Lamba retweeted
I give you a Local AI stack for office:
• MiniCPM-OCR – document parsing
• Liquid LFM2.5 – lightweight agents
• Qwen3 8B – office assistant
• MiniCPM-V – vision
• Qwen3 Embeddings – RAG
• Whisper – transcription
All local.
Shivay Lamba retweeted
Inspired by Daytona giving away an Nvidia DGX Spark supercomputer and $20,000 in cash if you're coming to Salt Lake City in November.
Join us: luma.com/cnh
Shivay Lamba retweeted
I'm building the coolest agentic mobile app on @Snapdragon that analyzes and sorts all of your videos and photos on device! This all runs locally on the 8 Elite Gen 5 NPU using @Qualcomm_Dev GenieX and the new @liquidai LFM3 models! Video editing just got a whole lot easier!
Shivay Lamba retweeted
We’re renaming our Max plans:
Max 5x → Max
Max 20x → Ultra
Max is still $100/month, and Ultra is still $200. If you’re a subscriber, there’s nothing you need to do.
We’re also removing the 5-hour limit for verified Ultra accounts, starting today with most existing subscribers. Newer accounts keep a temporary limit until they’re verified, which helps us prevent fraud and abuse. You'll get an email when it's removed.
Weekly limits remain unchanged. View our plans here: claude.com/pricing
Shivay Lamba retweeted
Starting today, eligible ChatGPT Pro and Business subscribers can claim up to $500 per month in API credits to build apps and agents.
Shivay Lamba retweeted
Today, we're shipping real-time graphics.
Using Sol 6.1 Ultrafast we can generate custom overlays, diagrams, and even interactive 3D scenes right on a user's screen, no chat.
It's a new visual interface for AI.
Here's me learning about LLMs + hardware.
Demo:
Shivay Lamba retweeted
Day 25/
We are doubling all usage limits for all Pro 100/200/500 tiers and Business Premium, effective today.
We heard your kind feedback and won’t be reducing usage limits for Pro 200, so your Work and Codex usage will stay the same. We’re also going to continue offering extended limits for dots for now, and usage of your dot will still continue to be separate from your default usage.
And we'll of course also keep shipping better models and other efficiency improvements that increase your effective usage over time.
Shivay Lamba retweeted
Learn Harness Engineering:
- Load the right code, docs and history
- Let the model propose the next action
- Check permissions before running tools
- Execute within time and cost limits
- Verify the result with tests and policy checks
- Feed failures back into a bounded retry loop
- Keep logs and traces so failures can be debugged
The harness defines what the agent can do, how its work gets checked and when it must stop.
Build these systems with AI Engineering from Scratch 👇
aiengineeringfromscratch.com
Shivay Lamba retweeted
Cached tokens, clearly explained!
(3,072 of 3,108 prompt tokens came from the cache. only 36 were new.)
your agent pays for the same prompt on almost every turn.
every call resends the system prompt, the tool definitions and the full history.
the model read all of it one turn ago.
here's what caching does with that:
→ turn 1: the model processes your prompt and stores the work in its KV cache
→ turn 2: the start of the prompt matches, so those tokens are reused, not recomputed
→ only the new message gets processed at full price
on Nebius Token Factory, DeepSeek V4.1 Flash 1017 bills cached input at $0.006 per 1M tokens vs $0.30 uncached.
that's 50x cheaper for the part of the prompt you were resending anyway.
the catch: it's best-effort. a repeated request can still come back with cached_tokens: 0, so log it every turn and budget at list price.
no cache keys, no opt-in. you just structure your prompt so the start stays the same.
(bookmark this)
Shivay Lamba retweeted
New in Claude for Legal: a gallery of ready-made artifacts for contract review, regulatory horizon scans, cite checks, and more.
Start from the built-in example, add your own documents and connectors, and share it with anyone across the business.
Shivay Lamba retweeted
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
Shivay Lamba retweeted
open models with 1M context are kind of wild for exploring codebases.
gave it the whole hono repo, 467 files and 660K tokens, in one prompt to DeepSeek V4.1 Flash 1017
few seconds later it had a map of the framework and a tour of how a request finds its route, every step pointing at real lines.
then i asked a question, a follow-up, and had it write docs for all 13 modules.
✅ basically gitdiagram + deepwiki + codetour + codeviz in one open source tool.
whole session cost $0.63 on @nebiustf
bring your own key. runs in one docker container, works on your local folders and PRs too
Shivay Lamba retweeted
Gemini 4 Carbon is our next frontier model.
It’s state-of-the-art across cybersecurity and long-horizon capabilities.
Shivay Lamba retweeted
Introducing Gemini 4 Carbon, it's a big jump over Argon which we released just 2 weeks ago.
Gemini 4 Carbon is SOTA on cyber, long horizon tasks, ranks #1 on Artificial Analysis V5, significantly improves on usability in domains like coding, and is more aligned.
Shivay Lamba retweeted
What actually happens when you call an LLM API? 🤔
I've been working on this visual explainer for a while, and I'm excited to finally share it! 🎬
Follow a single request from client → gateway → tokenizer → router → queue → GPU (prefill + decode) → streamed response.
Along the way, explore optimizations like:
KV caching / Prompt caching
Cache-aware routing
Continuous batching
Disaggregated prefill/decode
Speculative decoding
I recently gave a talk on this at Hacktoberfest 2026, but here's the entire journey in just 5 minutes!
Watch it and let me know what you think! 👇
Full talk and repo in the reply.