AI Architect & Builder - Inference, LLMs, SLMs GSoC Org Admin - Jenkins | Docker Captain | CNCF Ambassador
New Delhi
Joined December 2018
- Tweets7.3K
- Following1.3K
- Followers8.4K
- Likes38.1K
Pinned Tweet
What an amazing day! Presented a poster at @PyTorch Conference: how we optimised the RLM paper with pre-fix caching & batched Sub calls with @vllm_project
And a full house for our Pytorch Conference talk on running LLMs using Executorch on Android!
Shivay Lamba retweeted
Applications for our Codex Ambassador program closes tomorrow! Come be part of this crew :)
Meet our Codex Ambassadors π
We sat down with a few of our ambassadors from around the world to hear why they joined, what theyβre building, and what makes their local communities special. Proud to support them!
Join our Codex Ambassador program: developers.openai.com/communβ¦
Shivay Lamba retweeted
Build a tiny vLLM by yourself
no better way to learn than this
github.com/skyzh/tiny-llm
Shivay Lamba retweeted
Agentic kernel optimizations with Josh Lochner (@xenovacom)
#WebAI Summit
ALT Billions of devices on the web
Shivay Lamba retweeted
Introducing agent-desktop
agent-browser, for your whole computer
β Computer Use for any coding agent
β macOS, Windows and Linux
β Runs in bg while you keep working
β Agents use your computer in parallel, each with its own cursor
Shivay Lamba retweeted
Everyone who comes to the Cloud Native Hackathon in Salt Lake City will get a Keychron keyboard.
The winner will take home an NVIDIA DGX Spark supercomputer, and 4 more attendees will get $16,000 in cash.
Kelsey Hightower will be there as well!
I've been heading to KubeCon since 2019 and wanted to do this hackathon for a really long time. Over 100 people have registered since yesterday alone.
π Spots are limited. Join us: luma.com/cnh
Headed to KubeCon?
Win an NVIDIA DGX Spark AI & $20,000 in cash prizes! Kelsey Hightower is Judging!
Everyone who comes gets a FREE Keychron mechanical keyboard!
11/9 @ The Shop Workspace
Register Now: luma.com/cnh
Find More Events: KubeConParties.com
And now @NVIDIAAI Brev also got a security breach. At this point in time, every major tech company is going through some sort of security issues - CVEs, stolen keys/API, credentials of users π₯²
Shivay Lamba retweeted
I give you a Local AI stack for office:
β’ MiniCPM-OCR β document parsing
β’ Liquid LFM2.5 β lightweight agents
β’ Qwen3 8B β office assistant
β’ MiniCPM-V β vision
β’ Qwen3 Embeddings β RAG
β’ Whisper β transcription
All local.
Super excited to visit my first @NeurIPSConf in Sydney in December this year!
Who am I going to meet there?
Shivay Lamba retweeted
Inspired by Daytona giving away an Nvidia DGX Spark supercomputer and $20,000 in cash if you're coming to Salt Lake City in November.
Join us: luma.com/cnh
Shivay Lamba retweeted
I'm building the coolest agentic mobile app on @Snapdragon that analyzes and sorts all of your videos and photos on device! This all runs locally on the 8 Elite Gen 5 NPU using @Qualcomm_Dev GenieX and the new @liquidai LFM3 models! Video editing just got a whole lot easier!
Shivay Lamba retweeted
Weβre renaming our Max plans:
Max 5x β Max
Max 20x β Ultra
Max is still $100/month, and Ultra is still $200. If youβre a subscriber, thereβs nothing you need to do.
Weβre also removing the 5-hour limit for verified Ultra accounts, starting today with most existing subscribers. Newer accounts keep a temporary limit until theyβre verified, which helps us prevent fraud and abuse. You'll get an email when it's removed.
Weekly limits remain unchanged. View our plans here: claude.com/pricing
Shivay Lamba retweeted
Starting today, eligible ChatGPT Pro and Business subscribers can claim up to $500 per month in API credits to build apps and agents.
Shivay Lamba retweeted
Today, we're shipping real-time graphics.
Using Sol 6.1 Ultrafast we can generate custom overlays, diagrams, and even interactive 3D scenes right on a user's screen, no chat.
It's a new visual interface for AI.
Here's me learning about LLMs + hardware.
Demo:
Shivay Lamba retweeted
Day 25/
We are doubling all usage limits for all Pro 100/200/500 tiers and Business Premium, effective today.
We heard your kind feedback and wonβt be reducing usage limits for Pro 200, so your Work and Codex usage will stay the same. Weβre also going to continue offering extended limits for dots for now, and usage of your dot will still continue to be separate from your default usage.
And we'll of course also keep shipping better models and other efficiency improvements that increase your effective usage over time.
Shivay Lamba retweeted
Learn Harness Engineering:
- Load the right code, docs and history
- Let the model propose the next action
- Check permissions before running tools
- Execute within time and cost limits
- Verify the result with tests and policy checks
- Feed failures back into a bounded retry loop
- Keep logs and traces so failures can be debugged
The harness defines what the agent can do, how its work gets checked and when it must stop.
Build these systems with AI Engineering from Scratch π
aiengineeringfromscratch.com
Shivay Lamba retweeted
Cached tokens, clearly explained!
(3,072 of 3,108 prompt tokens came from the cache. only 36 were new.)
your agent pays for the same prompt on almost every turn.
every call resends the system prompt, the tool definitions and the full history.
the model read all of it one turn ago.
here's what caching does with that:
β turn 1: the model processes your prompt and stores the work in its KV cache
β turn 2: the start of the prompt matches, so those tokens are reused, not recomputed
β only the new message gets processed at full price
on Nebius Token Factory, DeepSeek V4.1 Flash 1017 bills cached input at $0.006 per 1M tokens vs $0.30 uncached.
that's 50x cheaper for the part of the prompt you were resending anyway.
the catch: it's best-effort. a repeated request can still come back with cached_tokens: 0, so log it every turn and budget at list price.
no cache keys, no opt-in. you just structure your prompt so the start stays the same.
(bookmark this)
Shivay Lamba retweeted
New in Claude for Legal: a gallery of ready-made artifacts for contract review, regulatory horizon scans, cite checks, and more.
Start from the built-in example, add your own documents and connectors, and share it with anyone across the business.
Shivay Lamba retweeted
Introducing π»π΅πΌπΉ: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> π»π΅πΌπΉ-π·.π½π±: text, 256k context window.
> π»π΅πΌπΉ-π
π»-πΈπ±: vision-language model with video-input support for spatial understanding
> π»π΅πΌπΉ-π°ππππ-πΈπ±: tool calls, voice chat, reasoning in 18 languages.
> π»π΅πΌπΉ-πΎπππ-πΈπ±: an omni model combines vision, audio, and language.
> ππ·-πΈπ± and ππ·-πΎπππ-πΈπ±: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> π»π΅πΌπΉ-π
π»-πΈπ± is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> π»π΅πΌπΉ-π·.π½π± is #1 on the Pareto frontier of device intelligence πππππππ.ππππππ.ππ
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ π§΅
Shivay Lamba retweeted
open models with 1M context are kind of wild for exploring codebases.
gave it the whole hono repo, 467 files and 660K tokens, in one prompt to DeepSeek V4.1 Flash 1017
few seconds later it had a map of the framework and a tour of how a request finds its route, every step pointing at real lines.
then i asked a question, a follow-up, and had it write docs for all 13 modules.
β
basically gitdiagram + deepwiki + codetour + codeviz in one open source tool.
whole session cost $0.63 on @nebiustf
bring your own key. runs in one docker container, works on your local folders and PRs too