adi (@jerkeyray) header
adi (@jerkeyray) profile picture

consummate tinkerer | 19

below rock bottom
Joined January 2025
Pinned Tweet
i've been reading quite a lot of research papers over the past couple of months (even got to help out on one), and i liked the idea of picking one problem and going deeper than necessary instead of building something i didn't need. so i tried that with prefix caching in llms. i built a profiler to see what happens between a coding agent and the inference engine, byte by byte, got some numbers, then spent way more time figuring out what they meant. this is me summing all that up in a neat lil writeup (kinda long tbh).
13
10
2
86
2,071
actually back this time fr
9
48
1,021
Hey everyone, I am currently looking for my next role in backend, systems, or ai infrastructure engineering. preferably remote-first. i'm based in india (ist) and comfortable working across us/eu time zones. i mostly work with go, along with python and typescript. i enjoy building things from scratch, understanding how they work under the hood, and occasionally going a little too deep into the details. some things i've built: mini-rpc — an rpc framework in go over raw tcp, with custom framing, serialization, and code generation. mini-protoc — a lightweight protocol buffers compiler in go for mini-rpc, with a hand-written lexer, parser, ast, schema validator, and go code generation for message structs and service/client stubs. pony — a go runtime for orchestrating multiple ai coding agents, with process lifecycle management and pty support. krafta — a kafka-inspired log system in java, with partitions, offsets, consumer groups, and multi-broker coordination. newsletter backend — built using go, postgresql, rabbitmq, and redis, with a focus on async processing, idempotency, and rate limiting. recently, i've been spending more time exploring the infrastructure side of ai —agent runtimes, llm gateways, mcp, and the engineering challenges involved in making these systems reliable. i'm open to junior / entry level opportunities in backend, go, systems, or ai infrastructure. remote roles would be ideal, but i'm also open to opportunities in bengaluru/hyderabad. if you're building something interesting, know of an opening, or think there might be a good fit somewhere, feel free to reach out. referrals and introductions are always appreciated. website: akansharoy.site
github: github.com/akansha204 my dms are open!
55
46
5
354
14,908
Version 3 of my ai solves 70 of 400 ARC-AGI-1 tasks. That ties Gemini 2.5 Pro (Thinking 1K) at 17.5%. It’s 93 KB of C. No neural networks, no pretrained weights, no GPU. Median 0.22 s and 32 MB of RAM per task. Here it is solving live.
24
9
3
162
10,240
1/ I built Latch, a small redis-distributed lock in typeScript, then wrote a test that proves it can fail
2
2
4
141
I'm currently looking for opportunities to work on and build interesting things. Here's my portfolio: lowkeydev.me/ It has everything about my work, projects, and experience. If you think I could be a good fit for your team, feel free to DM me. Would love to chat!
10
17
2
55
3,652
editing 1000 images at the same time in a single canvas @OffToning
34
22
4
121
6,985
Too much of a good thing is bad, and to some extent that's what's happening with this new cult of optimization. There's a fine line between optimizing a process to make your life better and turning life itself into an optimization project. I think the trend started with nutrition, then moved to sleep, training, and eventually every possible aspect of one's life, culminating in a new archetype: the optimizer. An individual who lives for the mere sake of optimizing aspects of their life. Sleep score, zero sugar, saving as much as possible. These things are great, but taken to the extreme they create a kind of soulless life, skipping time with friends or wonderful places because it would disrupt your sleep or break the routine. In medio stat virtus. I think that in most cases, people simply err either in choosing which parameters of their life to optimize, or in how they run the optimization itself.
1
1
22
594
and now i am officially open to work looking for backend engineering roles, specifically in Golang i've spent the past year building projects and getting deeper into backend and systems development projects below stevenstank.vercel.app github.com/stevenstank
12
5
2
46
3,494
hiatus over. hi, how have y'all been.
19
66
1,714
Made a video explaining Betterstack's system Do watch and give review!
6
9
111
3,794
Tried something new! Made a video about system design of reddit totally freestyle do watch and give reviews!
8
6
37
1,114
read a paper on relational llm serving. the main idea is that current llm serving engines are treating every inference call as an independent request, even when those calls are part of the semantic query plan. suppose your query is something like "for every contract, check whether it contains a confidentiality clause, extract the clause and summarize it". at the db level this should be one query plan so going through each contract and then filtering them and then extracting the clause and summarizing it. the above three operations may use the same long text but a normal llm serving engine sees it as individual requests. it does not know that they belong to the same db query, operate on the same contract, run in a particular order and share most of their prompt. the kv cache state should be reusable, but by the time the second operator runs the prefix has already been evicted, so it gets prefilled from scratch every time. what they did was give the serving layer the whole query plan so now we can say "contract x just passed the filter for a clause, run its extraction and summarize it while the kv cache is still hot in GPU memory." gonna be stealing chunks from this for the next project i am working on.
2
40
1,376
Made a video explaining the project which i am working on Do watch and give reviews!
3
6
31
759
adi retweeted
Day 11/30 of Inference Engineering Part 1: Why LLM Inference Frameworks Use torch.compile Topics covered in this part: - Why LLM inference is a perfect candidate for compilation - Training vs Inference - Autoregressive generation and repeated decoder execution - Compilation cost amortization - PyTorch Eager Execution - How eager execution works internally (Python → Dispatcher → CUDA → Python) - Why eager execution limits compiler optimizations - What torch.compile actually changes - High-level torch.compile pipeline (TorchDynamo → FX Graph → TorchInductor → Triton/C++) - Why inference frameworks like vLLM and SGLang use torch.compile youtube link shared below!
Day 10/30 of Inference Engineering spent some time reading through TorchDynamo today. PyTorch models are ordinary Python programs. Control flow, function calls, loops, and conditionals are all executed by the Python interpreter. While this makes PyTorch highly expressive, but it also means there isn't a graph sitting there for a compiler to optimize. Compilers don't optimize Python source code. They optimize an intermediate representation (IR) of the computation. Before passes like operator fusion, memory planning, scheduling, or code generation can even begin, PyTorch first needs a representation of the tensor program that was actually executed. TorchDynamo is responsible for constructing that representation. Instead of asking you to rewrite your model or trace it ahead of time, TorchDynamo intercepts Python frame execution, watches the PyTorch operations being executed, and builds an FX Graph that represents the tensor computation while leaving the original model unchanged. Once graph capture is complete, TorchDynamo steps out of the pipeline. The FX Graph is then handed off to downstream compiler stages such as TorchInductor, where graph-level transformations are applied before lowering the computation into optimized Triton or C++ kernels for execution. ill try by best to cover the few topics i talked about in last few days and post a video tmr, i was outside yesterday so couldnt record one for today
2
8
1
99
8,967
larp is my new fav word
7
26
1,047
damn that's crazy
6
12
787
tigerbeetle is doing very very interesting work
4
21
655