@nilenso

Employee-owned programmer cooperative in Bangalore.

nilenso.com
Joined May 2013
  • Tweets1.9K
  • Following474
  • Followers1.7K
248Media
In 2013, a group of makers got together to find new ways to work together. A lot has happened since. We recently celebrated our 10th birthday :) Over the years, we've had the privilege of working with some exceptional organizations and doing work we're proud of. 1/2
2
4
42
4,858
Nerding out with @GeoffreyHuntley, @headinthebox and the BAML folks was so much fun! Legends ❤️ Solid birthday treat!
geoff
@GeoffreyHuntley
Jun 30
conference: “omg loops” this table: “we need a final form of programming language. what are the properties? what does the model desire vs what are we projecting onto the model?”
2
7
352
Bangalore machas represent at AIE WF! @AtharvaRaykar @threepointone
1
12
317
Our poster is up in the Expo at @aiDotEngineer WF, come check it out! github.com/nilenso/context-v…
3
1
5
1,453
Oh look who showed up to our booth!
Srihari Sriraman
@SrihariSriraman
Jun 30
Our poster is up in the Expo at @aiDotEngineer WF, come check it out! github.com/nilenso/context-v…
1
5
290
Swyx wearing the @tracesdotcom shirt around AIE WF was unexpected and awesome! @swyx you’ve been very kind, and quite helpful too! Thank you for having us present, and thank you for AIE!
2
5
14
616
I'm at @aiDotEngineer WF with @AtharvaRaykar, and we'll be presenting our work on context-viewer in the Expo. Come over and lets talk about trace intelligence, error analysis, empirical rigor, semantic data extraction and continuous improvement!
A poster for context viewer: https://github.com/nilenso/context-viewer/

ALT A poster for context viewer: https://github.com/nilenso/context-viewer/

Srihari Sriraman, Partner nilenso | traces Atharva Raykar, Partner nilenso

ALT Srihari Sriraman, Partner nilenso | traces Atharva Raykar, Partner nilenso

5
2
12
1,047
what if you could have an agent search over your team's session history? - what did we get done this week? - should we skillify things we're all doing? - who is using agents in the most interesting ways? now you can! our new agent is available for our most active users
5
17
2
67
13,672
So, if you recognize these patterns in your agents, and they don't fit your task at hand, you could steer them accordingly. Full write-up: blog.nilenso.com/blog/2026/0… Analysis code and data: github.com/nilenso/swe-bench….
1
11
1,249
I wanted to check this on newer models, but no SWE-bench Pro trajectories exist for Opus 4.6 / GPT-5.4. So I pulled @badlogicgames' issue-fixing trajectories and ran the same analysis. Thanks for putting those out in public, they make this kind of analysis possible. Opus's first edit sits at 47% in your pi sessions, vs 35% for Sonnet 4.5 on SWE-bench. Harness and model differ too, so I can't isolate the prompt's effect, but the shape shifts in the direction you'd expect from the explicit analyze-dont-edit prompt. I think we can see the effect of the human-steering through explicit analysis / go-ahead / wrap-up cues in this comparison.
Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. I think it’s likely that the explicit analysis prompt is pushing the understand phase to be longer. The first edit is pushed from 35% to 47%.

ALT Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. I think it’s likely that the explicit analysis prompt is pushing the understand phase to be longer. The first edit is pushed from 35% to 47%.

Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. The edit duration has reduced significantly from 40% to 23%. I suspect this is a combination of the model upgrade and Mario’s steering towards “minimal” and “concise” solutions.

ALT Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. The edit duration has reduced significantly from 40% to 23%. I suspect this is a combination of the model upgrade and Mario’s steering towards “minimal” and “concise” solutions.

4
4
2
29
27,049
super interesting work! glad my open traces on @huggingface allow this! my workflow is mostly: - prompt template with injected gh issue url.and instructions on how to annalyze and present results + concise impl plan to me - i confirm analysis/plan either by knowing or double checking manually. may steer to adjust plan a few times until model knows what to do - tell model to implement - check results, steer if necessary - if all good (type checking, linting, tests, manual code review, manual tests), another prompt template is used to wrap up, i.e. changelog, docs, commit, push, comment on issue, close issue
Srihari Sriraman
@SrihariSriraman
Apr 23
Replying to @SrihariSriraman
I wanted to check this on newer models, but no SWE-bench Pro trajectories exist for Opus 4.6 / GPT-5.4. So I pulled @badlogicgames' issue-fixing trajectories and ran the same analysis. Thanks for putting those out in public, they make this kind of analysis possible. Opus's first edit sits at 47% in your pi sessions, vs 35% for Sonnet 4.5 on SWE-bench. Harness and model differ too, so I can't isolate the prompt's effect, but the shape shifts in the direction you'd expect from the explicit analyze-dont-edit prompt. I think we can see the effect of the human-steering through explicit analysis / go-ahead / wrap-up cues in this comparison.
Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. I think it’s likely that the explicit analysis prompt is pushing the understand phase to be longer. The first edit is pushed from 35% to 47%.

ALT Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. I think it’s likely that the explicit analysis prompt is pushing the understand phase to be longer. The first edit is pushed from 35% to 47%.

Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. The edit duration has reduced significantly from 40% to 23%. I suspect this is a combination of the model upgrade and Mario’s steering towards “minimal” and “concise” solutions.

ALT Comparison of trajectory-phase shapes between SWE-Bench Pro runs and Pi/SWE-agent-style traces. The edit duration has reduced significantly from 40% to 23%. I suspect this is a combination of the model upgrade and Mario’s steering towards “minimal” and “concise” solutions.

3
9
79
21,855
I analyzed 730 SWE-Bench Pro trajectories each for Sonnet 4.5 and GPT-5 and turned them into “trajectory shapes”: when they start editing, when they stop, how much they verify, how many steps they spend understanding vs doing. They have very different work habits.
Two stacked area charts comparing coding-agent trajectory phases over normalized progress from 0% to 100%. Sonnet starts editing earlier, at 35%, and finishes implementation much sooner, by 62%, then spends a long tail verifying after its last source edit. GPT-5 front-loads a lot more reading before it starts editing at 50%, and does very little verification afterward. Sonnet also has to clean up temporary files, while GPT-5 doesn’t.

ALT Two stacked area charts comparing coding-agent trajectory phases over normalized progress from 0% to 100%. Sonnet starts editing earlier, at 35%, and finishes implementation much sooner, by 62%, then spends a long tail verifying after its last source edit. GPT-5 front-loads a lot more reading before it starts editing at 50%, and does very little verification afterward. Sonnet also has to clean up temporary files, while GPT-5 doesn’t.

2
9
2
33
5,176
This was a great opportunity to bridge our industry work at @nilenso with academic research at CMU, and I am very grateful to the co-authors @heathermiller (CMU), Michael Isaac (CMU), and @AtharvaRaykar (@nilenso). We will be in the Bay Area for the conference soon. If you are building AI tooling or want to talk shop about context engineering, we would love to connect.
3
5
273
We’ll be presenting context-viewer at ACM @CAISconf! context-viewer is an observability tool for context engineering. It gives structure to LLM contexts using classification by topics, and allows you to compare runs side-by-side. Useful for things like analyzing agent failures, token spends, and evaluating context compaction. - github.com/nilenso/context-v… - caisconf.org/program/2026/de…
2
12
26
3,681
nilenso retweeted
Drew Breunig
@dbreunig
Apr 1
My takeaways from scanning the Claude Code code for ~45 min this evening: 1️⃣Harness engineering is hard. There's a lot of hard won knowledge in here and plenty of diagnostics to keep the feedback flowing. 2️⃣Harnesses and prompts smooth out model quirks. @SrihariSriraman and I covered this last month, but good to see it verified here. So many conditionals based on model types and specific contexts to deploy to mitigate model weirdness. 3️⃣So much of this is CLI app boilerplate. Fully expect a tool like @badlogicgames's pi to be the foundation for any CLI agent being built today. I talk about the last point, the opportunity for shared foundations, in a post today: dbreunig.com/2026/03/26/winc…

The Cathedral, the Bazaar, and the Winchester Mystery House

The internet made coordination cheap and gave us the Bazaar. Coding agents make code cheap and are giving us sprawling, idiosyncratic software.

dbreunig.com
1
5
73
3,638
Did you know claude code has "model counterweights"? These are patches in the system prompt that exist to balance model biases. These weren't visible earlier, but the leaked code has @[MODEL LAUNCH] annotations that call them out explicitly.
2
1
4
292
Replying to @SrihariSriraman @swyx @dbreunig
I did a compaction analysis on a couple of recent claude code sessions, and thought I'd share here too. You can use the link to explore further if you're interested. These are good compaction examples. I wish I could do the same analysis with some bad examples. nilenso.github.io/context-vi…
1
1
132
nilenso retweeted
Drew Breunig
@dbreunig
Feb 18
Somehow I didn't fully appreciate how strongly Claude Code's prompt has to fight against the weights to make parallel tool calls. blog.nilenso.com/blog/2026/0…
9
17
3
254
19,136
Something I've been thinking for a while, but finally got to writing it down. The core thesis is that building reliable AI applications requires a harness to be able to tinker, experiment and iterate, without which the project gets stuck in the prototyping phase. blog.nilenso.com/blog/2026/0…

Engineering Maturity is all you need

8:PM in the evening: it’s demo day tomorrow.

blog.nilenso.com
5
1
6
481
nilenso retweeted
Drew Breunig
@dbreunig
Feb 13
Really excited for this one: @SrihariSriraman and I took a deep dive into coding agent system prompts to understand their structure, similarities, and differences. dbreunig.com/2026/02/10/syst…

How System Prompts Define Agent Behavior

The model sets the ceiling on an agent’s performance, but the system prompt decides whether it ever gets there.

dbreunig.com
9
48
7
315
23,332
Replying to @badlogicgames
I've been studying the effect of system prompts in the model + tools + system-prompts + harness stack. So, I ran the same SWE-Bench-Pro task with Opus+Claude Code, but with different system prompts. One run used Codex's system prompt, and another run used Claude's system prompt. The workflows on the runs are different, and mirror these kinds of sentiments. You can see the corresponding differences in the system prompts too. We maybe mis-attributing some of these behaviours to the model, when they're attributable to the system-prompt.
1
1
1
3
388

No more items