
Software Engineer @Google Author @freeCodeCamp 🇺🇸.then(🇪🇨).then(🇨🇱).then(🇧🇷).then(🇩🇪).then(🇬🇧).finally(🌎) Thoughts are my own
Joined February 2012
- Tweets427
- Following1K
- Followers132
- Likes1.4K
David Good retweeted
I'm about to try something new. An experiment.
Can I create a new medical device from scratch? Hardware and all?
Can I do it solo on nights and weekends? Can I get it FDA cleared? How much can agents help? How long will it take? What will it cost?
Follow me on this journey.
David Good retweeted
When a new model comes out, are your instructions and skills still helping?
I wrote a walkthrough showing how to use DeltaSelect and Agent Layer to compare your setup to a bare model, based on the case study in my arXiv paper (arxiv.org/abs/2609.19607) View article
David Good retweeted
My first @arxiv publication is live! DeltaSelect: Affordable A/B Testing for Coding Agents (arxiv.org/abs/2609.19607)
Use it with Agent Layer to compare harnesses, skills, and more. See if your setup is better than a bare model. More coming soon.
agent-layer.dev/deltaselect
David Good retweeted
Simplify your AGENTS.md and skills. A two-line edit cut costs by 36.5% (p=.034), with a slightly better score on a four-task subset of DeepSWE by @datacurve. We have been told by @karpathy, @rubenhassid, @AnthropicAI, @OpenAI, and others to simplify. Listen to them.
I've been benchmarking my skills to try and do this in a principled way.
During benchmarking, I found that for hard tasks, the agent would review the plan and the code multiple times. It tended to continue beyond the point of diminishing returns, thinking that its findings were substantive when they were not.
So I made two minor changes to my skills that I have already significantly downsized.
Before:
> Update the plan to address any findings you agree with.
After:
> Update the plan to address any findings you agree with. Do not repeat plan review.
Before:
> Resolve any gaps, repeat any invalidated review, and run required checks before reporting completion.
After:
> Resolve any gaps and run required checks before reporting completion.
I tested both versions with GPT-5.6 Luna at low reasoning on eight hard DeepSWE tasks, with two runs per task.
Across all eight tasks, mean cost fell 19.0%, from $8.78 to $7.12 per repetition. The calibrated score was 2.48 points higher and passed a five-point noninferiority test (p-value=.032).
The 36.5% result came from the four tasks where the old instructions had caused the agent to continue or repeat reviews:
- Effect SSE HTTP API streaming
- Oxvg structural selector preservation
- Eicrud keyset-pagination cursor
- Yaegi Go embed directives
On those four tasks, mean cost fell from $5.83 to $3.70, or 36.5% (one-sided paired p-value=.034). The calibrated score was 6.48 points higher and passed the five-point noninferiority test (p-value=.037).
I also ran an ablation that removed only “repeat any invalidated review” without adding “Do not repeat plan review.” That did not reproduce the result. On the same subset, cost fell only 19.5% (p-value=.177), and score noninferiority missed significance (p-value=.073).
The explicit stopping rule appears to be the important part.
This is why I'm strongly against automated self-improvement by agents without human review. Not because I'm against the concept. But because it just doesn't work.
A harmless-sounding instruction can trigger another plan review, which finds more work, which triggers more implementation, which triggers another code review. It can hurt performance and, just as importantly, increase costs. These kinds of things compound.
Write your skills yourself. Have your agent edit them, but you should review every single word.
I expect the latest cleanup of my own skills and instructions to reduce my token costs by roughly 3x, primarily by removing inappropriate instructions left over from earlier, less sophisticated agents. I'm planning to share the final results once I'm done optimizing.
So why have skills and instructions at all if frontier models are so good without them? Two reasons.
1. To encode the things I care about as a developer, many of which are truthfully just opinions.
2. To facilitate fully automated and unattended, long-running work.
I regularly run cross-agent loops using both Claude and Codex, across several reasoning levels, from the command line. They last hours and sometimes days. Skills make that possible. They just need to be very light.
Before Opus 5 and gpt-5.6 (Luna, Terra, and Sol) came out, my instructions and skills were roughly 10x their current size. Now my instructions are around 3,000 total tokens, and they seem to get smaller every day.
More to come soon about this benchmarking approach, which focuses on finding statistical significance with minimal costs. Follow me to stay tuned.
Useful links:
Anthropic, “The new rules of context engineering for Claude”:
claude.com/blog/the-new-rule…
OpenAI, latest-model guidance:
developers.openai.com/api/do…
OpenAI, reasoning-model prompting guidance:
developers.openai.com/api/do…
David Good retweeted
I'm obsessed with data, yet I had none to show if Agent Layer skills/instructions helped or hurt performance. I'm building a low-cost benchmark with @datacurve's DeepSWE tasks. I hope to share it soon. I need to reduce cost without hurting scores. github.com/conn-castle/agent…
David Good retweeted
Anyone using coding agents can create a personal assistant. I first built Steward to control access to my email. Claude and Codex have plugins for this, but I’m REALLY glad I decided to build my own. Stew uses @LinkedIn, @X, @todoist, and more.
Read more: View article
David Good retweeted
Before posting more, I want to share how my personal agent Stew turns raw thoughts into X posts. I pick a topic or idea, type/dictate a free-form stream of consciousness. Stew uses my x-write skill (github.com/nicholasjconn/ski…) to write using my words. I edit and approve. Stew posts.
David Good retweeted
Stew, my personal agent, showed me a post from an AI newsletter framing agentic coding as the best video game ever. That really resonated. I'm lucky enough to play this game for work and fun. I'm going to start sharing what I learn, hoping it will help someone. Stay tuned!
Last night, after a weekend hackathon, I (soft) launched @iolify, a new desktop app console for AWS, initially focused on DynamoDB. More details coming soon.
iolify.github.io/
2
Just upgraded, and noticed a couple features seemingly broken in @intellijidea 2022.1.4 on macOS:
1) File > Open > Scratch file does nothing
2) Help > Find Action... does nothing (Shift Shift does not work either)
2
Any tips on how to "auto-authenticate" callers in lower environments based on the presence of an `auth-bypass` header with @SpringSecurity OAuth2 resource server?
3
Replying to @helloworldless @SpringSecurity
"Auto-authenticate" meaning the caller does not need to provide an access token
Do *not* want to disable/ignore auth entirely. This is just for ad-hoc bypass, e.g. for human users troubleshooting in lower environments
More than just your average code example from Effective Python by @haxor!
Use classmethod Polymorphism to Construct Objects Generically: github.com/helloworldless/ef…
#mapreduce #multithreaded #polymorphism #classmethod #python
2
> If you can possibly *not* build a distributed system, you're gonna go home earlier and be less stressed every single day.
- @tlberglund on The NoSQL Database Podcast NPD015
💯
2





