Greg Johns (@gjohnsx) header
Greg Johns (@gjohnsx) profile picture

AI developer building automation tools for real estate. MCP servers + AI agents. Personal liberty, sound money. Orlando.

Orlando
Joined February 2021
Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)Photo by Greg Johns (@gjohnsx)
Greg Johns retweeted
Nearly every process in the SDLC warrants a custom skill: /create-ticket /fix-bug /implement-ticket /review-code /open-pr /fix-pr-comments If our team repeats a process, a skill means we can standardize, automate, and continually improve the process.
45
23
2
492
41,530
Greg Johns retweeted
Most platform teams I speak with would disagree with this take. They are all trying to build factories because they want more automation, measurable improvement, and governance as agents scale across their orgs. Whether you call this a "factory" (a term I also don't love fwiw) or something else is less important than investing in infra for scaling coding agents. Whether that infra is just marketing depends on who is selling it to you. There are definitely folks pitching "factories" that aren't anything more than skills strung together, but there are also infra platforms emerging that let you define how all your org's agents work together in code, continually measure and improve them, govern and secure them, etc. Some folks will build this infra from scratch, some will build on something like what warp offers -- but I strongly believe everyone will eventually have closed-loop agentic systems in the cloud, and that to get there you need real infra investment beyond a new way of using local coding agents.
hot take: you don't need a software factory, and i don't love the term. it's just marketing all you need is good software engineering and investing your tokens in the right places. the more deterministic the better (and token efficient)
29
8
5
192
20,848
Greg Johns retweeted
hot take: you don't need a software factory, and i don't love the term. it's just marketing all you need is good software engineering and investing your tokens in the right places. the more deterministic the better (and token efficient)
all the companies selling you a software factory produce a shockingly tiny amount of software
439
581
180
8,133
509,005
Greg Johns retweeted
Your factory should be accessible everywhere your team works. This is key for adoption across a team. This means multiplayer prompting in Slack, tagging from task boards like Linear or Jira, or fully automating against a backlog. It should also be accessible via MCP to put work in through local agents, and via API. You’ll find over time that it's convenient to hop between all of these mediums throughout the lifecycle of a task.
11
7
2
151
17,069
Greg Johns retweeted
I often see tweets that a software factory can be built with a string of prompts tying together cloud agents. My experience is that this is false. Companies often start there and regret it because it doesn’t scale.
134
21
19
521
67,309
Greg Johns retweeted
I generally like @poteto's hot takes, but whether adversarial code review is useful or not is something to measure in your own system, not go on vibes with this is an example of the type of tweet that goes viral with lots of "yes, i agree!" but the factory engineering approach is to benchmark your own workflows with and without adversarial review and see if the results are better and if the cost is lower i'm going to do this internally and will publish what we see on Warp's codebase using Warp Factories. I encourage others to do the same, and, if you aren't set up to run this kind benchmark we can help
maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis
35
9
6
345
26,372
Greg Johns retweeted
Your team should have a shared configuration for your agents that lives in code. The same way you use Terraform to deploy cloud infrastructure, you use a factory configuration to deploy your software factory. This gives you the typical advantages of version controlled systems: check in changes with code owners, test changes with rollback support, and review updates as diffs.
25
7
2
67
6,596
Software engineers have been through a lot of change in the past two years, transitioning from writing code by hand to steering agents via local interactive prompting, the current paradigm. The next paradigm, software factories (where agents do more and more autonomous work) is particularly challenging. The metaphor of “factory” is kind of a bummer. It feels like your job as an engineer is either working on an assembly line, or building the assembly line that obviates the need for your talents. If you use the “AI teammate” metaphor (which I also dislike), then it feels like engineers are becoming managers of sycophantic and somewhat inept junior engineers. For folks who take pride in the craft of building, this all feels like commoditization of software creation. I disagree with this outlook. On our team, engineering involves two very important challenges today: building the product, and now, building the factory that builds the product. First, our engineering team is still responsible for the quality and usefulness of the product – this cannot be delegated to agents. Humans are uniquely positioned to know what to build, how the pieces should fit together, what the product experience ought to be, what constitutes good design. There are two caveats: Some types of product work that don’t need any human input at all and can be purely automated. (Fixing server crashes, simple bugs, dependency upgrades, etc.) The second caveat is that product engineering will work differently. Rather than engineers working locally with their own bespoke setups, they will hand off as much of the software lifecycle as possible to the factory. All of this work will be inherently visible and measurable, and that’s a feature, not a bug. Developers should expect to measure and iterate on other aspects of how they use the factory, including how much their agents ship and at what cost. Engineers could view this as a drag, but they also can view it as a game, and winning that game means shipping more for less, using an engineering mindset to improve. This brings me to the second job of the engineer, which is to design the factory itself. We call this factory engineering. It consists of measuring the performance of your factory. This needs to be done with an engineering mindset – you don’t adjust your factory on vibes; you measure, test, benchmark and then improve. At Warp we make it explicit that the expectation of engineers is not just to build products, but to improve factories. We envision both jobs as being the responsibility of all engineers to make it clear that factory engineering is now part of software engineering. On our team, the folks who like it most are senior engineers who love hard systems problems, who enjoy optimizing and debugging performance issues. It requires a lot of thought and testing and patience. We lean into it as a new skill to learn. Ultimately, this lets us ship better products more quickly, which has always been our mission.
37
26
12
341
68,608
Once engineers started working in public with a software factory, they typically had an aha moment on how it’s better. For instance, when you start work in Slack, you can easily tag a teammate in for their opinion and all the context is already there. You can continue that work on your phone from anywhere. You can search your team’s history, and use that work to inform the next prompt. There are a lot of non-obvious benefits that product engineers start to feel from the new way of working that make them want to stick with it.
11
1
1
80
14,686
Code review is now risk mitigation. The goal should be to reduce the risk of hitting "merge" without reading each diff line-by-line. Close PR review will become progressively less needed.
We made a skill for assessing the risk of merging a PR: /merge-risk This addresses the code review bottleneck by helping devs trust the agent for lower risk changes. It grades risk across 5 categories: alignment, impact, verification, rollback cost, and error detection. - Alignment: does code follow the PR description and related tickets? - Impact: what is the blast radius, and does it touch sensitive code paths? - Verification: does the change include end-to-end verification? This means computer use, screen recordings, before/after screenshots... going beyond unit test coverage. - Rollback cost: once it's shipped, how trivial is it to revert? @mattpocockuk calls this a "two-way door" vs. a "one-way door." - Error detection: are errors reported, logged, or simply swallowed? Try it out: > npx skills add warpdotdev/common-skills --skill merge-risk
20
7
232
40,039
The goal of software factories is not to get to full automation (not today). It's to get the most out of coding agents so that you are shipping the highest quality software with the most customer impact as quickly and efficiently as possible. You’ll want to prove that you are doing this by carefully measuring and tuning your factory. We call this “factory engineering.”
32
8
4
191
11,795
Greg Johns retweeted
pstack v0.15.27 ships with a new skill /create-formal-verification-skill! it's still in beta, and similar to /create-verification-skill, this is a meta-skill that creates a new skill and formal verification suite based on the characteristics you want to verify. for example, you may want to model state machines and systems with concurrency using TLA+, or use kani in rust to check for undefined behavior. if you're new to formal verification, think of it as mathematically proving properties and invariants (something that must always be true) or "laws" of your system. usually, this means modeling your programs in a separate language like with TLA+ and then running the solver on that spec. newer approaches like with kani and @bendlang take a different approach where there is no separate language: you write kani in rust, and bend in bend (it's also a programming language, and the proofs are written as laws and expressed in its type system). historically formal verification never reached mainstream status because of the difficulty in modeling and writing proofs. it's quite difficult to model your business requirements and keep them in sync! but i expect that we will see more innovation in formal verification in the future as being able to formally verifiy that the code does what you intended gives your agents superpowers in writing correct software. in the meantime though, i've been experimenting with modeling our code with TLA+ and have already found several bugs with the skill created by /create-formal-verification-skill. try it out and let me know how it goes!
188
171
48
3,180
397,633
Greg Johns retweeted
These are the golden years: 1. The chaos of leaving the house 2. Sticky handprints  3. Hearing them play without you   4. The “watch this” moments   5. Bedtime stories that never end 6. Their little voice saying "Dada"  7. The toys on the floor   8. Belly laughing together  9. Holding them when their sad Life is going so fast. Don't miss it parents.
11
72
3
1,331
46,632
Greg Johns retweeted
Give your agent a way to verify its work. This can dramatically improve the quality of output. With this process in place, Claude Code was able to write 90%+ of its code itself. When generating software at a faster rate than you can review it, the bottleneck shifts to proving that it works. @bcherny and team invested enough in verification, we've been able to move fast while maintaining stability. Code reviews aren't going away, but you need to think of it differently. One human reading through line-by-line is no longer going to work as a gate, it needs to evolve into a multi-layered verification system, built with fast deterministic feedback loops for your agents. This is where e2e tests, feature maps (as @poteto has been advocating), property tests and multi-agent adversarial reviews come in. These all provide opportunities to catch errors early and fix them before things get out of hand. You don't have enough eyeballs for the amount of code being written so your focus needs to be on product intent, risk assessment and evaluating evidence rather than reviewing code. Delegate the code checking work to your automated tests, scale your approvals based on risk, and turn your test failures into living evaluations. This approach helps maintain speed without losing reliability. Let the agents do what they do best and let you focus on what makes you irreplaceable: judgement, direction, and ownership. We tried to capture how we handle code review and verification in this writeup. Hopefully it's helpful.
Claude writes about 90% of Claude Code's code. From June to September, merged PRs more than doubled while the share reverted fell by more than 40%. We wrote about how we changed code review to get there, and how to adopt it: claude.dev/blog/solving-code…
220
176
61
2,536
200,704
Greg Johns retweeted
My take on agentic reviews. If you want best balance between cost and quality, don’t use the SaaS codex/claude reviewers. Instead… 1. create your own an agent, triggered by GitHub webhooks on PR raised or updated events 2. Give it company and architecture context. Any key insights that the SaaS reviewers would never have. 3. Give it risk guidelines and tell it to separate out nit picking things from those that should block PRs from merging. (Agent approves or request changes on GitHub PRs based on risk. This has the benefit that request changes on github block and coding agents have to address the risks before the PR gets through) 5. Probably goes without saying but the codex/claude agent still does the review just not via the SaaS service . As it stands today, you’ll get better results this way. Might even be cheaper but definitely will be better use of resources. A nit picking agentic code review can be worse than not having one at all . If you just want something off the shelf that ticks a box ignore the above and just use the SaaS reviewer as they aren’t inherently bad, just not great, yet!
I could not imagine being productive without Claude Code Review. It runs on all of my PRs, and catches almost every bug before it goes it. This is a piece of the secret sauce that lets the team and I move so quickly while raising the quality bar
38
4
4
136
15,228
Greg Johns retweeted
Some mistakes you might be making: Your effort level is too high Your skills are too long You're not aligning before you start work You're aligning too often, and too hard Your automated checks are too lax Your custom tooling is too token-inefficient You aren't doing enough automated review You aren't reviewing previous coding sessions (with an agent) You're doing too much human review You aren't vibe coding enough prototypes Your agent can't verify its own work You're babysitting the agent too much You've overloaded your global steering files Your codebase is hard to navigate You aren't building a shared language Your codebase architecture allows bad tests to be written You aren't thinking strategically enough You aren't thinking enough
272
246
60
4,022
212,497
Greg Johns retweeted
perhaps one of the most durable parts of pstack are its engineering principles: very concise skills (~10-20 loc) that distill down good practices into its core. as models get smarter, i expect most of pstack skills to compress into short principles or workflows (a generic list of steps) and that i will be able to delete a few of them. how it works is that /poteto-mode contains a short index of available principles and tells agents "In your reply, name each principle that shaped a decision and the specific choice it changed." this is why you'll often see a Principles Named or Used section at the end of the agent's response many of these principles were extracted from my sessions correcting agents since the start of this year. some are inspired by problems i see people talk about, like Attack the Premise, inspired by @VictorTaelin's post nitter.cf/VictorTaelin/status/20… you can also use the principles as skills if you want greater control, by invoking /principle-<name> github.com/cursor/plugins/bl…
I'm a newcomer to @poteto's plist; But it seems to me that there is some very significant wisdom encoded in there. I suspect you could teach a master class on the principles.
118
122
29
3,429
266,212
Greg Johns retweeted
Do you need agent skills if you're following a great process? Of course not It's just that skills make it easier to follow that process
95
16
5
612
50,766