TrueFoundry

A harnessed LLM agent, clearly explained! Two agents can run the same model on the same task and finish as expected. But one of them can spend nearly 3x the tokens to complete the task. The extra usage originates from the code wrapped around them, which decides what reaches the model's context on each call and how many calls there are. For instance, consider a tool that returned 50k tokens of JSON at some step. If it stays in the context, the model will continue to read that payload again at every subsequent step. Tool definitions behave the same way. A server can expose 50 tools, each with a name, a description, and an input and output schema. By default, all of them will stay in the prompt from the first call, whether the agent uses them or not. However, an optimally built harness can avoid that unnecessary cognitive load on the model. More specifically, one core design principle of harness engineering is to push things out of the model at the right time: - Memory holds the state that weights and context shouldn't carry. - Skills hold procedural knowledge. These cover the operating procedures and heuristics that specialize a general model. - Protocols hold the interaction contracts for users, other agents, and tools. Do note that the context never disappears permanently. It is always loaded when needed, and the harness decides how much is loaded and when. For instance, to manage a 50k token payload, a harness can write it to a file and keep a preview and a path in context, hand the work to a subagent whose context is discarded afterwards, or summarize the older messages once the conversation passes a threshold. If you want to see this in practice, TrueForge is an open-source harness that already implements these practices. Tool schemas are deferred unless preloading is switched on, large responses go to a sandbox file, and generated code calls tools back through the harness, so the sandbox never holds the credentials. The two agents I talked about at the top are from DevRev's Enterprise-Bench. TrueForge solved the same number of tasks as Claude Managed Agents on the same model, using a bit over a third of the tokens and around 40% fewer tool calls. Here's the GitHub repo: github.com/truefoundry/truef… (don't forget to star it ⭐ ) I also wrote a full breakdown of where agent tokens actually go inside a run, covering context accounting, the strategies above, and the benchmark in detail, and TrueForge worked with me to put this together. Read it below.