12-Factor Agents: Patterns of reliable LLM applications (AI column)

I watched with interest. speech studied GitHub 12 factor app for agents. Dexter Horthy is an experienced engineer and founder of the company. HumanLayer (startup). Dexter's key idea is that instead of making one big agent, you should make small, focused agents -- similar to monoliths and microservices. Dexter believes that agents are just software, so if you know how to swith and while, you can create agents. But to create them, it is better to use non-frameworks that allow you to obtain 70 - 80% quality, and then it is required to reverse engineering frameworks in order to push the agents to the remaining 20%. The author proposes a set of 12 factors that are similar to 12 factorsIt became part of the principles of creating cloud native applications. But now I can tell you about everything. 12 factor

1. Natural Language to Tool Calls. One of the most common patterns in building agents is the conversion of natural language into structured tool calls. This is a powerful template that allows you to create agents that can reason about tasks and perform them. 2. Own your prompts. Do not transfer the management of prompts to the side of the framework 3. Own your context window. LLMs are stateless functions that turn inputs into outputs. To get the best exits, you have to give them the best entrances. 4. Tools are just structured outputs. The tools don't have to be complicated. At its core, they’re just a structured output from your LLM that runs deterministic code. 5. Unify execution state and business state. Even outside of the AI world, many infrastructure systems are trying to separate “execution state” from “business state.” For AI applications, this can involve complex abstractions to track things like current step, next step, waiting status, number of repeated attempts, etc. This separation creates complexity that may be useful but may be unnecessary for your use case. 6. Launch/Pause/Resume with simple APIs. Agents are just programs, and we have certain expectations about how to run, poll, restart and stop them. 7. Contact humans with tool calls. Integration of interaction with people through tools (decision-making or supplementary information) 8. Own your control flow. Control flow allows you to add probes, tests and more 9. Compact Errors into Context Window. Error handling through the context box without blindly adding stack traces 10: Small, Focused Agents. Creating small agents with 3-10 Steps instead of monolithic solutions. 11. Trigger from anywhere, meet users where they are. Agents must be accessed from different input points: slack, email, API, ... 12. Make your agent a stateless reducer. Agents should be designed as stateless functions for better scalability

In summing up the presentation, 1. Frameworks and building blocks for agents. Dexter does not oppose frameworks, but suggests using them. 12 Factors to make more powerful and reliable agents (Maybe the frameworks will support these later. 12 factor) 2. An incremental approach. The fastest way for agent creators to get quality AI software is to take small, modular concepts and incorporate them into an existing product. 3. Engineering. Even if LLMs become exponentially more powerful, core engineering techniques will remain valuable for creating more robust, scalable, and supported Gen AI software. 4. Interaction between humans and AI. Future agents don’t have to be fully autonomous – it’s important to find ways for agents to cooperate with humans.

#AI #ML #Software #Architecture #Processes #DevEx #Devops #Agents