AI4SDLC: What I Would Do Differently

The opening framework divides the stack into three parts. Obtain frontier models, a standard agent loop, and commodity infrastructure from providers where possible: these change faster than a typical product team can maintain equivalent implementations. Constraints on deployment and the operating environment may justify exceptions. Even then, the need to control execution should be distinguished from the ambition to build everything. Adaptation belongs where company-specific requirements begin: routing work to an appropriate model and budget, preparing context, and connecting internal systems. The organization must retain responsibility for access rules, agent identity, observability, and evidence of quality. This makes ownership a concrete discussion instead of a general demand for the best model. A team of twenty to fifty people may cover much of the first two categories with existing products. A large organization has more resources, but also more systems, approval processes, and established habits that slow change.

Context is the material needed to complete the task. It may include a ticket, requirements in Confluence, and agreements from work chats. A separate service can assemble these for a coding agent, but increasing the amount of text does not establish that it helps. The test is whether relevant tasks are completed better after changing the context. In the code-review example, options include providing only a diff, adding surrounding lines, or including the structure of the affected class. These alternatives should be compared on the same evaluation cases. Adapters solve a different problem: the model may recognize a standard tool while the company runs a heavily modified version. Without information about local behavior, the agent applies familiar assumptions to the wrong environment. Routing, context, and tools therefore provide a relatively fast improvement cycle. Model training and computing infrastructure evolve on a slower cycle that is chiefly available to larger players.

A capable model does not by itself produce a working process. Outcomes depend on the harness, context, environment, and permissions. An agent that can plan a fix but has read-only access still leaves execution to a person. Access should therefore be designed with security teams around specific scenarios. The episode discusses a distinct agent identity and delegated, limited permissions: for example, reading selected data and writing results into a designated workspace. If underlying systems cannot yet enforce this delegation, platform tools may need to enforce the restrictions. This is shared engineering work. Asking a person to approve every step does not automatically solve the problem either. After a long autonomous investigation, the user may not understand all the consequences of the proposed action and may approve it out of habit. Trust requires clear boundaries and a way to inspect what actually happened.

Tool interfaces need to support those boundaries. MCP provides tool discovery and invocation, but does not guarantee a safe effect. Before a call, the system should establish who is acting, which permissions apply, and which policy governs the action. During execution, useful properties include idempotence or an explicit indication that an action is irreversible, understandable errors, and concise responses. Afterwards, the resulting state needs verification and an audit trail. Alexander uses verbose Figma responses as an example: a large JSON payload consumes context and money when the next step needs only a small portion. Exposing an existing OpenAPI definition through MCP can work formally while remaining a poor interface for an agent. A collection of skills has the same problem: publication counts do not establish usefulness. Tools and instructions need evaluation against real tasks, beyond checking that they are connected.

Demos, telemetry, and evaluations answer different questions. A demo presents a successful scenario. A trace exposes actions, errors, and cost. An evaluation set supports comparison after the system changes. In the project-management example, an assistant works well on its developers’ own tickets but struggles when projects span several trackers with inconsistent links. Some users will report the problem; others will simply stop using it. Feedback and adoption metrics alone cannot explain the difference. The speaker does not propose postponing every launch until a perfect evaluation system exists. His criticism is that evaluation often remains postponed after initial demos. Real cases should become repeatable tasks that can be rerun after changes to the model, context, or harness. In the code-review agent example, offline checks were combined with observations of user reactions and acceptance of recommendations after release.

The next step is to divide development into verifiable units of work: fixing a defect, reviewing a change, generating tests, or investigating an incident. An existing agent loop with suitable skills may be enough for each unit; a custom implementation needs a separate justification. Once inputs, tools, and success criteria are understood, a smaller model becomes a candidate whose quality and cost can be compared. A larger model is particularly useful when the problem remains ambiguous, spans several systems, and requires exploration. Alexander also discusses separating planning from execution: an external model receives permitted metadata to prepare a plan, while a local model performs well-defined operations on internal data. This is an example of workflow design, not a universal solution to access constraints. Specialization also has a limited shelf life: the next model may perform the same task without accumulated workarounds. The organization therefore commits to reevaluating its adaptations and retaining success criteria as implementations change.