AI-Assisted Engineering AMA #2

A harness and evaluation loops can compensate for some limits of a local model, but that compensation becomes expensive. Popular agent environments evolve around newer, stronger models; a team must maintain the adaptations for an older or weaker one. The expected savings may also disappear when local inference requires substantial hardware and throughput, especially with parallel agents. Durable acceptance criteria are a better investment because they continue working when the model changes.

A multi-agent workflow is not valuable by default. Separating planning, implementation, and verification reduces the tendency of one actor to confirm its own solution, while parallel search helps with research and independent audits. Every extra agent, however, reloads context, spends tokens, and makes the system harder to understand. Start with one agent and add a distinct role only after a measurable speed or quality constraint appears.

For a service without documentation, the first step is not to generate a large description. Define the observable behavior and completion condition for one change. Capture what the system does today, place characterization tests around the affected area, and build a short feedback loop from compilation, linting, tests, UI snapshots, or a specialized verifier. The agent then receives boundaries, an error signal, and a provable outcome instead of being asked to understand everything at once.

A technical loop cannot repair an organization's loss of meaning. If nobody can explain the product's purpose or domain ownership, generated diagrams merely multiply ownerless artifacts. Information security follows the same logic: begin with a threat model spanning the harness, model, tools, repository, and data. Full isolation is possible but costly, while corporate gateways and access policies offer intermediate configurations. Controls should mitigate a specific risk without destroying the system's economic purpose.

AI budgets are more useful when allocated to a project or task type rather than as an identical personal allowance. Tracing through a corporate gateway reveals the cost of an action and lets the organization compare it with the previous process. Once evaluation sets and telemetry exist, requests can be routed by difficulty: a cheaper model handles a repeatable operation, and a stronger one receives the hard case. Routing should follow local quality measurements rather than a model's public ranking.

Tokens, pull requests, and running agents are not evidence of value. Faster delivery exposes the next constraint: an unprepared backlog, a weak product hypothesis, or limited user capacity to absorb change. The outcome metric must therefore describe changed product behavior and customer response. After a product matures, each additional feature brings less marginal utility, while large or regulated companies accept risk more slowly. Engineering acceleration matters only when the vehicle is heading somewhere worthwhile.