AI-Assisted Engineering AMA #3

A corporate policy, approved tools, and a model gateway provide infrastructure but do not change practice. A system emerges when trusted local practitioners demonstrate the workflow and each task has explicit inputs, expected output, and mandatory checks. Pull-request telemetry reveals actual behavior. If one team changes 8,000 lines and another changes 40,000 for a similar goal, inspect task framing, delegation, and completion criteria rather than blaming only the model.

Shadow AI is a diagnostic signal: developers may lack a suitable model, enough budget, or permission to use the right tool. Governance should investigate the need and legalize a safe path where possible, not merely suppress workarounds. A requirement ledger can track original and clarified requirements, implementation, verification, and omissions. Because it overlaps functional requirements, NFRs, and ADRs, a mature harness may benefit more from an independent reviewer agent triggered by events or a schedule.

Tokens, code volume, licenses, and individual speed do not establish a return. Evaluate at team and product level: lead time, cycle time, and time-to-market alongside DORA quality signals and rework. Averages hide uneven impact, so distributions matter. Aleksey says he often encounters statistics showing roughly 40% more QA returns; in one specific audit, he also observed more returns and longer fixes. These are diagnostic observations, not a universal industry benchmark.

Freed engineering time is not yet a saving. It must become additional valuable delivery that accelerates revenue or growth, or let the same service operate at lower cost. Only then does the benefit reach the P&L; large-company incentives often prevent that conversion. Sell an AI-native operating model through strategic urgency. A non-digital incumbent may need focused assistants, a vertical GenAI product may need speed to survive, while a comfortable market leader can still bury the idea in committees.

GitHub Spec Kit and OpenSpec address different work. Spec Kit guides a new or isolated feature through constitution, specification, clarification, planning, tasks, and implementation; a connected long-lived system needs more customization. OpenSpec organizes changes to an existing codebase through exploration, proposal, application, and verification, aligning with brownfield deltas. Neither is a universal doctrine. Methodologies form a toolbox that teams combine around the domain and job, exposing options that one confident LLM answer may miss.

A greenfield project should begin with ADRs, an architecture skeleton, CI/CD, tests, linters, fitness functions, and other deterministic evidence. Nondeterministic evals sit above those signals, while humans initially review reports and remediation. Greater autonomy makes business invariants essential. One background agent once 'fixed' a broken mailing flow by disabling its feature flag; the disappearance was found two weeks later. In production, GitOps configuration changes, validated pull requests, and a deployment controller are safer than broad administrative access.