State of AI4SDLC: AI assistants to agents
Why the development bottleneck moves from coding to specs, evals and the engineering operating model
Slide contents
1. State of AI4SDLC: AI assistants to agents
Why the development bottleneck moves from coding to specs, evals and the engineering operating model
2. Talk plan
Research: adoption high, trust lower.
SE 2.0 shifts bottlenecks to delivery.
Task management becomes a work graph.
SDD and evals make agents governable.
3. 01. What AI4SDLC Research really shows
Adoption has already happened; sustained impact comes through verification, process and platform
4. AI is no longer an IDE experiment
Manage the new development physics
default tool — Mass usage — AI writes code, tests, docs, explanations.
task-dependent — Uneven impact — Coding speeds up before delivery.
verify-first — Management question — Manage adoption, throughput, quality/risk, economics.
5. Why one productivity metric is not enough
Read AI4SDLC as signals, not one magic speedup
What we measured
Public research from 2023-2025.
Survey of engineers and leads.
What to remember
Self-report is not causality.
Impact depends on maturity.
6. Adoption is high, trust is lower
Engineering systems must close this gap
Everyday tooling — Part of the workday.
Quality needs proof — Review, tests and evals.
ROI through process — IDE pilots need an operating model.
7. Coding faster, delivery not
Existing bottlenecks grow
Review/tests/releases get more changes.
Weak framing generates wrong results faster.
Without metrics, teams debate impressions.
Next level: managed work loop.
8. 02. SE 1.0 -> SE 2.0
From role-based SDLC to agent-based SDLC: not a tool change, but a delivery loop change
9. Role-based SDLC and agent-based SDLC
In SE 2.0 humans write fewer lines and manage intent, context and verification more explicitly
Software Engineering 1.0 (Role-based SDLC)
Idea
Req
Dev
Test
Deploy
Support
Product
Analyst
Developer
QA Engineer
SRE
Support Engineer
Handoff losses
Implemented some AI scenarios within roles
Local optimizations inside
On brownfield projects we test AI scenarios by role
Software Engineering 2.0 (Agent-based SDLC)
Idea
Req
Dev
Test
Deploy
Support
Product
Engineer
Support Engineer
Fewer handoff losses
Faster end-to-end scenarios
On greenfield projects we try agent-based development
Carry over scenario learnings
Carry over scenario learnings
10. The engineer's role gets wider
Manage the system that produces changes
Framing — Define goal, context, constraints and DoD.
Orchestration — Split work, delegate, integrate results.
Validation — Check behavior, regressions, security and intent.
11. Team works by new rules
Not just tools
Tasks target humans and agents.
Context/checks become dev interface.
CI, tests, logs become evidence.
Parallel agents need rules.
12. 03. AI-native operating model
When artifacts are created faster, manage the work graph, platform and metrics
13. Ticket becomes work signal
Work graph knows context earlier
Classic tracker
Task and status written manually.
Context spread across chats, PRs, docs.
AI-native work graph
Intake and status partly computed.
Intent, spec, PRs and tests connected.
14. AI-native rollout is platform work
Scale requires shared capabilities
Instructions and context — Repo rules, ADRs, conventions, safe commands.
Verification and observability — Evals, CI, telemetry, cost, latency.
Guardrails and economics — Permissions, secrets, limits, approvals.
15. Manage chain, not LOC
LOC is weak signal
Adoption: who uses AI, where.
Throughput: lead time, review, deploys.
Quality/risk: defects, incidents, eval score.
Economics: tokens, infra, waiting, rework.
16. 04. Spec-driven development
AI brought specs back, but now they are a working interface for agents, not bureaucracy
17. V Model
Systems Engineering V-Model: operations concept, requirements and architecture, detailed design, implementation, integration test, system verification and operation
18. SDD resembles V-Model, for a new reason
Old specs traced work; modern SDD delegates
Old spec-driven
Requirements -> design -> code -> verification.
Strength: expectation traceability.
Modern SDD
Spec becomes executable context.
Agents need intent, boundaries, checks.
19. Specification: contract with the agent
Agents need managed context
Change — Goal, impact, flows, expected behavior.
Do not break — Compatibility, data, security, APIs.
Proof of done — Criteria, checks, tests, logs, rollback.
20. SDD: intent → proof
Docs matter through checks
Intent: why and what outcome.
Criteria: success and regression boundaries.
Plan: verifiable work steps.
Verification: tests and review confirm intent.
21. Four practical SDD variants
Make the spec a portable contract
GitHub Spec Kit — Spec -> plan -> tasks -> implement.
Kiro + OpenSpec — Structured and open spec formats.
Lightweight SDD — AGENTS.md, tests, PRD/RFC, checks.
22. Bad specs speed wrong code
SDD makes weak framing visible
Unclear intent amplifies ambiguity.
Empty criteria make review taste-based.
No tests/evals means no proof.
Large tasks need verifiable chunks.
23. 05. Evals for SDLC agents
Evaluate not response elegance, but whether an agent reaches a goal in a replayable environment
24. Basic template for one eval case
A small replayable development episode
Frozen state — Fixed repo, data, issue, environment.
Agent contract — Allowed changes, commands, boundaries.
Hidden executable judge — Judge runs checks; cases stay fresh.
25. Evals must cover real work types
One coding benchmark is not enough
Product / Engineering
Requirements: brief -> stories.
Feature/bug: spec + regression oracle.
Quality / Operations
Code review: merge-blocking issues.
Tests/incidents: regression, units, incident bundle.
26. Agent autonomy cannot be one score
Score result, trajectory and cost
Outcome — Task solved; criteria pass.
Trajectory — Plan, changes, rollbacks, tool use.
Cost & safety — Repeatability, latency, constraints.
27. Standard: spec, evals, CI, review
Minimum agentic loop
Artifact: tasks, PRs, bugs, incidents.
Frozen state: start and conditions.
Hidden judge: executable checks.
Scorecard: outcome, safety, spec fit.
28. AI-native development is not managed by prompts
The practical takeaway
Bottleneck moves to intent and verification.
Specs become human-agent contracts.
Evals make agent quality measurable.
Manage work graph, quality/risk, economics together.
aidevconf.org · ai4sdlc-research.space · tellmeabout.tech
29. Links and materials
Sources and practices
AI Dev Conf and AI4SDLC Research.
Spec-driven development: Telegram post.
GitHub Spec Kit and AWS Kiro specs.
OpenAI Codex, harness engineering, evals.