AI-Enhanced API Design

Alexander opens the episode with another Google paper, a continuation of the API governance work from the first episode. He found it scanning Google's papers from the newest backwards: inside were references to an earlier work — from roughly 2023, by his reckoning — and to a 2016 study, so the series came together for him as a saga of Google's approach to APIs. The paper, he says, has four authors from different universities and one from Google, while the main author of the previous work and the book left for Meta years ago and appears only in the acknowledgements. Alexander then offers a technique: before reading a paper, break it into parts — abstract, related work and the gap left open, methodology, results, discussion, limitations, conclusion. With no limitations section, it looks as if the authors' approach had no alternative. He caricatures bad methodology: at a conference, survey only those who made it to the talk, and only architects did, then declare the level of knowledge excellent.

They start the walkthrough with the introduction. The authors note there is plenty of documentation about APIs, while how usable, flexible and secure a contract is rarely gets worked through. As Pavel reads it, the focus is not governance but developer experience and how contracts get implemented: those three questions arise at design time. The part about AIPs, API improvement proposals, carries over from the previous paper, and Alexander asks Pavel to recap it for anyone who missed the first episode: in Pavel's telling, standards, best practices and recommendations for contracts, while an AIP is also a process for adding rules that reminds Alexander of Python. Agreed rules are encoded into a linter that runs over specs and gives feedback immediately. The author of the previous paper, Alexander recalls, insisted that what matters is not REST as such but a resource-oriented API: fix the actions on resources and you can build one on top.

Pavel immediately maps this onto the bank. The context is narrowed to REST, with grpc and GraphQL set aside; the base is OpenAPI with a pinned version, and on top sits a corporate standard invented inside T-Bank five years ago for all REST contracts: it standardises response codes and makes the OpenAPI approach stricter. The result is a matryoshka: the public API platform adds its own tags, extra parameters and stricter demands on responses and nesting. Everything stays backwards compatible, the final contract fully conforms to OpenAPI, and an outside developer opens it in Swagger Editor as an ordinary, merely stricter spec. The difference from Google is the price of changing rules: the corporate standard is changed by a board of developers, and one version, by Pavel's account, took a year to ship. The platform standard is flexible — a spec from a systems analyst, a conversation with the tech lead — but it does not scale; OpenAPI itself ships a version roughly every five years.

The compact team runs no fundamental research of the Google kind, so Pavel starts from the obvious. Surveys placed inside the documentation work fine. He trusts deep interviews less: you catch the wrong person and draw the wrong conclusions — right now a systems analyst is publishing on the platform a redundant contract full of his own patterns, formally valid. Then metrics: time from idea to an integration's first calls, the time to change an existing method's contract (an order of magnitude different from creating one), how fast the same developer produces a second method, click maps in the docs. Pavel splits developers into light and dark ones: almost everything is known about internal producers — language, code in git, rough time to market — while a fog of war hangs over consumers, and lifting it is one of the main goals of the research. His last ideas are about data sources: open source, where outside developers comment on published contracts (once, before anything was published, someone brought an SDK in Crystal), in-source if you cannot open up, and public events: at the IT Picnic, a stand with a game about writing a contract with ChatGPT drew 500 people in a day.

The paper's methodology rests on three pillars: a controlled experiment, qualitative interviews about how consumers perceive the resulting specs, and experts assessing what the model produced — it was given product requirements, a social network for children and parents as far as Alexander recalls, and asked to write specs. Three groups: standards plus linter; standards but no linter and so no feedback; and nothing at all. Participants got about two hours, a screen recording and eighty dollars. The first group met 71% of the requirements against 62–63% for the others, and the average number of errors the linter later found was around 14 versus roughly 19. The funnel is a story in itself: 234 people filled in the interest group's survey, about 6%; 2.5% of developers agreed to take part; fewer than 1% finished — 34 subjects out of 4,000, and by Alexander's memory either 34 or 33. That is why Pavel calls the experiment unrepeatable, and Alexander does the arithmetic: with ten thousand engineers in the bank you would gather maybe four hundred, three would finish, with nothing left to measure.

What Alexander liked most was the consumer-side assessment: pieces of contracts were cut into samples and shown in pairs — which do you prefer. The standards-plus-linter group was chosen noticeably more often, while the other two did not differ statistically: standards are hard to follow, and without feedback you never know whether you used them correctly. The model-generated variant was judged by around ten experts with some twenty years of experience each, and standards adherence is where it sagged worst. Pavel's team got a similar insight at their stand: a plain GPT with no fine-tuning produces valid but poor YAML, yet it works for teaching juniors, whose fear of the blank page is highest. Hence Alexander's thought: the model was evaluated standalone, whereas the internal assistant that already helps write code should have been fine-tuned instead. Pavel calls the measurement a speedrun: time to market for a method is 30 days, of which the YAML takes one, the rest going to clarifying business requirements and fighting the tooling.