Managing Effectiveness in Tech and Product

The host opens with a show of hands: who in the last six months has been asked by their managers about the effectiveness of their processes, or wanted to do something about it themselves; the rest, he jokes, came for the food, and Avito does have food. Yaroslav Tulupov, engineering director of Avito Pro — monetisation, advertising, seller experience, listing goods on the platform and fintech — describes effectiveness as the problem of a heavily distributed structure of verticals and horizontals with a decent shortage of everything, people and hardware alike. His horizontal builds solutions for every vertical, from cars to property; the business runs in with a brilliant new idea and no money for it, and the question is how to squeeze the unsqueezable into the resource well and what to stop in order to start something new. Part of the answer is technology against routine: an internal PaaS where a developer launches a service with one button.

Vlad Plyushchev runs technology for Sber’s eighteen-month-old b2c structure, which builds the services retail customers use. His formula is "do the right things right". The right things start with strategy and prioritisation: if a product does not fit the cost-income norms, dropping it at the gate is cheaper than later picking the process, the tool and the team. The host objects that the best prioritisers are companies with very little money — when you can afford three features you think ten times. Vlad agrees that efficiency is never a hundred per cent and adds a constraint: Sber has a mandatory agenda, products it cannot abandon even when the numbers are poor, and the point is to take that decision consciously. In twelve years there he has concluded that any seed you sow will come up: an audience always appears, but separating a project’s effect from the wider national context is nearly impossible.

Ilya Losev runs the Yandex Tasks service — what remained of Toloka after it left Russia: where machine logic gives up, people are plugged into business processes. His unit keeps both its own internal PnL and the PnL of the product sold outside. Resources inside Yandex are harder to come by than at Sber, where he worked before: "you put together a presentation, came, told the story, got half the resources" — the room laughs at that. And the resource granted is only the right to hire: a new hire lands in your PnL with taxes and the mandatory costs the organisation imposes, such as biscuits at the coffee points, so staff cost about the same as a contractor. The service has roughly as many performers as Yandex has employees, and it tracks their LTV and the cost of hiring. In IT they have built three metrics over the past year: useful capacity, time to market, and backlog convergence close to a hundred per cent. Asked about outsourcing from the floor, he answers briefly: Yandex barely uses it, with too much know-how inside.

Alexander, a CTO at T-Bank responsible for customer interfaces, marketing and engagement, starts simply — earn more, spend less — and immediately breaks that down by type of structure. Business lines such as debit, credit and investment products have their own PnL, and there everything adds up. For service lines such as customer acquisition, effectiveness shows up indirectly: the cost of acquiring a user against the product’s NPV model. For platforms the investment is large and there are no alternatives, so the economics are built from the question of what it would cost to buy the platform from outside: back in 2019, he recalls, some twenty to twenty-five per cent of product teams were building their own tooling for code, CI/CD and monitoring. The second half is "doing it right": a recent Google paper on developer productivity measures engineering effectiveness from an engineer’s daily scenarios and tunes the processes of design, coding, quality, deployment and operations.

The host throws in a hypothesis: any assessment system at team level will be gamed. Alexander recalls Goodhart’s law — when a metric becomes a target, a good metric turns bad: while decisions are made on it everything is fine, but the moment it is used to judge people’s work they start painting the traffic lights green. What helps is a system of metrics that is hard to paint all over: you paint one face of the Rubik’s cube and the others fall apart. So does an Amazon-style deep dive: if a green report is enough for a top manager, that is what he will be brought; with about a thousand engineers in his area he goes and asks questions himself. Yaroslav adds the physical meaning: Avito looks not only at service nines but at the stability of critical scenarios — could the user actually pay. His payments team’s scenarios were green while he knew that was untrue. Vlad offers the image of a watermelon metric: green outside, cut it open and it is red inside. Ilya calls development metrics the topic he hates most: lines of code and commit frequency are gamed with ease.

The second big theme is the question chief executives keep putting to their CTOs: ten people are doing the job and five would be nicer. Ilya suggests not sliding into headcount: the same budget buys two seniors or roughly five juniors, and the questions come more often when the CEO is poorly immersed; then they agree a grace period during which they either lower the cost and watch which business processes it hits, or restructure the team. Alexander splits the work into change and run: if a product is only being operated, fewer people may do, but development stalls, and cutting means giving something up. Vlad reduces it all to trust between two management levels and to driver models, which removed a mass of internal arguments: if the driver grows you get headcount as part of the process. Yaroslav names two levers: move operational processes out of the engineering function, and invest in the platform so the next slice of business costs close to nothing to build. The closing advice is short: read the context, think, and learn to say no to yourself.