How we searched for ecosystem effects: trial and error (Category Metrics)

I saw an interesting one. report Vladimir Abazov, Director of Non-Financial Services Analytics at T-Bank. In this report, Vladimir spoke about how the guys from the non-financial services of T-Bank checked whether they provide non-financial services. (In particular, shopping services) Significant contributions to classic banking products and customer behavior. Vova shares how they formulated the hypotheses, how and how they measured "ecosystem effects," what worked and what didn't. In general, the report is directly very interesting and practical, but the main ideas can be formulated in this way.

1. Ecosystem effects ные beliefThey should be measured and confirmed, not adjusted to a beautiful history. 2. Hypotheses → metrics → experimentsWithout a common layer of events/identifiers and correct attribution, the effects are lost. 3. Not everything - honest A/B testsWhen a “pure” experiment is not possible, the design of the experiment, control of seasonality/cannibalization, and sound skepticism about correlations are important. 4. Mistakes are part of the methodThe team systematically reassembled metrics and design until the signals became stable. (Hence the "Trial and Error" method.).

The report could be useful 1. EngineersTo understand what data and events are really needed for cross-service attribution; why unify user-ID/event schema; where the bottlenecks of experimental toggles and event tracking are. 2. Analytics/MLTo get a practical look at the measurability of ecosystem hypotheses, work with noise/shifts, and the choice of sustainable business metrics. 3. ManagersTo understand how to negotiate ownership of metrics between products in the ecosystem, how to build prioritization and not waste months on “effects” that don’t exist.

All in all, it’s a good report to synchronize the product, analytics, and platform and start speaking the same language.

#Data #Metrics #Software #Engineering #DS