[2/2] Observability in an Asynchronous World • James Eastham • GOTO 2024 (Category Architecture)

The second part of the post about observability in asynchronous systems just speaks about observability, because in the first We only discussed the asynchrony:)

11. Asynchronous systems use standard approaches to observability: logs, metriics, traces. But these are tools to ensure observability.
12. And the author proposes to return to the question of why we need observability in our system.

The ability to ask questions of your system, you didn't know you'd need to ask from outside your system That is, if there are problems, we can figure it out.

13. Asynchronous systems are complex in terms of observability due to the many moving parts. Essential complexity of the domain remains in place, and accidental complexity in the asynchronous system is high.
14. To fight it, we need to use distributed tracing correctly. Tracing helps to understand the impact of events on the system.
15. Asynchronous architecture simplifies evolution by adding new event handlers, but you need to keep an eye on how the pattern of events evolves over time – this can lead to interesting problems. As an example, James cites a change to support paying for pizza in multiple currencies. That is, it is important to consider the impact of changes on all systems involved in event processing.
16. Next, the author talks about the specification of events and tells about the cloud events about event catalog. These approaches determine what to include in events, which helps with idempotence in processing and with tracing.
17. Next, the author talks about the approach API First Design And it goes on to say that events are also APIs in some way, so we need to treat them with respect. And use an event catalog to document event-driven systems.
18. In our observability system, it is often important to preserve not the content of events, but their pattern. This allows you to ask questions about the system and improve distributed tracing.
19. In general, observability and tracing are fun to see in the report.What Is This OpenTelemetry Thing?"that I've been talking about recently." told
20. Separately, the author talks about useful metrics by events: queue depth, number of published and processed messages, age of messages. The choice of metrics depends on the context and goals.
21. Observability platform allows you to find events, analyze their structure, track the spread of events and detect regressions.
22. I liked the author’s question about how to understand that this or that part of the code works on the market. In fact, through the same logs, tracks and metrics.

For those of you who want to see how it works in practice, James code The pizza system he was talking about. If you wish, you can learn how this works using Datadog as an observability platform.

#SRE #Architecture #DistributedSystems #Software