[2/2] Monarch: Google's Planet-Scale In-Memory Time Series Database (Category Architecture)

But this history The monarch is not over. Already in 2023 ICSE-SEIP Confederation23 The guys at Google posted. whitepaper (what? handler). This paper can be shortened to the following points.

1. Reason: Doubling year-to-year growth of QPS and row numbers, problems with support and further hitting the SLO
2. The team decided to redesign based on quality-attribute scenarios + lightweight UML models.
3. In fact, Leaf was decomposed on Leaf. (KV storage)Leaf Index Server and Leaf Mixer.
4. This had a positive effect on availability and accompaniment, and latency grew moderately, although hops became more common. 2  (growth12–14 before 16–18 RPC)

But that wasn’t all – Monarch was the foundation of the cloud. Managed Service for PrometheusThis is a service on the same Monarch backend that Google monitors its services with. PromQL queries are partially computed on the Monarch side. For engineering teams, this gives a “global” Prometheus experience out of the box. By the way, the issue of scaling individual Prometheus arises in large companies often and to solve this issue, a project appeared. Thanos. You can still look at VIctoriaMetrics, this project also struggles with the limitations of Prometheus. (You can see mine. podcasting Where the author of the project talked about it)

If you learn from the history of Monarch, you should Collect metrics closer to the source, aggregate early (reduction of cardinality and cost) Cheap predicate index → cheaper distributed queries (Cut the fanout before execution) Strictly separate stateful and stateless parts (easier to accompany, easier to withstand SLO when growing) - Focus on this in the second whitepaper about the redesign

• Sharding the business key (target) Local aggregations/joins are cheaper.

#Software #Architecture #DistributedSystems #SRE #Engineering #Databases #Data