[2/3] System Design for Interviews and Beyond (Category Architecture)

I continue the story about the cool course with Leetcode, in which it was accessible and clear about what a system design interview is and what it is designed to check with candidates. In previous post We discussed work with requirements, important architectural characteristics, infrastructure, caching and queues. We will continue to talk about data storage, interaction of system components.

6. Data store internals. Here the author briefly discusses the topic of data storage: Log is the easiest way to save data, but it is difficult to read it in this form. (full scan for any request) Index – Indices as a way to prepare data for effective reading queries Time series data - a separate type of data that is useful, for example, in monitoring Simple key/value database - the author explains how the database will work with the simplest data model B-Tree index - the author tells about the ubiquitous b-tree indexes, how they are arranged and for which scenarios are optimally suited Embedded databases – sometimes it is convenient to embed the database directly into the application process, for example, LevelDB, RocksDB, DuckDB can do this.

• RocksDB - the author tells how this database works and here we are talking about memtable, write-ahead log and SSTables Comparison of LSM-tree and B-Tree - the author shows the compromises of each approach and compares their limits of applicability Page cache - ends with a story about how to land all this on the file system inside the OS. Without this knowledge, much of the above will not work well.

7. How to build efficient communication in distributed systems. In this part, the author speaks about the classics of communication. Push vs pull interaction models What is host and service discovery? (There's DNS.)and peer discovery How to choose a network protocol for the task (UDP, TCP, HTTP) and how they behave in practice How video streams are usually transmitted and what is a CDN (content delivery network) What are short pooling, long pooling, web-socket, server-sent events, why they are needed and how they behave in practice? At the end of the section, the author shows how fluffs can work for customers on a large scale. (Netflix)

8. How to delivery data reliably. This section begins with a known list. fallacies of distributed systemsThe author then proceeds to practical means of ensuring reliability. Timeouts and strategies for unsuccessful requests: cancel, retry, failover, fallback On specific examples, the author analyzes when and how to make retries Next comes the time to discuss the guarantee of delivery of messages: at most once, at least once, exactly once Next, the author examines how log-based message queues work (kafka) What is consumer offset?

9. How to deliver data quickly Here, the author talks about approaches to batching and data compression, which provides better bandwidth.

10. How to deliver data at large scale In this part, it is time to discuss the issues of scaling data processing. The author talks about

• Partisanship (sharding) Lookup strategy, range strategy, hash strategy How partisanship works in the real world and what pros/cons it has
• What routing requests look like
• What to do with rebalancing the shards
• What is consistent hashing?

Continuation of the review will be in next post.

#Software #Architecture #DistributedSystems #SystemDesign #Engineering