Publications on Google Research
For the past couple of days, I have been actively preparing for my RnD report and researching the page. research.google/pubs/Check out what white papers Google used and when. Finally, I compiled a list of key documents focusing on services and infrastructures. (except for ML)which he decided to share
- 2003 The Year: The Google File System Distributed file system from Google
- 2004 MapReduce: Simplified data processing on large clusters The concept of parallel processing in MapReduce format (It was based on Hadoop.)
- 2006 Bigtable: A Distributed Storage System for Structured Data Distributed NoSQL database (Based on BigTable and Amazon DynamoDB appeared Cassandra)
- 2006 - The Chubby lock service for loosely-coupled distributed systems About the service of distributed locks, which can be used instead of embedding the console in the services themselves
- 2007 - Engineering Reliability into Web Sites: Google SRE The role of SRE in ensuring reliability
- 2010 - Dapper, a Large-Scale Distributed Systems Tracing Infrastructure about tracing in distributed systems (open source followers of Zipkin, Jaeger, OpenTelemetry)
- 2012 - Spanner: Google's Globally-Distributed Database - about NewSQL database with scale of both NoSQL and ACID transactions, under the hood of TrueTime for accurate timing, which is necessary to determine the order of transactions (open source followers of Cockroach DB)
- 2013 - Omega: flexible, scalable schedulers for large compute clusters - about the workload crestorator (Borg heir, but less successful)
- 2015 - Large-scale cluster management at Google with Borg About the workload orchestrator that preceded Omega and ended up being more successful and outlived it
- 2015 - TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems About the framework for machine learning, which was immediately released in open source
- 2016 - Borg, Omega, and Kubernetes Comparison of two internal and one public (K8s) loader (Kubernetes originally made Google)
- 2016 - Ubiq: A Scalable and Fault-tolerant Log Processing Infrastructure - about processing logs on a scale
- 2017 - Spanner, TrueTime and the CAP Theorem About the CAP theorem and Spanner from the creator of the CAP theorem, Eric Brewer, who had already worked at Google for a long time.
- 2018 - Advantages and disadvantages of a monolithic repository: a case study at google Google Monorepository and how it helps them develop
- 2019 - Zanzibar: Google’s Consistent, Global Authorization System About ReBAC authorization system, which is tied to the relationship between entities (We already have this white paper. debate Code of Architecture)
- 2020 - Monarch: Google's Planet-Scale In-Memory Time Series Database Time-series database
- 2020 - Scaling PageRank to 100 Billion Pages About scaling the key algorithm on graphs (Page Rank) super-scale
- 2020 - Autopilot: Workload Autoscaling at Google Scale Autoscale workloads in the cloud
- 2022 - Deployment Archetypes for Cloud Applications Interesting research about types of deployments
- 2023 - A Model-based, Quality Attribute-guided Architecture Re-Design Process at Google - An interesting document about the architectural processes in Google on the example of the redesign of the Monarch system, about which there was a white paper 2020 year
If you summarize my thoughts about Google and how they created articles, you can see that they were the first to write about a lot of complex things, but they didn’t create open source solutions at first, and they had open source analogues. And these peers were incompatible with Google’s internal tools, making it difficult to get help from the community. Significant exceptions in terms of openness are: Android, Chrome, Kubernetes, TensorFlow.
#RnD #WhitePaper #Software #SoftwareDevelopment #Architecture #SoftwareArchitecture #DistributedSystems #SystemDesign #SystemEngineering