10 Learnings From Running Production Infrastructure at Google • Christof Leng • GOTO 2023

Good report. Lead for Google's SRE Engagement Model and SRE Review Programs. The report is well structured and can serve as some extract from Google’s SRE books that are available on their website. website And the ones I told you about earlier: SRE Book and Building Secure and Reliable Systems. Going back to the report itself, it consists of the following parts:

0. Culture It begins with the famous phrase “Culture eats strategy for breakfast.” But it's also an engineering problem that can be looked at in terms of affordable tuning, a methodology that engineers can use to improve reliability.
1. Reliability can't be taken for granted This is what I said at the beginning of my report.We design reliable systems - is the game worth the candle"but the point is to be mindful of the reliability and the risks if the system is not reliable enough - you can look at it from a risk perspective." Plus, it should be remembered that adding reliability after creating a system is impossible or too expensive, so reliability requirements should be discussed at the right time.
2. Cattle vs. Pets A popular metaphor for treating our systems not as pets, but as animals from the herd. This allows you to automate herd support activities and scale approaches, which is not possible with pets:)
3. Blamelessness A story about the right approach to charges. You can read more about this in Vestrum, who wrote about the typology of cultures.A typology of organisational cultures", as well as the flow of information in them and the attitude to errors -"The study of information flow: A personal journey"
4. Measure what matters - SLO story. (service level objectives) And that they really should be what's important to users of the service.
5. Failure modes A story about how participating in oncall incidents allows you to understand how systems fall apart, feel gunpowder and understand that a lot is at stake. This is not conveyed by reading post-mortems ... even by hearing and roles:)
6. No heroes A culture of heroes who save everyone is bad for everyone. We are not interested in overcoming problems, but in preventing them.
7. Automation - here we are talking about the automation of their work, which will allow you to do more complex tasks and less engage in routine manual labor - laziness is the engine of the process.
8. Change is No. 1 reason for outages Everything around is constantly changing and this is often the cause of interruptions in work. And you have to be prepared for that. It is necessary to minimize the risk of each individual change and automation of their verification and application. (For example, use GitOps.)
9. Outages are inevitable Problems are inevitable and must be prepared for them, limiting the impact, frequency and generally managing risk.
10. No haunted graveyards Do not start ghost cemeteries, which Chistof understands as important applications that no one touches and knows how they work, and therefore can not fix them in case of problems with them.

Finally, the author says that you should not make complex systems. Instead, try to make simple systems, although this is much more difficult than making complex systems.) The author extols simplicity and says that boring is beautiful:)

#Software #Engineering #Architecture #SoftwareArchitecture #SystemDesign #DistributedSystems #SRE #Reliability #Conference