Distributed System Patterns

1. Clock-Bound Wait (https://lnkd.in/gHjr74cb)

2. Consistent Core (https://lnkd.in/g4XcrJUD)

3. Emergent Leader (https://lnkd.in/gZdf_AWx)

4. Fixed Partitions (https://lnkd.in/gSJuYfCc)

5. Follower Reads (https://lnkd.in/gmUw5b7a)

6. Generation Clock (https://lnkd.in/g92gJSHS)

7. Gossip Dissemination (https://lnkd.in/gBdD77Fc)

8. HeartBeat (https://lnkd.in/g6RR9swi)

9. High-Water Mark (https://lnkd.in/g3Kx3gux)

10. Hybrid Clock (https://lnkd.in/gjPiB_GM)

11. Idempotent Receiver (https://lnkd.in/gVnNEAS9)

12. Key-Range Partitions (https://lnkd.in/g9k_W2DN)

13. Lamport Clock (https://lnkd.in/gn_Mjn5h)

14. Leader and Followers (https://lnkd.in/guv739mY)

15. Lease (https://lnkd.in/gX9cpWYA)

16. Low-Water Mark (https://lnkd.in/ggEnd765)

17. Paxos (https://lnkd.in/gzXpcqzG)

18. Quorum (https://lnkd.in/g_zM69XD)

19. Replicated Log (https://lnkd.in/g8dh64wZ)

20. Request Batch (https://lnkd.in/gutqpHJB)

21. Request Pipeline (https://lnkd.in/gVXknRem)

22. Request Waiting List (https://lnkd.in/ghB6HfdH)

23. Segmented Log (https://lnkd.in/gzvS48Za)

24. Single Socket Channel (https://lnkd.in/gdJapnNc)

25. Singular Update Queue (https://lnkd.in/gAnmUBvp)

26. State Watch (https://lnkd.in/g8EuZTPG)

27. Two Phase Commit (https://lnkd.in/g3Btwu3Z)

Updated list of engineering papers worth reading.

1. Google File System - https://lnkd.in/d2-wnyqZ
2. Map Reduce Big Data Algorithm - https://lnkd.in/dvE8-s8M
3. BigTable NoSQL Document Store - https://lnkd.in/drmvvSAK
4. Colossus Next Gen File Store - https://lnkd.in/dERKhwMf
5. Megastore Large Object Store - https://lnkd.in/d5JDs2-K
6. Monarch Time Series DB - https://lnkd.in/d3kH_NCp
7. Chubby Distributed Lock Management - https://lnkd.in/dYy-w5rW
8. Spanner Distributed Database - https://lnkd.in/d6Emnycp
9. Spanner - CAP theorem considerations - https://lnkd.in/dq29BAWQ
10. Dapper Tracing System - https://lnkd.in/dm36-6jn
11. Borg Cluster Management - https://lnkd.in/dnveV-HU
12. Zanzibar Authentication System - https://lnkd.in/d5Vf7sRD
13. Pregel Graph Processing - https://lnkd.in/daq4576Y
14. Napa - Data Warehousing - https://lnkd.in/dbEfsa5B
15. Napa - Partitioning Algorithm - https://lnkd.in/dkhA7efJ
16. TensorFlow - Machine Learning at Scale - https://lnkd.in/d-4NfV2Z
17. Google F1 - Fast Analytics - https://lnkd.in/dbZqEKuf
18. HALP - YouTube Content Delivery Network - https://lnkd.in/dHzJtUc7
19. Mesa - Data Warehousing - https://lnkd.in/dFJ_Jrz6
20. Google Firestore - https://lnkd.in/drtEN9qR
21. Amazon Aurora DB Architecture - https://lnkd.in/dcevpwFt
22. Dynamo DB NoSQL Database - https://lnkd.in/dMD8C_WK
23. Apple Foundation DB - NewSQL database - https://lnkd.in/dG75i_9K
24. TikTok Monolith - Embedding in real-time - https://lnkd.in/dcjBXCnc
25. Scalability at what COST - https://lnkd.in/dJ9ScYKq
26. Gorilla - Time Series DB - https://lnkd.in/d3AeN2kB
27. Cassandra - NoSQL DB - https://lnkd.in/d-_nhtED
28. FlexiRaft - Distributed Consensus Tradeoffs - https://lnkd.in/dX3nMvmt
29. Memcache - In-memory Cache at Facebook - https://lnkd.in/dKeYK67g
30. Millisampler Network Sampling - https://lnkd.in/dsj9FuD6
31. TAO Graph Database - https://lnkd.in/daasJpYf
32. MineSweeper - Root Cause Analysis - https://lnkd.in/dEsd6iwj
33. Facebook Prophet - Forecasting at Scale - https://lnkd.in/daCmAjak
34. Facebook ShardManager - https://lnkd.in/dDy9Dp2h
35. Hive - Map Reduce Jobs - https://lnkd.in/dpV8BM2R
36. Apache Thrift - Definition Language - https://lnkd.in/d7NzhP54
37. Meta Twine - Cluster Management System - https://lnkd.in/d5t7VFKE
38. Meta ServiceRouter - Service mesh - https://lnkd.in/dVnkv_bV
39. Apache Hadoop - Distributed File System - https://lnkd.in/dHsQu9FN
40. Apache Kafka - Event Bus - https://lnkd.in/dyxuKbMb
41. Apache Flink - https://lnkd.in/dn_gMvaR
42. Maglev: A Fast and Reliable Software Network Load Balancer - https://media.licdn.com/dms/document/media/D561FAQF_KIvtA8POhQ/feedshare-document-pdf-analyzed/0/1702277625362?e=1703116800&v=beta&t=ovfz-5wP_ywIUzwd4BmZJhg_4UTZEaGsEDxSZAlweSg

20 Excellent Database Engineering Blogs, grouped by topic:

𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞 𝐑𝐞𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧

1. MySQL replication: https://lnkd.in/dcrX972v
2. Netflix Change Data Capture: https://lnkd.in/dWjkCqav
3. LinkedIn Logging Use cases: https://lnkd.in/dhwgSUpZ

𝐈𝐧 𝐌𝐞𝐦𝐨𝐫𝐲 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞 - 𝐑𝐞𝐝𝐢𝐬

4. Redis Official Documentation: https://redis.com/
5. Learn Redis through Redis University: https://lnkd.in/dd_6Tj2J
6. Redis Open Source Repo: https://lnkd.in/dUv3Z-Nx
7. Redis Architecture: https://lnkd.in/dCUEjbsC

𝐍𝐨𝐒𝐐𝐋 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬

8. Cassandra Architecture: https://lnkd.in/d8grN7JB
9. Google BigTable Architecture: https://lnkd.in/drmvvSAK
10. Amazon Dynamo DB Internals: https://lnkd.in/d_TajpEk
11. Design Patterns in Amazon Dynamo DB: https://lnkd.in/dDWpMDp5
12. Internals of Amazon Dynamo DB: https://lnkd.in/d7jmg8Wh
13. Hyperloglog Algorithm: https://lnkd.in/dEXxPCqM
14. Log Structured Merge Trees: https://lnkd.in/dUvEqSTb
15. Sorted String Tables and Compaction Strategies: https://lnkd.in/diNcN3a3
16. Leveled Compaction Cassandra: https://lnkd.in/dRQc6zPR
17. Scylla DB Compaction: https://lnkd.in/diNcN3a3
18. Indexing in Cassandra: https://lnkd.in/dAbVcauy

𝐓𝐢𝐦𝐞 𝐒𝐞𝐫𝐢𝐞𝐬 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬

19. Pinterest Time Series Database: https://lnkd.in/dRVqxHDS
20. Uber Time Series DB: https://lnkd.in/dxXqMEEv
21. TimeSeries Relational DB: https://lnkd.in/dTf9Fmjc
22. Facebook Gorilla Time Series DB: https://lnkd.in/dk5BfZ7N

𝐓𝐞𝐱𝐭 𝐒𝐞𝐚𝐫𝐜𝐡 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬

23. Facebook real-time text search engine: https://lnkd.in/dRtbQ5a3
24. Elastic Search Time-Based Querying: https://lnkd.in/d_QvUr7Y
25. Elastic Search Aggregation: https://lnkd.in/dHVzGejb

Discord

26. how-discord-stores-trillions-of-messages - https://discord.com/blog/how-discord-stores-trillions-of-messages

system design, learn these case studies:

1. URL shortener or TinyURL: https://lnkd.in/gJPWZsX3

2. Social media app like Twitter: https://lnkd.in/guParNhY

3. File sharing service like Dropbox: https://lnkd.in/gnSQ6rAd

4. Video streaming service like Netflix: https://lnkd.in/gnHSyCj8

5. Autocomplete feature of Google: https://lnkd.in/g3VFcaYN

6. Messaging app like WhatsApp: https://lnkd.in/g5zJa_4W

7. Online document editor like Google docs: https://lnkd.in/gBvztEqV

8. Proximity service like Uber: https://lnkd.in/grtPbr6k

9. Hotel booking service like Airbnb: https://lnkd.in/gWjrGwCM

10. Web crawler: https://lnkd.in/gTkfCH8U

11. Gaming Leaderboard https://lnkd.in/edfTDq5R

12. Real-Time Live Comments https://lnkd.in/e8g3dZ2i

13. How Uber Computes ETA https://lnkd.in/eVKV2ePC

14. URL Shortener https://lnkd.in/evFTZVQq

15. Slack Architecture https://lnkd.in/eATMDjrK

16. Zoom Architecture https://lnkd.in/edidhxZw

17. Distributed Counter https://lnkd.in/eGwaA62J

18. Virtual Waiting Room Architecture https://lnkd.in/eD3kPiPv

19. Pastebin https://lnkd.in/eZpfaVjc

20. User Online Status Indicator https://lnkd.in/eW77FXAh

21. How Does Netflix Work? https://lnkd.in/ehqyWMiW

35 most important System Design Concepts in one post:

1. Content Delivery Network (CDN): https://lnkd.in/gjJrEJeH

2. Caching: https://lnkd.in/gC9piQbJ

3. Distributed Caching: https://lnkd.in/g7WKydNg

4. Latency vs Throughput: https://lnkd.in/g_amhAtN

5. CAP Theorem: https://lnkd.in/g3hmVamx

6. Load Balancing: https://lnkd.in/gQaa8sXK

7. ACID Transactions: https://lnkd.in/gMe2JqaF

8. SQL vs NoSQL: https://lnkd.in/g3WC_yxn

9. Consistent Hashing: https://lnkd.in/gd3eAQKA

10. Database Index: https://lnkd.in/gCeshYVt

11. Rate Limiting: https://lnkd.in/gWsTDR3m

12. Microservices Architecture: https://lnkd.in/gFXUrz_T

13. Strong vs Eventual Consistency: https://lnkd.in/gJ-uXQXZ

14. REST vs RPC: https://lnkd.in/gN__zcAB

15. Batch Processing vs. Stream Processing: https://lnkd.in/g4_MzM4s

16. HeartBeat: https://lnkd.in/g4x7sMrF

17. Circuit Breaker: https://lnkd.in/gCxyFzKm

18. Idempotency: https://lnkd.in/gPm6EtKJ

19. Database Scaling: https://lnkd.in/gAXpSyWQ

20. Data Replication: https://lnkd.in/gVAJxTpS

21. Data Redundancy: https://lnkd.in/gNN7TF7n

22. Database Sharding: https://lnkd.in/gMqqc6x9

23. Proxy Server: https://lnkd.in/gi8KnKS6

24. Domain Name System (DNS): https://lnkd.in/gkMcZW8V

25. Message Queues: https://lnkd.in/gTzY6uk8

26. WebSockets: https://lnkd.in/g76Gv2KQ

27. Consensus Algorithms: https://lnkd.in/ggc3tFbr

28. Fault Tolerance: https://lnkd.in/dVJ6n3wA

29. Failover: https://lnkd.in/dihZ-cEG

30. Gossip Protocol: https://lnkd.in/gfPMtrJZ

31. API Gateway: https://lnkd.in/gnsJGJaM

32. Distributed Locking: https://lnkd.in/gRxNJwWE

33. Distributed Tracing: https://lnkd.in/d6r5RdXG

34. Bloom Filters: https://lnkd.in/dt4QbSUz

35. Checksum: https://lnkd.in/ghNc5pfn