├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Developer-Y 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Scalable-Software-Architecture 2 | 3 | Collection of tech talks, papers and web links on Distributed Systems, Scalability and System Design. 4 | 5 | ## Tech Talks 6 | 7 | #### General Advice on System Design and Scalability 8 | * [Lecture - Scalability - Harvard Web Development, David Malan](https://www.youtube.com/watch?v=-W9F__D3oY4) 9 | * [Building Software Systems At Google and Lessons Learned](https://www.youtube.com/watch?v=modXC5IWTJI) 10 | * [Scalable Internet Architectures - Theo Schlossnagle](https://www.youtube.com/watch?v=2WuT2rdLK5A) 11 | * [Seattle Conference on Scalability - Jeff Dean](https://www.youtube.com/watch?v=YFGFvCCFk0g) 12 | * [Best Practices for Scaling Web Apps](https://www.youtube.com/watch?v=tQ2V9QSv48M) 13 | * [Building a Scalable Architecture for Web Apps](https://www.youtube.com/watch?v=yeyugGZO2qc) 14 | * [Web application architecture: The whole stack - Allen Holub](https://www.youtube.com/watch?v=SgEFNRf67Cw) 15 | * [Scalable Distributed Design](https://www.youtube.com/watch?v=E9GFA-YkzFY) 16 | * [Building Software at Google Scale Tech Talk](https://www.youtube.com/watch?v=2qv3fcXW1mg) 17 | * [Seattle Conference on Scalability: Scaling Google for Every User](https://www.youtube.com/watch?v=LT1UFZSbcxE) 18 | * [Seattle Conference on Scalability: Lessons In Building Scalable Systems](https://www.youtube.com/watch?v=mS48X9oEar0) 19 | * [Workers, Queues, and Cache](https://www.infoq.com/presentations/workers-queues-cache) 20 | * [Velocity 2012: Jay Parikh, "Building for a Billion Users"](https://www.youtube.com/watch?v=oodS71YtkGU) 21 | * [Building Large Systems at Google](https://www.youtube.com/watch?v=_cltA0W-ehQ) 22 | * [3000 images per second - Henna Kerman - @Scale 2016](https://www.youtube.com/watch?v=CEZ6oO1u1u8) 23 | * [Scaling to over 1,000,000 requests per second](https://www.youtube.com/watch?v=v-KfhCGW3MA) 24 | * [Jeff Dean: "Achieving Rapid Response Times in Large Online Services" Keynote - Velocity 2014](https://www.youtube.com/watch?v=1-3Ahy7Fxsc) 25 | * [Getting Things Done at Scale](https://www.infoq.com/presentations/cultural-change-management) 26 | * [Scale-oriented Architecture with APIs](https://www.infoq.com/presentations/queues-proxy-microservices-automation) 27 | * [You Won't Believe How the Biggest Sites Build Scalable and Resilient Systems!](https://www.infoq.com/presentations/scalable-resilient-systems) 28 | * [Scalable Distributed Design](https://www.youtube.com/watch?v=E9GFA-YkzFY) 29 | * [Distributed Patterns you should know by Eric Redmond](https://www.youtube.com/watch?v=otWvr8VBRx0) 30 | * [GOTO 2012 • Runaway Complexity in Big Data Systems...and a Plan to Stop it • Nathan Marz](https://www.youtube.com/watch?v=ucHjyb6jv08) 31 | * [Seattle Conference on Scalability: Abstractions for handling large datasets](https://www.youtube.com/watch?v=oRwFpQKgRps) 32 | * [High Performance Web Sites and YSlow](https://www.youtube.com/watch?v=BTHvs3V8DBA) 33 | 34 | 35 | #### Company/Product specific tech talks 36 | * [Seattle Conference on Scalability: YouTube Scalability](https://www.youtube.com/watch?v=w5WVu624fY8) 37 | * [How to answer design question: How do you design a twitter?](https://www.youtube.com/watch?v=gX8S7b8UYl8) 38 | * [Operations at Twitter: Scaling Beyond 100 Million Users](https://www.youtube.com/watch?v=z8LU0Cj6BOU) 39 | * [How We've Scaled Dropbox](https://www.youtube.com/watch?v=PE4gwstWhmc) 40 | * [Keynote: Twitter's search architecture](https://www.youtube.com/watch?v=AguWva8P_DI) 41 | * [Marco Cecconi - "The Architecture of StackOverflow"](https://www.youtube.com/watch?v=t6kM2EM6so4) 42 | * [Scalability at YouTube](https://www.youtube.com/watch?v=G-lGCC4KKok) 43 | * [Lessons of Scale at Facebook](https://www.youtube.com/watch?v=QCHiNEw73AU) 44 | * [Scale at Facebook](https://www.infoq.com/presentations/Scale-at-Facebook) 45 | * [Flight Lightning - Scaling Twitter core infrastructure](https://www.youtube.com/watch?v=6OvrFkLSoZ0) 46 | * [Scaling Instagram with Mike Krieger](https://www.youtube.com/watch?v=oNA2C1vC8FQ) 47 | * [GOTO 2014 • Scaling Pinterest • Marty Weiner](https://www.youtube.com/watch?v=jQNCuD_hxdQ) ([InfoQ link](https://www.infoq.com/presentations/Pinterest)) 48 | * [Real-Time Delivery Architecture at Twitter](https://www.youtube.com/watch?v=J5auCY4ajK8) 49 | * [OSCON 2014: How Instagram.com Works; Pete Hunt](https://www.youtube.com/watch?v=VkTCL6Nqm6Y) 50 | * [Scaling the Data Infrastructure at Instagram](https://www.youtube.com/watch?v=1sPgogJlKWM) 51 | * [O'Reilly MySQL CE 2011: Jeremy Cole, "Big and Small Data at @Twitter"](https://www.youtube.com/watch?v=5cKTP36HVgI) 52 | * [O'Reilly Webcast: How Pinterest Architected and Built Their Sharded MySQL Datastore](https://www.youtube.com/watch?v=aFJm1YlRr5Q) 53 | * [Timelines at Scale (Twitter)](https://www.infoq.com/presentations/Twitter-Timeline-Scalability) 54 | * [Architecture at Scale at ESPN](https://www.infoq.com/presentations/Architecture-Scale-ESPN) 55 | * [Building Highly-resilient Systems at Pinterest](https://www.infoq.com/presentations/pinterest-resilient-systems) 56 | * [Scaling Uber](https://www.infoq.com/presentations/uber-scalability-arch) 57 | * [How Zoom works](http://highscalability.com/blog/2020/5/14/a-short-on-how-zoom-works.html) 58 | * [Scaling Engineering Culture at Twitter](https://www.youtube.com/watch?v=H9wTuT6yQlc) 59 | * [Keynote - Systems at Facebook Scale](https://www.youtube.com/watch?v=dlixGkelP9U) 60 | * [Scaling YouTube's Backend: The Vitess Trade-offs](https://www.youtube.com/watch?v=5yDO-tmIoXY) 61 | * [Hacker Way: Rethinking Web App Development at Facebook](https://www.youtube.com/watch?v=nYkdrAPrdcw) 62 | * [Data Platform Architecture, Evolution, and Philosophy at Netflix](https://www.youtube.com/watch?v=YT-ztW5QNUQ) 63 | * [Data Platform Architecture, Evolution, and Philosophy at Netflix](https://www.youtube.com/watch?v=uH8T7JMzloM) 64 | * [Structure, Personalization, Scale: A Deep Dive into LinkedIn Search](https://www.infoq.com/presentations/linkedin-search) 65 | * [Scaling Foursquare: From Check-ins to Recommendations](https://www.infoq.com/presentations/scale-foursquare) 66 | * [How Netflix Leverages Multiple Regions to Increase Availability: An Active-Active Case Study](https://www.infoq.com/presentations/netflix-availability-regions) 67 | * [That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp](https://www.infoq.com/presentations/whatsapp-scalability) 68 | * [How Facebook Scales Big Data Systems](https://www.infoq.com/presentations/scale-facebook-big-data) 69 | * [Scaling Uber's Real-time Market Platform](https://www.infoq.com/presentations/uber-market-platform) 70 | * [Software Development & Architecture @ LinkedIn](https://www.infoq.com/presentations/linkedin-architecture-practices) 71 | * [Etsy Search: How We Index and Query 26 Million One-of-a-kind Items](https://www.infoq.com/presentations/etsy-search-ecosystem) 72 | * [Scalability Lessons from eBay, Google, and Real-time Games](https://www.infoq.com/presentations/scalability-ebay-google-kixeye) 73 | * [How SoundCloud Uses Cassandra](https://www.infoq.com/presentations/soundcloud-cassandra) 74 | * [Service Architectures at Scale: Lessons from Google and eBay](https://www.infoq.com/presentations/service-arch-scale-google-ebay) 75 | * [Solidifying the Cloud: How Google Backs up the Internet](https://www.infoq.com/presentations/google-cloud-backup-failure) 76 | * [Real-Time Systems at Twitter](https://www.infoq.com/presentations/real-time-twitter) 77 | * [Serving user intent : Facebook style notifications using HBase and Event streams](https://www.youtube.com/watch?v=hIxXaH0CrVQ) 78 | * [Netflix's Distributed Computing Strategies: Optimistic Design for the Eventual Consistency Model](https://www.youtube.com/watch?v=6R1WhWkh6pg) 79 | 80 | #### Distributed Computing 81 | * [Intro to Hadoop and MapReduce (Udacity)](https://www.youtube.com/playlist?list=PLAwxTw4SYaPkXJ6LAV96gH8yxIfGaN3H-) 82 | * [Introduction to Hadoop](https://www.youtube.com/watch?v=Pq3OyQO-l3E) 83 | * [MapReduce Flow Chart](https://www.youtube.com/watch?v=6OemZEJdMp8) 84 | * [Distributed Computing CS 61A UC Berkeley](https://www.youtube.com/watch?v=zGTZ-wwA9O8&list=PL6BsET-8jgYWkCySVpSbOj-MCk6xC0i4N) 85 | * [MapReduce CS 61A UC Berkeley](https://www.youtube.com/watch?v=6mABLxxtIlg&list=PL6BsET-8jgYVivQ1AS4UsPR2tyr830OHj) 86 | * [Cluster Computing and MapReduce Lecture 1](https://www.youtube.com/watch?v=yjPBkvYh-ss) 87 | * [Cluster Computing and MapReduce Lecture 2](https://www.youtube.com/watch?v=-vD6PUdf3Js) 88 | * [Introducing Apache Hadoop: The Modern Data Operating System](https://www.youtube.com/watch?v=d2xeNpfzsYI) 89 | 90 | #### Distributed Database/Large-Scale Storage 91 | * [Spanner: Google’s Globally-Distributed Database](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett) ([Youtube link](https://www.youtube.com/watch?v=NthK17nbpYs)) 92 | * [Spanner - multi-version, globally- distributed, and synchronously-replicated database](https://www.youtube.com/watch?v=Q7pcMn0_tac) 93 | * [BigTable: A Distributed Structured Storage System](https://www.youtube.com/watch?v=2cXBNQClehA) ([slides](http://static.googleusercontent.com/media/research.google.com/en//people/jeff/bigtable-uw-2005.pdf)) 94 | * [Large-Scale Low-Latency Storage for the Social Network - Data@Scale](https://www.youtube.com/watch?v=5RfFhMwRAic) 95 | * [Structured Data at Box: How We're Building for Scale](https://www.youtube.com/watch?v=KHrw0Xo8tYo) 96 | * [F4 - Photo Storage at Facebook](https://www.youtube.com/watch?v=34e_g-Ji_30) 97 | * [The Storage Technologies Behind Facebook Messages](https://www.youtube.com/watch?v=cSNGGAKJqwk) 98 | * [Cold Storage at Facebook](https://www.youtube.com/watch?v=FGZqToLuwU0) 99 | * [Taking Storage for a Ride with Uber](https://www.youtube.com/watch?v=Dg76cNaeB4s) 100 | * [Zen: Pinterest's Graph Storage Service - @Scale 2014 - Data](https://www.youtube.com/watch?v=yI0vHfgK6oI) ([With Slides](https://www.infoq.com/presentations/zen-pinterest-graph-storage-service)) 101 | * [Storage Systems at a Rapidly Scaling Startup with a Small Team - Data@Scale](https://www.youtube.com/watch?v=bLyv8zKa5DU) 102 | * [f4: Facebook's Warm BLOB Storage System](https://www.youtube.com/watch?v=imlP3lxwGgQ) 103 | 104 | #### Distributed graph processing 105 | * [Giraph](https://www.youtube.com/watch?v=xyvs7NQdfZk) 106 | * [Apache Giraph Large Scale Graph Processing On Hadoop](https://www.youtube.com/watch?v=uXw2TKHsqvM) 107 | * [Processing Over a Billion Edges on Apache Giraph](https://www.youtube.com/watch?v=b5Qmz4zPj-M) 108 | * [Graph Search: The Power of Connected Data](https://www.infoq.com/presentations/graph-search-connected-data) 109 | * [Using Graph Partitioning in Distributed Systems Design](https://www.youtube.com/watch?v=QHkhyY9atkE) 110 | * [Let Me Graph That For You: Building a Graph Database Application](https://www.infoq.com/presentations/graph-db-tools) 111 | * [GraphChi: Large-Scale Graph Computation on Just a PC](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/kyrola) 112 | * [PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/gonzalez) 113 | 114 | #### Stream Processing 115 | * [Heron: Real-time Stream Data Processing at Twitter](https://www.youtube.com/watch?v=pUaFOuGgmco) 116 | * [Samza in LinkedIn: How LinkedIn Processes Billions of Events Everyday in Real-time](https://www.infoq.com/presentations/samza-linkedin-2014) 117 | * [Mantis: Netflix's Event Stream Processing System](https://www.infoq.com/presentations/mantis) 118 | * [High Throughput Stream Processing with ACID Guarantees](https://www.infoq.com/presentations/acid-stream-processing) 119 | * [Martin Kleppmann — Event Sourcing and Stream Processing at Scale](https://www.youtube.com/watch?v=avi-TZI9t2I) 120 | * [ETE 2012 - Nathan Marz on Storm](https://www.youtube.com/watch?v=bdps8tE0gYo) 121 | * [Cassandra NYC 2011: Nathan Marz - The Storm and Cassandra Realtime Computation Stack](https://www.youtube.com/watch?v=cF8a_FZwULI) 122 | 123 | #### API Design 124 | * [How To Design A Good API and Why it Matters](https://www.youtube.com/watch?v=aAb7hSCtvGw) 125 | * [How to Design Great APIs - Parse Developer Day 2013](https://www.youtube.com/watch?v=qCdpTji8nxo) 126 | * [Google I/O 2010 - How Google builds APIs](https://www.youtube.com/watch?v=nyu5ZxGUfgs) 127 | * [Designing a Beautiful REST+JSON API](https://www.youtube.com/watch?v=5WXYw4J4QOU) 128 | 129 | #### Web Services and SOA 130 | * [Introduction to Service Design and Engineering - University of Trento, Italy](https://www.youtube.com/playlist?list=PLBdajHWwi0JCn87QuFT3e58mekU0-6WUT) 131 | * [REST+JSON API Design - Best Practices for Developers](https://www.youtube.com/watch?v=hdSrT4yjS1g) 132 | * [What is a Service Oriented Architecture?](https://www.youtube.com/watch?v=nRayJZmj2oY) 133 | * [Webinar : Practical SOA for the Solution Architect](https://www.youtube.com/watch?v=1KXKppaOgtY) 134 | 135 | #### Caching 136 | * [Scaling Redis at Twitter](https://www.youtube.com/watch?v=rP9EKvWt0zo) 137 | * [Facebook and memcached - Tech Talk](https://www.youtube.com/watch?v=UH7wkvcf0ys) 138 | * [Scaling Memcache at Facebook](https://www.youtube.com/watch?v=6phA3IAcEJ8) 139 | * [How Netflix and reddit scale to handle massive demand](https://www.youtube.com/watch?v=zVqA7Ogutiw) 140 | * [An analysis of Facebook photo caching](https://www.youtube.com/watch?v=ENaQScyvOzY) 141 | 142 | #### NoSQL 143 | * [Introduction to NoSQL • Martin Fowler](https://www.youtube.com/watch?v=qI_g07C_Q5I) 144 | * [NoSQL Distilled to an hour by Martin Fowler](https://www.youtube.com/watch?v=ASiU89Gl0F0) 145 | * [NoSQL Distilled • Pramod Sadalage](https://www.youtube.com/watch?v=l68zxgZS_wc) 146 | * [Tech Talk: Cassandra Data Modeling](https://www.youtube.com/watch?v=tg6eIht-00M) 147 | * [NoSQL Explained](https://www.youtube.com/watch?v=Q6gn2kuOTV0) 148 | * [Big Data Architecture Patterns](https://www.youtube.com/watch?v=-N9i-YXoQBE) 149 | * [Graph Databases Exposed](https://www.youtube.com/watch?v=A44CQ4P7oWw) 150 | 151 | #### Messaging 152 | * [Queue It! What Job Queues Can Do for You!](https://www.infoq.com/presentations/job-queue-gearman-beanstalkd) 153 | * [Joydeep Sen Sarma - Messaging architecture at Facebook](https://www.youtube.com/watch?v=5hUmdoMOrpo) 154 | * [Messaging at Scale at Instagram](https://www.youtube.com/watch?v=E708csv4XgY) 155 | * [Building a Distributed Data Ingestion System with RabbitMQ](https://www.infoq.com/presentations/data-replication-rabbitmq) 156 | * [scaling web applications with message queues - Lenz Gschwendtner](https://www.youtube.com/watch?v=aOrGq9yb6og) 157 | 158 | #### Object Oriented Analysis and Design 159 | * [Software Architecture & Design | Udacity](https://www.youtube.com/playlist?list=PLAwxTw4SYaPkMTetlG7xKWaI5ZAZFX8fL) 160 | * [OOSE: Software Dev Using UML and Java](https://www.youtube.com/playlist?list=PLJ9pm_Rc9HesnkwKlal_buSIHA-jTZMpO) 161 | * [Eric Evans — Tackling Complexity in the Heart of Software](https://www.youtube.com/watch?v=dnUFEg68ESM) 162 | * [Domain Driven Design](https://www.youtube.com/watch?v=BPAZwFNxgrg) 163 | * [Domain-Driven Design](https://www.youtube.com/watch?v=RNUn2R7TptM) 164 | * [How You Can Architect and Develop Enterprise Mission-Critical Applications with Domain-Driven Design]( https://www.youtube.com/watch?v=aieoAWXNjl0) 165 | * [DDD: putting the model to work](https://www.infoq.com/presentations/model-to-work-evans) 166 | * [Eric Evans on DDD: Strategic Design](https://www.infoq.com/presentations/strategic-design-evans) 167 | * [Architecting and Implementing Domain-Driven Design Patterns with Microsoft .NET](https://www.youtube.com/watch?v=DaQCGy_GIPE) 168 | * [SOLID Design Patterns in C#](https://www.youtube.com/playlist?list=PL8m4NUhTQU48oiGCSgCP1FiJEcg_xJzyQ) 169 | * [Object Oriented Design](https://www.youtube.com/playlist?list=PLGLfVvz_LVvS5P7khyR4xDp7T9lCk9PgE) 170 | * [Design Patterns Video Tutorial](https://www.youtube.com/playlist?list=PLF206E906175C7E07) 171 | * [Object Oriented Design Interview Question: Design a Car Parking Lot.](https://www.youtube.com/watch?v=2vtT6TBnOAM) 172 | * [Google's Clean Code Talks](https://www.youtube.com/playlist?list=PL693EFD059797C21E) 173 | * [NYC Tech Talk Series: How Google Backs Up the Internet](https://www.youtube.com/watch?v=eNliOm9NtCM) 174 | * [Robert C Martin(Uncle Bob) -Clean Architecture and Design-2012](https://www.youtube.com/watch?v=asLUTiJJqdE) 175 | * [Robert C Martin - Clean Architecture and Design](https://www.youtube.com/watch?v=Nsjsiz2A9mg) 176 | * [Robert C Martin - The Single Responsibility Principle](https://www.youtube.com/watch?v=Gt0M_OHKhQE) 177 | * [Robert C Martin - Clean Architecture](https://www.youtube.com/watch?v=Nltqi7ODZTM) 178 | * [The S.O.L.I.D. Principles of OO and Agile Design - by Uncle Bob Martin](https://www.youtube.com/watch?v=t86v3N4OshQ) 179 | * [Solid Principles by Uncle Bob Martin](https://www.youtube.com/watch?v=oar-T2KovwE) 180 | * [The Principles of Clean Architecture by Uncle Bob Martin](https://www.youtube.com/watch?v=o_TH-Y78tt4) 181 | * [Unleash Your Domain - Greg Young](https://vimeo.com/19428577) 182 | 183 | #### Misc 184 | * [Differential Synchronization](https://www.youtube.com/watch?v=S2Hp_1jqpY8) 185 | * [Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303) | AWS re:Invent 2013](https://www.youtube.com/watch?v=LpNbjXFPyZ0) 186 | * [Deepak Agarwal: Recommender Systems - The Art and Science of Matching Items to Users](https://www.youtube.com/watch?v=bRzOBGLCRbc) 187 | * [Lecture 12 -Analyzing Big Data with Twitter: Recommender Systems by Alpa Jain](https://www.youtube.com/watch?v=NSscbT7JwxY) 188 | * [Transactions across Datacenters](https://www.youtube.com/watch?v=srOgpXECblk) ([Slides](http://snarfed.org/transactions_across_datacenters_io.html)) 189 | * [Finding the Needle in the Haystack - or - Troubleshooting Distributed Systems](https://www.infoq.com/presentations/mondemand-troubleshoot-distributed-system) 190 | * [Finding the Needle in a Big Data Haystack](https://www.infoq.com/presentations/search-hadoop-data-hub) 191 | * [Large scale image processing on the fly in 25ms with Google's first Network Engineer](https://www.youtube.com/watch?v=H_9cH1tiigs) 192 | * [Bringing Push Notifications to the Mobile Web](https://www.youtube.com/watch?v=HbmcnjWFGbY) 193 | 194 | --------------------------------------------------------------- 195 | 196 | ## Papers 197 | 198 | #### General 199 | * [papers-we-love](https://github.com/papers-we-love/papers-we-love) 200 | * [Google Research](http://research.google.com/pubs/papers.html) 201 | * [Facebook Research](https://research.facebook.com/publications/) 202 | * [MIT PDOS](https://pdos.csail.mit.edu/papers/) 203 | * [Distributed Systems Reading List](https://dancres.github.io/Pages/) 204 | * [Hints for Computer System Design](http://research.microsoft.com/en-us/um/people/blampson/33-Hints/Acrobat.pdf) 205 | * The Little Manual of API Design 206 | * [On Designing and Deploying Internet-Scale Services](https://www.usenix.org/event/lisa07/tech/full_papers/hamilton/hamilton.pdf) 207 | * [Time, Clocks, and the Ordering of Events in a Distributed System](http://research.microsoft.com/en-us/um/people/lamport/pubs/time-clocks.pdf) 208 | * [Above the Clouds: A Berkeley View of Cloud Computing](http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf) 209 | * [The Byzantine Generals Problem](http://www.andrew.cmu.edu/course/15-749/READINGS/required/resilience/lamport82.pdf) 210 | * [How to Design a Good API and Why it Matters - Google Research](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/32713.pdf) 211 | * Twitter - Automatic Management of Partitioned, Replicated Search Services 212 | * [High-Availability at Massive Scale: Building Google’s Data Infrastructure for Ads](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44686.pdf) 213 | * [Making Reliable Distributed Systems in the Presence of Software Errors](http://www.erlang.org/download/armstrong_thesis_2003.pdf) 214 | * [Fallacies of Distributed Computing Explained](http://www.rgoarchitects.com/Files/fallacies.pdf) 215 | 216 | #### Search 217 | * [The Anatomy of a Large-Scale Hypertextual Web Search Engine (Google Paper)](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/334.pdf) (Weblink) 218 | * [The PageRank Citation Ranking: Bringing Order to the Web](http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf) 219 | * [Web Search for a Planet: The Google Cluster Architecture](http://research.google.com/archive/googlecluster-ieee.pdf) 220 | * [Unicorn: A System for Searching the Social Graph](http://db.disi.unitn.eu/pages/VLDBProgram/pdf/industry/p871-curtiss.pdf) ([FB link](https://research.facebook.com/publications/unicorn-a-system-for-searching-the-social-graph/)) 221 | 222 | #### P2P 223 | * [Chord: A scalable peer-to-peer lookup service for Internet applications](https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf) 224 | * [Building peer-to-peer systems with Chord, a distributed lookup service](https://pdos.csail.mit.edu/papers/chord:hotos01/hotos8.pdf) 225 | * [Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web](https://www.akamai.com/us/en/multimedia/documents/technical-publication/consistent-hashing-and-random-trees-distributed-caching-protocols-for-relieving-hot-spots-on-the-world-wide-web-technical-publication.pdf) ([CS 168: Consistent Hashing](http://web.stanford.edu/class/cs168/l/l1.pdf) | [Algorithmic Nuggets in Content Delivery](http://www.sigcomm.org/sites/default/files/ccr/papers/2015/July/0000000-0000009.pdf)) 226 | * Web Caching with Consistent Hashing ([Web link](http://www8.org/w8-papers/2a-webserver/caching/paper2.html)) 227 | * [Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems](https://github.com/papers-we-love/papers-we-love/blob/master/distributed_systems/pastry-scalable-decentralized-object-location-and-routing-for-large-scale-peer-to-peer-systems.pdf) 228 | * [Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/33339.pdf) 229 | 230 | #### Distributed Computing 231 | * [MapReduce: Simplified Data Processing on Large Clusters](http://research.google.com/archive/mapreduce-osdi04.pdf) 232 | * [Resident Distributed Datasets: a Fault-Tolerant Abstraction for In-Memory Cluster Computing (Zahari et al.)](https://www.usenix.org/system/files/conference/nsdi12/nsdi12-final138.pdf) 233 | * [Kafka: a Distributed Messaging System for Log Processing](http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf) 234 | 235 | 236 | #### Distributed Database/Large-Scale Storage 237 | * [Dynamo: Amazon's Highly Available Key-value Datastore](http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) 238 | * [The Google File System](http://research.google.com/archive/gfs-sosp2003.pdf) 239 | * [Bigtable: A Distributed Storage System for Structured Data](http://research.google.com/archive/bigtable-osdi06.pdf) 240 | * [Spanner: Google's Globally-Distributed Database - Google Research](http://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf) 241 | * TAO: Facebook’s Distributed Data Store for the Social Graph 242 | * [F1: A Distributed SQL Database That Scales](http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/41344.pdf) 243 | * [Scuba: Diving into Data at Facebook](http://db.disi.unitn.eu/pages/VLDBProgram/pdf/industry/p767-wiener.pdf) 244 | * f4: Facebook’s Warm BLOB Storage System 245 | * [Finding a needle in Haystack: Facebook’s photo storage](https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf) 246 | * [Cassandra - A Decentralized Structured Storage System](https://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf) 247 | 248 | 249 | #### Consistency 250 | * [Consistency Tradeoffs in Modern Distributed Database System Design](http://cs-www.cs.yale.edu/homes/dna/papers/abadi-pacelc.pdf) 251 | * [Paxos Made Live - An Engineering Perspective](http://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf) 252 | * [Paxos Made Simple](https://github.com/papers-we-love/papers-we-love/blob/master/distributed_systems/paxos-made-simple.pdf) 253 | * [Existential Consistency: Measuring and Understanding Consistency at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xpt1/t39.2365-6/12057015_1010569855670644_348748003_n/Existential_Consistency_Measuring_and_Understanding_Consistency_at_Facebook.pdf) 254 | * [In Search of an Understandable Consensus Algorithm](https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf) 255 | 256 | 257 | #### Distributed Graph processing 258 | * [SQLGraph: An Efficient Relational-Based Property Graph Store](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43287.pdf) 259 | * [One Trillion Edges: Graph Processing at FacebookScale](http://www.vldb.org/pvldb/vol8/p1804-ching.pdf) 260 | * [Pregel: A System for Large-Scale Graph Processing](http://kowshik.github.io/JPregel/pregel_paper.pdf) 261 | * [Dremel: Interactive Analysis of Web-Scale Datasets](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/36632.pdf) 262 | 263 | 264 | #### Company/Product specific 265 | * [Scaling Memcache at Facebook](https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf) 266 | * [Realtime Data Processing at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xat1/t39.2365-6/13331599_975087972607457_1796386216_n/Realtime_Data_Processing_at_Facebook.pdf) 267 | * [Holistic Configuration Management at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xta1/t39.2365-6/12057212_1623087361291843_1316956191_n/Holistic_Configuration_Management_at_Facebook.pdf) 268 | * [The Unified Logging Infrastructure for Data Analytics at Twitter](http://vldb.org/pvldb/vol5/p1771_georgelee_vldb2012.pdf) 269 | * [Scaling Big Data Mining Infrastructure: The Twitter Experience](http://www.kdd.org/sites/default/files/issues/14-2-2012-12/V14-02-02-Lin.pdf) 270 | * [Large-scale cluster management at Google with Borg](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf) 271 | 272 | #### Misc 273 | * [Differential Synchronization](http://research.google.com/pubs/archive/35605.pdf) 274 | * A Low-bandwidth Network File System 275 | * [Maglev: A Fast and Reliable Software Network Load Balancer](http://research.google.com/pubs/archive/44824.pdf) 276 | * [The Chubby Lock Service for Loosely-Coupled Distributed Systems](http://research.google.com/archive/chubby-osdi06.pdf) 277 | * [Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask](http://sigops.org/sosp/sosp13/papers/p33-david.pdf) 278 | * [Transactional storage for geo-replicated systems](https://github.com/papers-we-love/papers-we-love/blob/master/datastores/transactional-storage-for-geo-replicated-systems.pdf) 279 | * [Highly Available Transactions: Virtues and Limitations](http://www.bailis.org/papers/hat-vldb2014.pdf) 280 | * [The Log-Structured Merge-Tree](http://www.cs.umb.edu/~poneil/lsmtree.pdf) 281 | 282 | 283 | -------------------------------------------------------- 284 | 285 | 286 | ## Web Links 287 | 288 | #### General 289 | * https://www.hiredintech.com/ 290 | * https://github.com/checkcheckzz/system-design-interview 291 | * https://github.com/shashank88/system_design 292 | * http://highscalability.com/ ([All Time Favorites](http://highscalability.com/all-time-favorites/)) 293 | * http://blog.gainlo.co/index.php/category/system-design-interview-questions/ 294 | * http://www.allthingsdistributed.com/archives.html (Back-to-Basics series) 295 | * [ Scalability for dummies - Part 1](http://www.lecloud.net/post/7295452622/scalability-for-dummies-part-1-clones) ([Part 2](http://www.lecloud.net/post/7994751381/scalability-for-dummies-part-2-database) | [Part 3](http://www.lecloud.net/post/9246290032/scalability-for-dummies-part-3-cache) | [Part 4](http://www.lecloud.net/post/9699762917/scalability-for-dummies-part-4-asynchronism)) 296 | * [Scalable Web Architecture and Distributed Systems](http://aosabook.org/en/distsys.html) 297 | * [Introduction to Architecting Systems for Scale](http://lethain.com/introduction-to-architecting-systems-for-scale/) 298 | * [Software Architect Roadmap](https://roadmap.sh/software-architect) 299 | 300 | #### Examples 301 | 302 | Highscalability.com has wide collection of articles on Scalable architecture. Individual web links will be added below if they are not already highlighted in popular sites like highscalability. 303 | 304 | * [Trending at Instagram](http://instagram-engineering.tumblr.com/post/122961624217/trending-at-instagram) 305 | * [Search Architecture - Instagram](http://instagram-engineering.tumblr.com/post/124162066737/search-architecture) 306 | * [THE UBER ENGINEERING TECH STACK, PART I](https://eng.uber.com/tech-stack-part-one/) ([PART II](https://eng.uber.com/tech-stack-part-two/)) 307 | * [Processing Payments At Scale](https://engineering.groupon.com/2016/misc/processing-payments-at-scale/) 308 | * [Personalized Group Recommendations on Flickr](http://code.flickr.net/2016/09/30/personalized-group-recommendations-on-flickr/) 309 | * [Building The LinkedIn Knowledge Graph](https://engineering.linkedin.com/blog/2016/10/building-the-linkedin-knowledge-graph) 310 | * [Personal recommendations for the Foursquare homescreen](http://engineering.foursquare.com/2015/08/06/personal-recommendations-for-the-foursquare-homescreen/) 311 | 312 | -------------------------------------------------- 313 | 314 | ## Books 315 | * [Microsoft Application Architecture Guide, 2nd Edition (Online)](https://msdn.microsoft.com/en-us/library/ff650706.aspx) 316 | * [The Architecture of Open Source Applications](http://aosabook.org/en/index.html) 317 | * Design Patterns: Elements of Reusable Object-Oriented Software 318 | * Head First Design Patterns 319 | * Patterns of Enterprise Application Architecture 320 | * Domain Driven Design by Eric Evans 321 | * Agile Software Development, Principles, Patterns and Practices by Robert Martin 322 | 323 | -------------------------------------------------- 324 | --------------------------------------------------------------------------------