├── iOS ├── images │ └── notificationCenter.png └── NotificationCenter.md ├── CONTRIBUTING.md ├── LICENSE.md ├── exercises ├── README.md ├── file-downloader-library.md ├── image-library.md └── caching-library.md ├── README.md ├── images ├── resumable-uploads.svg ├── exercise-caching-library-encryption.svg ├── exercise-caching-library-high-level-diagram.svg ├── exercise-caching-library-high-level-diagram-simplified.svg ├── exercise-image-library-high-level-diagram.svg ├── exercise-file-downloader-library-job-diagram.svg ├── exercise-file-downloader-library-high-level-diagram.svg ├── exercise-file-downloader-library-deep-dive-download-dispatcher-diagram.svg └── exercise-caching-library-dispatcher-sequence.svg ├── BLOGPOSTS.MD └── common-interview-mistakes.md /iOS/images/notificationCenter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/levabond/ios-mobile-system-design/HEAD/iOS/images/notificationCenter.png -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Thanks for helping out the project! You're awesome! 4 | 5 | All the contributors to the `mobile-system-design` repo should sign the [Individual Contributor License Agreement (CLA)](https://forms.gle/oqgHfGiJZ28KJy7w9). This form ensures we can provide a solid open source project and makes life easier. 6 | 7 | Please, make sure to agree to the terms and conditions before sending a [pull request](https://github.com/weeeBox/mobile-system-design/pulls). 8 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright (c) 2021-present, Alex Lementuev 2 | All rights reserved. 3 | 4 | All material (“content”) is protected by copyright under U.S. Copyright laws and is the property of Alex Lementuev or the party credited as the provider of the content. You may not copy, reproduce, distribute, publish, display, perform, modify, create derivative works, transmit, or in any way exploit any such content, nor may you distribute any part of this content over any network, including a local area network, sell or offer it for sale, or use such content to construct any kind of database. You may not alter or remove any copyright or other notice from copies of the content in the repository. Copying or storing any content except as provided above is expressly prohibited without prior written permission of the owner or the copyright holder identified in the individual content’s copyright notice. 5 | -------------------------------------------------------------------------------- /exercises/README.md: -------------------------------------------------------------------------------- 1 | # Typical Mobile System Design Interview Questions 2 | The following exercises emulate typical interview scenarios using common system design questions. The solution to each question might not be accurate and should not be taken as the "ground truth". The purpose of the exercises is to demonstrate different interviewer-candidate interactions. 3 | - [Design "File Downloader Library"](/exercises/file-downloader-library.md) 4 | - [Design "Caching Library"](/exercises/caching-library.md) 5 | - [Design "Image Library"](/exercises/image-library.md) 6 | - Design "Location Sharing Library" 7 | - Design "Network Library" 8 | - Design "Json Parsing Library like Moshi/Gson" 9 | - Design "Pagination Library" 10 | - Design "Analytics Library" 11 | - Design "A/B Experiment Library" 12 | - Design "Photo App" 13 | - upload / sync photos with backend 14 | - [Design "Chat App"](/exercises/chat-app.md) 15 | - Design "Clock App" 16 | - Design "Recipe App" 17 | - Design "Photo Editing App" 18 | - Design "Live Wallpaper Display App" 19 | - Design "Instagram Post Feed/Facebook News Feed" 20 | - Design "Facebook/Instagram story" 21 | - Design "Flight booking system" 22 | - How to cache status 23 | - How to handle case like leaving a page and coming back without previous information loss 24 | - How to handle transaction failure 25 | - How to lock seat selection 26 | - Design "Contact app with real time status" 27 | - Design "Push notifications system" 28 | - broadcast receiver + notification manager + persistent connection 29 | - Design "File uploader library" 30 | 31 | 32 | Feel free to [suggest](https://github.com/weeeBox/mobile-system-design/issues/new) your own topics. 33 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Для чего этот форк? 2 | Хочется обогатить его в контексте iOS разработчика с упором дополнений из 3 | - https://www.mobilesystemdesign.com/ 4 | - https://www.mobileatscale.com/ 5 | - Множество других инструментов и книг из соседних платформ (бэк, фронт, андроид) 6 | - Сбор мнений и валидация через комьюнити экспертов https://t.me/iosmakesmehate 7 | 8 | ## Простой мобильный фреймворк для System Design 9 | Ниже представлена ​​простая схема собеседований по проектированию мобильных систем. В качестве примера мы собираемся использовать вопрос «Дизайн ленты Twitter». Предлагаемое решение далеко от совершенства, но оно не является целью собеседования по проектированию системы: никто не ожидает, что вы создадите надежную систему всего за 30 минут - интервьюер в основном ищет конкретные «сигналы» из вашего мыслительного процесса и коммуникация. Все, что вы говорите или делаете, должно демонстрировать ваши сильные стороны и помогать интервьюеру оценить вас как кандидата. 10 | 11 | ## Disclaimer 12 | Изучение фреймворка не гарантирует прохождение собеседования. Структура процесса собеседования зависит от личного стиля интервьюера. Диалог действительно важен — убедитесь, что вы понимаете, что ищет интервьюер. Не делайте предположений и задавайте уточняющие вопросы. 13 | 14 | ## Цель интервью 15 | Проверяем что кандидат умеет и понимает, как строить архитектуру клиент-сервер или проектировать клиентское приложение. 16 | Важно, что мы хотим проверить как кандидат двигается по процессу. Нет задачи оценивать финальное решение, потому что за час невозможно спроектировать идеальную систему. 17 | Главное, как он принимает решение, какие вопросы задает, какие компромиссы находит. 18 | 19 | ## Процесс собеседования 20 | - 2–5 мин - знакомства 21 | - 5 мин - постановка задачи и сбор требований 22 | - 10 мин - обсуждение на высоком уровне 23 | - 20-30 мин - подробное обсуждение 24 | - 5 мин - вопросы интервьюеру 25 | 26 | 27 | ## Как проводить интервью? 28 | Проверяем умения: 29 | - Собирать и уточнять требования и ограничения 30 | - Строить разумные предположения в условиях неопределённости 31 | - Предлагать варианты решений и компромиссы 32 | - Проектировать решение задачи и визуализировать его 33 | - Объяснять решение другому человеку так, чтобы он понял 34 | 35 | ## Как начать интервью 36 | Сегодня у нас с тобой секция проектирования. Мы попробуем спроектировать %экран / библиотеку / кусок функциональности%. Сразу оговорюсь, что в этой секции нет единственно правильного ответа, ход твоих мыслей так же важен, как итоговый результат. Нужно задавать вопросы, говорить про плюсы и минусы решений, озвучивать почему ты принял те или иные решения. 37 | 38 | ## Подготовка к проведению интервью 39 | До проведения интервью HR должен рассказать, как будет проходить секция и сообщить следующую информацию: 40 | Понадобится использовать онлайн-доску и шарить экран. Рекомендуемые инструменты: доска для рисования Miro, Whimsical или Draw.io или другой подходящий инструмент. Желательно зарегистрироваться заранее (если это необходимо) и немного освоиться с инструментом до собеседования. 41 | 42 | Интервью проходит в формате беседы, где интервьюер - заказчик. Он даёт задание и отвечает на вопросы кандидата, предоставляя информацию о требованиях, условиях и ограничениях. 43 | Мы ожидаем увидеть архитектурную схему решения. Начинаем с простого варианта, если останется время, то поговорим про расширение и усложнение требований и адаптацию решения под них. 44 | Можно использовать в качестве нотации диаграмму классов UML но в целом схему можно нарисовать в свободном формате, это не влияет на итоговый результат. Можно даже накидывать интерфейсы и комментарии в IDE или текстовом редакторе, если кандидату так проще и в итоге получится понятно для интервьюера. 45 | 46 | 47 | ## Инструменты 48 | Онлайн whiteboard: [Draw.io](https://draw.io/), [Miro](https://miro.com/), Whimsical или Excalidraw 49 | -------------------------------------------------------------------------------- /images/resumable-uploads.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /BLOGPOSTS.MD: -------------------------------------------------------------------------------- 1 | # Curated List of Real-world Design Blog Posts 2 | 3 | Real world examples where companies discuss trade-offs of certain technologies and how they design things to fit their use case. Contains a selection of posts that may be targeted to a platform, but concepts should be relevant to all 4 | 5 | [Here](https://github.com/sumodirjo/engineering-blogs) is a big list of company engineering blogs 6 | 7 | ## Backend 8 | 9 | - Airbnb 10 | - [How Airbnb is Moving 10x Faster at Scale with GraphQL and Apollo](https://medium.com/airbnb-engineering/how-airbnb-is-moving-10x-faster-at-scale-with-graphql-and-apollo-aa4ec92d69e2) 11 | - Instacart 12 | - [Building Instacart’s view model API — Part 1: Why view model?](https://tech.instacart.com/building-instacarts-view-model-api-part-1-why-view-model-4362f64ffd2a) 13 | - Netflix 14 | - [It’s All A/Bout Testing: The Netflix Experimentation Platform](https://netflixtechblog.com/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15) 15 | 16 | ## API Design 17 | - Slack 18 | - [How We Design Our APIs at Slack](https://slack.engineering/how-we-design-our-apis-at-slack/) 19 | - [Evolving API Pagination at Slack](https://slack.engineering/evolving-api-pagination-at-slack/) 20 | - Trello 21 | - [Adopting GraphQL and Apollo in a Legacy Application](https://tech.trello.com/adopting-graphql-and-apollo/) 22 | - Twitter 23 | - [Pagination Overview](https://developer.twitter.com/en/docs/twitter-api/pagination) 24 | 25 | ## Build / Platform 26 | - Airbnb 27 | - [Designing for Productivity in a Large-Scale iOS Application](https://medium.com/airbnb-engineering/designing-for-productivity-in-a-large-scale-ios-application-9376a430a0bf) 28 | - Facebook 29 | - [Rethinking Android app compilation with Buck](https://engineering.fb.com/2017/11/09/android/rethinking-android-app-compilation-with-buck/) 30 | - Pinterest 31 | - [Developing fast & reliable iOS builds at Pinterest](https://medium.com/pinterest-engineering/developing-fast-reliable-ios-builds-at-pinterest-part-one-cb1810407b92) 32 | - Reddit 33 | - [iOS and Bazel at Reddit: A Journey](https://www.reddit.com/r/RedditEng/comments/syz5dw/ios_and_bazel_at_reddit_a_journey/) 34 | - Uber 35 | - [The Journey To Android Monorepo: The History Of Uber Engineering’s Android Codebase Organization](https://eng.uber.com/android-engineering-code-monorepo/) 36 | - Various 37 | - [Monorepo Discussion](https://github.com/MobileNativeFoundation/discussions/discussions/31) 38 | 39 | ## Caching / Offline 40 | - Instagram 41 | - [Building an Open Source, Carefree Android Disk Cache](https://instagram-engineering.com/building-an-open-source-carefree-android-disk-cache-af57aa9b7c7) 42 | - Trello 43 | - [Airplane Mode: Enabling Trello Mobile Offline](https://tech.trello.com/sync-architecture/) 44 | - [Syncing Changes](https://tech.trello.com/syncing-changes/) 45 | - [Sync Failure Handling](https://tech.trello.com/sync-failure-handling/) 46 | - [The Two ID Problem](https://tech.trello.com/sync-two-id-problem/) 47 | - [Offline Attachments](https://tech.trello.com/sync-offline-attachments/) 48 | - [Sync is a Two-Way Street](https://tech.trello.com/sync-downloads/) 49 | - [Displaying Sync State](https://tech.trello.com/sync-indicators/) 50 | 51 | ## Emerging Markets 52 | - Facebook 53 | - [How we built Facebook Lite for every Android phone and network](https://engineering.fb.com/2016/03/09/android/how-we-built-facebook-lite-for-every-android-phone-and-network/) 54 | - Microsoft 55 | - [Microsoft Teams — Designing for Emerging Markets — Part 1 (Network Profile)](https://medium.com/microsoft-mobile-engineering/microsoft-teams-designing-for-emerging-markets-part-1-network-profile-2daeaa09f313) 56 | - Spotify 57 | - [How We Built It: Spotify Lite, One Year Later](https://engineering.atspotify.com/2020/12/how-we-built-it-spotify-lite-one-year-later/) 58 | - Uber 59 | - [Expanding Access: Engineering Uber Lite](https://eng.uber.com/engineering-uber-lite/) 60 | 61 | ## Networking / Performance 62 | - Dropbox 63 | - [Making camera uploads for Android faster and more reliable](https://dropbox.tech/mobile/making-camera-uploads-for-android-faster-and-more-reliable) 64 | - Instagram 65 | - [Improving performance with background data prefetching](https://instagram-engineering.com/improving-performance-with-background-data-prefetching-b191acb39898) 66 | - [Making Direct Messages Reliable and Fast](https://instagram-engineering.com/making-direct-messages-reliable-and-fast-a152bdfd697f) 67 | - LinkedIn 68 | - [Building a smooth Stories experience on iOS](https://engineering.linkedin.com/blog/2020/building-stories-on-ios) 69 | - Uber 70 | - [How Uber’s New Driver App Overcomes Network Lag](https://eng.uber.com/driver-app-optimistic-mode/) 71 | 72 | ## Networking / Reliability 73 | - SnapChat 74 | - [QUIC at Snapchat](https://eng.snap.com/quic-at-snap) 75 | - Uber 76 | - [Engineering Failover Handling in Uber’s Mobile Networking Infrastructure](https://eng.uber.com/eng-failover-handling/) 77 | - [Employing QUIC Protocol to Optimize Uber’s App Performance](https://eng.uber.com/employing-quic-protocol/) 78 | 79 | ## Server-Driven UI 80 | - AirBnB 81 | - [A Deep Dive into Airbnb’s Server-Driven UI System](https://medium.com/airbnb-engineering/a-deep-dive-into-airbnbs-server-driven-ui-system-842244c5f5) 82 | - DoorDash 83 | - [Improving Development Velocity with Generic, Server-Driven UI Components](https://doordash.engineering/2021/08/24/improving-development-velocity-with-generic-server-driven-ui-components/) 84 | - Uber 85 | - [Building the New Uber Freight App as Lists of Modular, Reusable Components](https://eng.uber.com/uber-freight-app-architecture-design/) 86 | - [Architecting a Safe, Scalable, and Server-Driven Platform for Driver Preferences with RIBs](https://eng.uber.com/carbon-driver-app-preferences-ribs/) 87 | - Various - Strategies Discussion 88 | - [Server-driven UI (or Backend driven UI) strategies](https://github.com/MobileNativeFoundation/discussions/discussions/47) 89 | 90 | ## Architecture 91 | - [Martin Fowler](https://martinfowler.com/architecture/) 92 | -------------------------------------------------------------------------------- /common-interview-mistakes.md: -------------------------------------------------------------------------------- 1 | # Common Interview Mistakes 2 | 3 | ## As a candidate 4 | 5 | ### Not being prepared 6 | - **Study open-source projects** - not every project is perfect but you can still learn lots of useful things from each of them. 7 | - **Study company dev blog** - you can learn a lot about their underlying technical stack and think about potential questions they might ask. 8 | - **Conduct mock interviews with your friends** - time management is crucial: your goal is to cover as much ground as possible in the shortest amount of time. Having some practice also eases up the stress from the actual interview. 9 | 10 | ### Rushing towards a solution 11 | - **Not gathering system requirements** - the interview question may be purposely vague. The candidate is expected to ask more questions to define the task better. For more information see [Gathering Requirements](https://github.com/weeeBox/mobile-system-design#gathering-requirements). 12 | - **Not asking clarifying questions** - it might be useful to get some information about the target market, the size of the audience, and the dev team details. 13 | - **Jumping straight to implementation** - it's generally a bad sign if the candidate immediately starts discussing low-level topics. For example, which view classes to use or what UI-architecture pattern to apply. The interviewer might not be interested in implementation details in the first place. 14 | 15 | ### Being unresponsive 16 | - **Keeping silence** - make sure to talk through your solution. 17 | - **Waiting until the interviewer starts asking questions** - it is preferable for the candidate to "drive" the discussion (especially for more senior candidates). 18 | 19 | ### Giving up early 20 | A single failure might de-rail the candidate and make them give up the whole interview. This is a poor strategy since most of the interviewers are trying to evaluate the candidate's abilities and not only looking for "correct" answers. 21 | 22 | ### Talking too much 23 | - **Long introduction** - digging deeper into your professional background does not provide much "signal". The interviewer would most likely form an opinion based on the candidate's interview performance and not the summary of their professional experience. 24 | - **Trying to force an interviewing framework** - the said guide represents a set of recommendations and does not define an official interview protocol for any company. The candidate should follow the interviewer's lead in each particular case. 25 | - **Ignoring the interviewer** - it is better to stop talking if the interviewer interrupts you: do not try "to finish the thought" - it is better to move to another topic and not waste time. 26 | - **Repeating yourself** - talking about the same things again: it does not provide any additional information to the interviewer. 27 | - **Jumping from a topic to a topic** - frequent context switching might be hard to follow for the interviewer. 28 | - **Going too broad with the answers** - covering irrelevant things does not provide meaning full "signal" to the interviewer. Try to stick to the original question. 29 | 30 | ### Treating the interviewer as an adversary 31 | The primary role of the interviewer is to determine if the candidate is fit for the job. If the candidate feels any kind of hostility from the interviewer - it should be reported to the recruiting specialist. 32 | 33 | ### Trying to fit a solution into an existing scheme 34 | Most pronounced for the candidates who learn particular designs on the Internet and attempt fitting the questions into an unrelated solution. 35 | 36 | ### Being toxic 37 | - **Being opinionated** - holding a strong belief about certain technology and rejecting any alternative approaches. 38 | - **Interrupting the interviewer** - the interviewer might provide some useful hints or suggest a direction toward the solution. The candidate should let the interviewer finish their thought before bringing up any kind of disagreement. 39 | - **"Educating" the interviewer** - if the candidate believes that the interviewer is wrong/incorrect - it's better to politely suggest this instead of lecturing them on their ignorance. Spending time on "educating" the interviewer provides no useful "signal" and may raise some red flags. 40 | - **Being a "pleaser"** - trying to compliment the interviewer hoping for a positive feedback report in return. 41 | 42 | ### Not explaining decisions or not suggesting alternatives 43 | It is a red flag when the candidate offers a concrete solution right away without any prior justification. The interviewer might not be able to understand why the candidate prefers a specific approach without comparing it to possible alternatives. 44 | 45 | ### Lying 46 | Making up things and lying about them is risky since there's a chance that the interviewer might be a domain expert in the area of the discussion. 47 | 48 | ### Not having any structure for the solution 49 | Most pronounced for the candidates who don't prepare for their rounds. Not having a plan for structuring the solution would put additional pressure during the interview. 50 | 51 | ### Speaking in terms of specific vendors 52 | Using vendor-specific solutions during the interview makes it more about the implementation details and not about architecture design. The candidate may omit all vendor information unless the interviewer explicitly asks about it. 53 | 54 | ### Being overly optimistic 55 | Sometimes, the candidate does not take the "unhappy" path into account. For example, running out of memory or disk space. 56 | 57 | ### Making assumptions 58 | It's better to admit that you don't know how things work instead of making assumptions. It's ok to reason about how a certain functionality _could_ be implemented but you need to state your lack of knowledge first. 59 | 60 | ### Ignoring the interviewer's hints 61 | The interviewer is there to help you - take a hint! 62 | 63 | ## As an interviewer 64 | _Note: Cognitive hiring biases are left out of scope for this guide. You can [find](https://blog.staffingadvisors.com/5-cognitive-biases-that-get-in-the-way-of-hiring) lots of information on the Internet._ 65 | 66 | ### Not being prepared 67 | The interviewer should carefully think through the question and figure out possible topics for discussion and follow-up questions. The selected topics should reflect the candidate's strengths and not the interviewer's experience. 68 | 69 | ### Being toxic 70 | - **Being a jerk** - treat the candidate with disrespect. 71 | - **Acting "superior"** - showing your expertise and trying to belittle the candidate. 72 | - **"Educating" the candidate** - some interviewers may ask a question and then answer it themselves when the candidate fails to do so. It does not help to evaluate the candidate and wastes the interview time. 73 | 74 | ### Being unresponsive 75 | - **Not paying full attention** - checking the phone, daydreaming, or responding to emails. 76 | - **Eating** - no comments. 77 | 78 | ### Being too engaged 79 | - **Suggesting solution to the candidate** - pushing a candidate in the "right" direction, giving too many hints. 80 | 81 | ### "Grilling" the candidate 82 | - **Asking narrow sub-domain questions** - a type of questions focusing on specific things the candidate does not need to know to be successful at the job. 83 | - **Trying to corner the candidate** - asking intentionally tough and tricky questions to make the candidate admit they don't know the answer. 84 | 85 | ### Asking meaningless questions 86 | - "Why did you choose a relational database (ORM) to store structured data that requires complex querying?" - there's little "signal" a candidate can provide here (unless they advocate for text files or user preferences). 87 | - "Why caching/modularization/testing/etc is necessary?" 88 | - "Why would you use an industry-standard library instead of a custom implementation?" - this question might make sense when it comes to framework development but _could_ be meaningless when talking about an app. See "Chapter 21. Dependency Management" from the "Software Engineering at Google" book. 89 | 90 | Everything the interviewer asks should help determine if the candidate is a fit for the job. Any other questions should be left out of scope. 91 | 92 | ### Not moving on when the candidate is stuck 93 | Letting the candidate "struggle a little" and allowing long pauses. It's better to give a hint sooner than later or move to the next topic. 94 | 95 | ### Forcing the candidate into a solution they have in mind 96 | It won't help the candidate to show off their strong side. 97 | 98 | ### Digging too much into details (BFS approach) 99 | Overly concentrating on a narrow topic and leaving high-level stuff behind. 100 | -------------------------------------------------------------------------------- /images/exercise-caching-library-encryption.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /images/exercise-caching-library-high-level-diagram.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /images/exercise-caching-library-high-level-diagram-simplified.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /exercises/file-downloader-library.md: -------------------------------------------------------------------------------- 1 | # Mobile System Design Exercise: File Downloader Library 2 | The task might be simply defined as "Design File Downloader Library". The definition is purposely vague and requires some clarification. 3 | 4 | ## Gathering Requirements 5 | You are expected to ask clarifying questions and narrow down the scope of the task. This should not take more than 5 minutes. 6 | 7 | > **Candidate**: "Are we designing a part of an application or a general-purpose library?" 8 | > **Interviewer**: "A general-purpose library." 9 | 10 | > **Candidate**: "Are we downloading a file from the internet and saving it to the disk?" 11 | > **Interviewer**: "Yes." 12 | 13 | > **Candidate**: "What kind of files do we need to download? Any size constraints?" 14 | > **Interviewer**: "Any kind - let's assume that we're working with binary files without specific format and no size limit." 15 | 16 | > **Candidate**: "Should we support pausing/resume/canceling/listing downloads?" 17 | > **Interviewer**: "Yes." 18 | 19 | > **Candidate**: "Do we need to support simultaneous file downloads?" 20 | > **Interviewer**: "I don't know - what do you think?" 21 | > **Candidate**: "I think, there might be some good use-cases where simultaneous downloads would be useful. For example, downloading a video playlist." 22 | 23 | > **Candidate**: "Should we limit the number of active simultaneous downloads?" 24 | > **Interviewer**: "I don't know - what do you think?" 25 | > **Candidate**: "I think, having unrestricted parallel downloads might hurt application performance and quickly exhaust system resources. At the same time, we can't make any assumptions about app-specific use-cases, so the best approach might be selecting a sensible default value (for example, no more than `4` parallel downloads) and letting developers configure it based on their application's need". 26 | 27 | > **Interviewer**: "Why do you think `4` is a good number for parallel downloads?" 28 | > **Candidate**: "It's a tricky question. We can use different heuristics to select the best number. For example, we can limit the size to the number of CPU cores on the device." 29 | > **Candidate**: "On the other hand - each downloader would be blocked on the network I/O so we can use a higher number like `16`." 30 | > **Candidate**: "It's hard to figure this out for the general case. So `4`-`16` seems reasonable by default. The user may pick a better size depending on the use-case." 31 | 32 | > **Candidate**: "Should we support progress reporting for active downloads?" 33 | > **Interviewer**: "Might skip it for now and discuss it if we have time" 34 | 35 | > **Candidate**: "Do we need to handle authentication?" 36 | > **Interviewer**: "Let's skip it." 37 | 38 | > **Candidate**: "Do we need to support [HTTP ranges](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests) for resumable downloads?" 39 | > **Interviewer**: "We can leave it out of scope" 40 | 41 | Make sure not to overload the system requirements with unnecessary features. Think in terms of MVP (Minimum Viable Product) and pick features that have the biggest value. You can learn more about requirements gathering [here](https://github.com/weeeBox/mobile-system-design#gathering-requirements). 42 | ### Functional requirements 43 | - Developers should be able to simultaneously download multiple files over HTTP to the disk. 44 | - Developers should be able to pause/resume/cancel downloads. 45 | - Developers should be able to list active downloads. 46 | 47 | ### Non-functional requirements 48 | - Limited active simultaneous downloads. 49 | - Unlimited download file size. 50 | 51 | ### Out of scope 52 | - Login/Authentication. 53 | - Resumable HTTP-downloads. 54 | 55 | ## Client Public API (Optional) 56 | Your interviewer might want to discuss the client public API. In the simplest form it might look like this: 57 | ``` 58 | FileDownloader: 59 | + init(config: FileDownloaderConfig) 60 | + download(request: FileDownloadRequest): FileDownloadTask 61 | + pauseAll() 62 | + resumeAll() 63 | + cancelAll() 64 | + activeTasks(): [FileDownloadTask] 65 | 66 | FileDownloadRequest: 67 | + init(sourceUrl: Url, destPath: String) 68 | 69 | FileDownloadTask: 70 | + addDownloadCallback(callback: FileDownloadCallback) 71 | + pause() 72 | + resume() 73 | + cancel() 74 | 75 | FileDownloaderConfig: 76 | + init(maxParallelDownloads: Int) 77 | 78 | FileDownloadCallback: 79 | + onComplete(request: FileDownloadRequest) 80 | + onFail(request: FileDownloadRequest, error: String) 81 | + onCancel(request: FileDownloadRequest) 82 | ``` 83 | 84 | - **FileDownloader** - represents a single file downloader instance; schedules and maintains active downloads. 85 | - **FileDownloadRequest** - encapsulates a single download request (source, dest, headers, etc). 86 | - **FileDownloadTask** - a handle to an asynchronous file downloading operation. 87 | - **FileDownloaderConfig** - encapsulates file downloader configuration. Simplifies downloader instance creation and future refactorings. As an alternative, you may suggest the Builder pattern. 88 | - **FileDownloadCallback** - encapsulate completion/failure callbacks. 89 | 90 | ## High-Level Diagram 91 | A high-level diagram shows all major system components and their interactions (without implementation details). You can learn more about high-level diagrams [here](https://github.com/weeeBox/mobile-system-design#high-level-diagram). 92 | 93 | ![High-level Diagram](/images/exercise-file-downloader-library-high-level-diagram.svg) 94 | 95 | ### Components 96 | - **File Downloader** - the central component which provides the client API and brings all components together. 97 | - **Download Request** - encapsulates a single file downloading request; accepted by file downloader as input. 98 | - **Download Task** - represents an asynchronous download operation; produced by the file downloader as output. 99 | - **Download Dispatcher** - schedules and dispatches download operations. 100 | - **Network Client** - handles receiving bytes over HTTP. 101 | - **File Store** - writes file contents to the disk. 102 | 103 | ## Deep Dive: Download Dispatcher 104 | After a high-level discussion, your interviewer might want to discuss some specific components of the system. Make sure to keep your explanation brief and don't overload it with details - let your interviewer guide the conversation and ask questions instead. You can learn more about deep-dive discussions [here](https://github.com/weeeBox/mobile-system-design#deep-dive-tweet-feed-flow). 105 | 106 | ![Download Dispatcher Diagram](/images/exercise-file-downloader-library-deep-dive-download-dispatcher-diagram.svg) 107 | 108 | > **Candidate**: "Download Dispatcher maintains a _concurrent_ dispatch queue of jobs. Each job consists of a download request, a download task, and a state (PENDING, ACTIVE, PAUSED, COMPLETED, FAILED). The active jobs are dispatched by download workers." 109 | 110 | > **Interviewer**: "Why do you need to maintain a job state?" 111 | > **Candidate**: "To keep track of _pending_, _active_, and _completed_ jobs: we limit the number of parallel downloads by design. Also, we need to maintain the `pause` state of the scheduled jobs." 112 | 113 | > **Interviewer**: "What's the difference between a `Job` and a `Download Worker`?" 114 | > **Candidate**: "A `Job` encapsulates a single downloading request from the user. A `Download Worker` is responsible for actual data transmission from the network." 115 | > **Candidate**: "A `Job` is cheap and doesn't do anything. A `Download Worker` is expensive and handles blocking I/O." 116 | > **Candidate**: "There might be an unlimited number of jobs and only a handful of workers (limited by the pool size)." 117 | > **Candidate**: "There might be _multiple jobs_ for the same URL but only _one worker_ per download. For example, the same video file can be included in multiple playlists. The user can download these playlists in parallel - as a result, there might be different jobs for the same worker." 118 | > **Candidate**: "If a single job gets canceled - its worker keeps downloading until done or all the associated jobs are canceled, too." 119 | 120 | ![Download Job Diagram](/images/exercise-file-downloader-library-job-diagram.svg) 121 | 122 | > **Interviewer**: "Why do you need Init and Complete/Fail operations in the File Store?" 123 | > **Candidate**: "This gives you an ability to pre-allocate disk space before downloading a file and perform post-processing/cleanup after a complete download." 124 | 125 | ## Follow-up Questions 126 | Some interviewers might ask follow-up questions that might change the original design and introduce new requirements. 127 | 128 | > **Interviewer**: "How would you change your design if the library needs to keep track of downloaded files?" 129 | > **Candidate**: 130 | > - "We can add a component ("Progress Store") that maintains a structured record of scheduled jobs in a relational database." 131 | > - "Each job would have an associated "progress" callback to signal every time the next chunk of bytes is received from the network." 132 | > - "The Progress Store component will use the job callbacks to update the database." 133 | > - "Once the job is complete - the corresponding record is deleted." 134 | 135 |
136 | 137 | | name | type | 138 | |------------------|--------| 139 | | url | String | 140 | | path | String | 141 | | created_at | Date | 142 | | total_bytes | Int | 143 | | downloaded_bytes | Int | 144 | | state | Int | 145 | 146 |
147 | 148 | > **Interviewer**: "Why won't you store files in the database?" 149 | > **Candidate**: "It's easier to access a file on the disk than from the database. We can partially read BLOBs and provide the content as a stream but it might not be sufficient in the general case." 150 | 151 | > **Interviewer**: "How would you handle downloading sensitive information?" 152 | > **Candidate**: "We might introduce an "encrypted" File Store implementation and encrypt a fully-received file in post-processing. Not sure if we can do it on the fly - don't have much experience working with encryption libraries." 153 | 154 | > **Interviewer**: "How would you change your design to support download requests of different priorities? For example, user-critical, UI-critical, UI-non-critical, low-priority?" 155 | > **Candidate**: "The first option: update Download Dispatcher to use a priority queue; the second option: configure on the File Downloader instance level and maintain different instances based on the priority (would also require synchronization between instances). The priority information could be included in a Download Request." 156 | 157 | ## Foreground and Background Downloads (Optional) 158 | You may ask your interviewer if they want to discuss foreground/background downloads. 159 | ### Foreground Download 160 | Uses a worker thread in the host application process. Works best for short-running and urgent downloads. 161 | #### pros: 162 | - Immediate execution within the host app. 163 | - Limited only by system resources. 164 | - Easy to set up and troubleshoot. 165 | 166 | ### cons: 167 | - Stopped when the application is killed or suspended (works slightly different between iOS and Android). 168 | 169 | ### Background Download  170 | Might run in a separate process while the app is suspended. Best for long-running and nonurgent downloads. 171 | 172 | #### pros: 173 | - Guaranteed execution (iOS - survives backgrounding the app; Android - survives device restart). 174 | 175 | #### cons: 176 | - Hard to set up and debug. 177 | - The host OS might wait for optimal conditions to perform the transfer, such as when the device is plugged in or connected to Wi-Fi. 178 | - Must comply with Background Transfer Limitations. 179 | 180 | ### Implementation Details 181 | - A preferred download method is selected with the File Downloading request. 182 | - The Download Dispatcher selects appropriate Download Worker implementation based on request configuration. 183 | 184 | ## Major Concerns and Trade-Offs 185 | 186 | ### Network/CPU/Battery Usage 187 | - Having too many parallel downloads might negatively impact device battery life and increase cellular network usage. A possible workaround is to adjust the concurrency settings depending on the device state. For example, limit the number of parallel downloads to `1` when using the cellular network or having a low battery charge, etc. 188 | 189 | ### Disk Space Allocation 190 | - Another major decision is whether to pre-allocate disk space or not. For example, the library can create place-holder files for every download in the queue to ensure enough space regardless of the file system state. It's hard to argue about this in a general case - as a workaround this can be configurable while initializing a File Downloader instance. 191 | 192 | ## Conclusion 193 | - Keep this in mind while preparing for a system design interview: 194 | - Don't try to make it perfect - provide a "signal" instead. 195 | - Listen to your interviewer and keep track of the time. 196 | 197 | Try to cover as much ground as possible without digging too much into the implementation details. 198 | 199 | ## Looking for more content? 200 | I’m thinking about creating an in-depth mobile system design course on top of the free articles. Please, fill out this [form](https://forms.gle/KfvmZhPNPMRBE8Jj9) to express your interest! 201 | -------------------------------------------------------------------------------- /iOS/NotificationCenter.md: -------------------------------------------------------------------------------- 1 | # Задачи на проектирование: кастомный Notification Center 2 | 3 | * Что такое Notification Center 4 | * Дефолтная реализация 5 | * Улучшение реализации 6 | 7 | Одна из частых задач, которую можно встретить — это написать свою реализацию Notification Center’а. 8 | 9 | Почти каждый разработчик использовал NSNotificationCenter. Это очень полезный механизм подписок и подписчиков, который легко помогает передавать данные между экранами. 10 | В статье будет 3 примера, которые иттеративно улучшаются в зависимости от поставленых требований: от худшего к лучшему. 11 | 12 | В основе него лежит два паттерна: Singletone и Observer. 13 | 14 | Паттерн Observer состоит наблюдаемого (Observable) класса, который генерирует события (events) и наблюдателей (Observers). Наблюдатели, которые на него подписаны, получают уведомления при генерации событий. 15 | 16 | ![title](./images/notificationCenter.png) 17 | 18 | ## Реализация 19 | 20 | Как упоминалось ранее, наш пользовательский Notification Center будет синглтоном, только один экземпляр будет использоваться всеми, кто использует этот API в приложении. 21 | 22 | ```swift 23 | final class CustomNotificationCenter { 24 | static private(set) var shared = CustomNotificationCenter() 25 | private var observers: [String: [String: [(String, Any?) -> ()]]] = [:] 26 | 27 | private init() {} 28 | } 29 | ``` 30 | 31 | Каждый раз, когда нам нужно использовать MyNotificationCenter, мы будем обращаться к нему с помощью переменной MyNotificationCenter.shared. 32 | 33 | ## Добавление класса в качестве наблюдателя (observer) 34 | 35 | Нам нужны методы, которые могут нам добавить класс в качестве наблюдателя. 36 | 37 | Один класс может быть наблюдателем для нескольких уведомлений. Также может быть случай, когда несколько классов могут регистрироваться для одного и того же уведомления, но могут выполнять разные действия. 38 | 39 | Таким образом, наш API addObserver будет принимать 3 параметра: 40 | 41 | 1. Class — экземпляр класса, который действует как наблюдатель. 42 | 2. Name - Название уведомления. 43 | 3. closure — это блок кода, который выполняется после запуска уведомления. Замыкание может содержать произвольный код, но оно всегда будет вызываться с двумя параметрами: именем уведомления и объектом — данными, которые мы хотим передать с уведомлением. 44 | 45 | ```swift 46 | func addObserver( 47 | _ inputClass: T, 48 | name: String, 49 | closure: @escaping (String, Any) -> () 50 | ) { 51 | let className = String(describing: inputClass.self) 52 | 53 | if observers[className] != nil && observers[className]?[name] != nil { 54 | observers[className]?[name]?.append(closure) 55 | } else { 56 | observers[className] = [name: [closure]] 57 | } 58 | } 59 | ``` 60 | 61 | ## Отправка уведомления 62 | 63 | Что происходит, когда мы отправляем уведомление? Вызываются все наблюдатели, наблюдающие за уведомлением с этим именем, и вызываются соответствующие блоки. 64 | 65 | Когда кто-то публикует уведомление с этим именем и параметром объекта, мы просматриваем контейнер NotificationsStorage, находим все уведомления с этим именем и запускаем их одно за другим. 66 | 67 | Если мы не сможем найти какой-либо блок уведомлений с таким именем, мы просто выдадим исключение NotFound. 68 | 69 | ```swift 70 | 71 | func post( 72 | name: String, 73 | object: Any? = nil 74 | ) { 75 | for (_, names) in observers { 76 | for (observeName, closures) in names { 77 | guard name == observeName else { return } 78 | 79 | for observeClosure in closures { 80 | observeClosure(name, object) 81 | } 82 | } 83 | } 84 | } 85 | ``` 86 | 87 | ## Удаление наблюдателя 88 | 89 | Наконец, мы можем захотеть удалить класс в качестве наблюдателя, чтобы предотвратить утечки памяти. Когда экземпляр класса удаляется из числа наблюдателей, он не получает никаких уведомлений. В нашем случае это довольно просто. У нас есть NotificationStorage как контейнер, в котором хранятся все данные уведомлений, связанные на верхнем уровне по имени класса. Таким образом, когда наблюдатель желает быть удаленным, он просто передает свой экземпляр методу RemoveObserver. 90 | 91 | ```swift 92 | func removeObserver(_ _class: AnyObject) throws { 93 | let className = String(describing: _class) 94 | 95 | observers.removeValue(forKey: className) 96 | } 97 | 98 | ``` 99 | 100 | Полный код 101 | 102 | ```swift 103 | final class CustomNotificationCenter { 104 | static private(set) var shared = CustomNotificationCenter() 105 | private var observers: [String: [String: [(String, Any?) -> ()]]] = [:] 106 | 107 | private init() {} 108 | 109 | func addObserver( 110 | _ inputClass: T, 111 | name: String, 112 | closure: @escaping (String, Any) -> () 113 | ) { 114 | let className = String(describing: inputClass.self) 115 | 116 | if observers[className] != nil && observers[className]?[name] != nil { 117 | observers[className]?[name]?.append(closure) 118 | } else { 119 | observers[className] = [name: [closure]] 120 | } 121 | } 122 | 123 | func post( 124 | name: String, 125 | object: Any? = nil 126 | ) { 127 | for (_, names) in observers { 128 | for (observeName, closures) in names { 129 | guard name == observeName else { return } 130 | 131 | for observeClosure in closures { 132 | observeClosure(name, object) 133 | } 134 | } 135 | } 136 | } 137 | 138 | func removeObserver(_ _class: AnyObject) throws { 139 | let className = String(describing: _class) 140 | 141 | observers.removeValue(forKey: className) 142 | } 143 | } 144 | ``` 145 | 146 | ## Улучшения 147 | 148 | В примере выше есть две потенциальных проблемы: 149 | 150 | * Нет потокобезопасности 151 | * Проблемы с хранением неотписанных подписчиков 152 | 153 | Давайте перепишем на код лучше 154 | 155 | ## Потокобезопасность 156 | 157 | Главная проблема кода — это отсутствие потокобезопасности. Наш код будет просто крашиться 158 | 159 | Одно из решений — добавить барьер 160 | 161 | ```swift 162 | final class CustomNotificationCenter { 163 | static private(set) var shared = CustomNotificationCenter() 164 | private var observers: [String: [String: [(String, Any?) -> ()]]] = [:] 165 | private let notificationQueue = DispatchQueue( 166 | label: "ios.makes.me.hate", attributes: .concurrent 167 | ) 168 | 169 | .... 170 | 171 | ``` 172 | 173 | * Полный код 174 | ```swift 175 | final class CustomNotificationCenter { 176 | static private(set) var shared = CustomNotificationCenter() 177 | private var observers: [String: [String: [(String, Any?) -> ()]]] = [:] 178 | private let notificationQueue = DispatchQueue(label: "ios.makes.me.hate", attributes: .concurrent) 179 | 180 | private init() {} 181 | 182 | func addObserver( 183 | _ inputClass: T, 184 | name: String, 185 | closure: @escaping (String, Any) -> () 186 | ) { 187 | notificationQueue.async(flags: .barrier) { 188 | let className = String(describing: inputClass.self) 189 | 190 | if self.observers[className] != nil && self.observers[className]?[name] != nil { 191 | self.observers[className]?[name]?.append(closure) 192 | } else { 193 | self.observers[className] = [name: [closure]] 194 | } 195 | } 196 | } 197 | 198 | func post( 199 | name: String, 200 | object: Any? = nil 201 | ) { 202 | notificationQueue.async(flags: .barrier) { 203 | for (_, names) in self.observers { 204 | for (observeName, closures) in names { 205 | guard name == observeName else { return } 206 | 207 | for observeClosure in closures { 208 | observeClosure(name, object) 209 | } 210 | } 211 | } 212 | } 213 | } 214 | 215 | func removeObserver(_ _class: AnyObject) throws { 216 | notificationQueue.async(flags: .barrier) { 217 | let className = String(describing: _class) 218 | 219 | self.observers.removeValue(forKey: className) 220 | } 221 | } 222 | } 223 | ``` 224 | 225 | ## Memory Safety 226 | 227 | Нужно чтобы наш custom NC хранил подписчиков как слабую для обнуления ссылку 228 | 229 | Это поможет для: 230 | 231 | 1. Никаких сбоев 232 | 2. Нет необходимости отменять регистрацию вручную. 233 | 234 | Для начала создадим WeakObject — это объект, который будет хранить слабые ссылки 235 | 236 | ```swift 237 | struct WeakObject: Equatable, Hashable { 238 | private let identifier: ObjectIdentifier 239 | weak var object: T? 240 | init(_ object: T) { 241 | self.object = object 242 | self.identifier = ObjectIdentifier(object) 243 | } 244 | 245 | func hash(into hasher: inout Hasher) { 246 | hasher.combine(self.identifier) 247 | } 248 | 249 | static func == (lhs: WeakObject, rhs: WeakObject) -> Bool { 250 | return lhs.identifier == rhs.identifier 251 | } 252 | } 253 | ``` 254 | 255 | Также создадим WeakObjectSet — он заменит нам массив 256 | 257 | ```swift 258 | struct WeakObjectSet: Sequence { 259 | 260 | var objects: Set> 261 | 262 | init() { 263 | self.objects = Set>([]) 264 | } 265 | 266 | init(_ object: T) { 267 | self.objects = Set>([WeakObject(object)]) 268 | } 269 | 270 | init(_ objects: [T]) { 271 | self.objects = Set>(objects.map { WeakObject($0) }) 272 | } 273 | 274 | var allObjects: [T] { 275 | return objects.compactMap { $0.object } 276 | } 277 | 278 | func contains(_ object: T) -> Bool { 279 | return self.objects.contains(WeakObject(object)) 280 | } 281 | 282 | mutating func add(_ object: T) { 283 | //prevent ObjectIdentifier be reused 284 | if self.contains(object) { 285 | self.remove(object) 286 | } 287 | self.objects.insert(WeakObject(object)) 288 | } 289 | 290 | mutating func add(_ objects: [T]) { 291 | objects.forEach { self.add($0) } 292 | } 293 | 294 | mutating func remove(_ object: T) { 295 | self.objects.remove(WeakObject(object)) 296 | } 297 | 298 | mutating func remove(_ objects: [T]) { 299 | objects.forEach { self.remove($0) } 300 | } 301 | 302 | func makeIterator() -> AnyIterator { 303 | let objects = self.allObjects 304 | var index = 0 305 | return AnyIterator { 306 | defer { index += 1 } 307 | return index < objects.count ? objects[index] : nil 308 | } 309 | } 310 | } 311 | ``` 312 | 313 | Поменяем хранимые объекты в нашем NC 314 | 315 | ```swift 316 | final class CustomNotificationCenter { 317 | static private(set) var shared = CustomNotificationCenter() 318 | private var observers: [String: Any] = [:] 319 | .... 320 | } 321 | ``` 322 | 323 | И изменим методы по работе с NC 324 | 325 | 1. Меняем подписку на слушатель регистрацией 326 | 327 | ```swift 328 | func register( 329 | _ inputClass: T.Type, 330 | observer: T 331 | ) { 332 | 333 | notificationQueue.async(flags: .barrier) { 334 | let key = "\\(inputClass)" 335 | if var set = self.observers[key] as? WeakObjectSet { 336 | set.add(observer as AnyObject) 337 | self.observers[key] = set 338 | } else{ 339 | self.observers[key] = WeakObjectSet(observer as AnyObject) 340 | } 341 | } 342 | } 343 | ``` 344 | 345 | 1. А также слушатель заменяем уводемлением 346 | 347 | ```swift 348 | func notify( 349 | _ _class: T.Type, 350 | block: (T) -> Void 351 | ) { 352 | let key = "\\(_class)" 353 | var objectSet: WeakObjectSet? 354 | notificationQueue.sync { 355 | objectSet = self.observers[key] as? WeakObjectSet 356 | } 357 | 358 | guard let objects = objectSet else { return } 359 | 360 | for observer in objects { 361 | if let observer = observer as? T { 362 | block(observer) 363 | } 364 | } 365 | } 366 | ``` 367 | 368 | Как использовать? 369 | 370 | Для подписки события нам нужно создавать протокол, который будет классом слушателя 371 | 372 | ```swift 373 | protocol TestProto: AnyObject { 374 | func changeLink(test: String) 375 | } 376 | 377 | class ViewController: UIViewController, TestProto { 378 | override func viewDidLoad() { 379 | super.viewDidLoad() 380 | 381 | DispatchQueue.main.async { 382 | let vc = ViewController2() 383 | 384 | self.present(vc, animated: true) 385 | } 386 | 387 | CustomNotificationCenter.shared.register(TestProto.self, observer: self) 388 | } 389 | 390 | @objc func changeLink(test: String) { 391 | print(test) 392 | } 393 | } 394 | ``` 395 | 396 | Для нотификации этот протокол будет ключем 397 | 398 | ```swift 399 | class ViewController2: UIViewController { 400 | override func viewDidLoad() { 401 | super.viewDidLoad() 402 | 403 | 404 | CustomNotificationCenter.shared.notify(TestProto.self) { test in 405 | test.changeLink(test: "test") 406 | } 407 | } 408 | } 409 | 410 | ``` 411 | 412 | Ресурсы: 413 | 414 | [Implementing a Custom NotificationCenter Efficiently in iOS](https://betterprogramming.pub/ios-implementing-a-custom-notification-center-manually-in-the-most-efficient-way-e6b86a4bee80) 415 | 416 | [Custom Local Notification center in Swift](https://medium.com/@jeevaneashwar2/custom-local-notification-center-in-swift-2f341f1dba1c) 417 | 418 | [How to implement custom NotificationCenter in Swift](https://jayeshkawli.com/implementing-nsnotificationcenter-in-swift/) 419 | 420 | [Сделай кастомный Notification Centre для iOS приложения. Пиши код на языке...](https://www.perplexity.ai/search/Notification-Centre-iOS-wvj79s_HTEKZOc.nQCgPtQ) 421 | -------------------------------------------------------------------------------- /images/exercise-image-library-high-level-diagram.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /exercises/image-library.md: -------------------------------------------------------------------------------- 1 | # Mobile System Design Exercise: Image Library 2 | The task might be simply defined as "Design Image Library". The definition is purposely vague and requires some clarification. 3 | 4 | ## Gathering Requirements 5 | You are expected to ask clarifying questions and narrow down the scope of the task. This should not take more than 5 minutes. 6 | 7 | > **Candidate**: "Are we designing a part of an application or a general-purpose library?" 8 | > **Interviewer**: "A general-purpose library." 9 | 10 | > **Candidate**: "Are we designing a library that loads images or handles a collection of photos?" 11 | > **Interviewer**: "The one that loads images." 12 | 13 | > **Candidate**: "Where do we need to load images from?" 14 | > **Interviewer**: "I don't know - you tell me." 15 | 16 | _NOTE: some interviewers may want you to "drive" the conversation and would "subtract points" if you try to ask them what you need to do._ 17 | 18 | > **Candidate**: "Network, filesystem, and app resources would make sense. We can expose an API to create custom loaders but we could leave it out of scope for now." 19 | > **Interviewer**: "Yes, let's leave it out of scope." 20 | 21 | > **Candidate**: "Do we need to support UI-targets?" 22 | > **Interviewer**: "Not sure what you mean." 23 | > **Candidate**: "Do we want the library to load images directly to UI components (like ImageView, Button, etc)?" 24 | > **Interviewer**: "Yes." 25 | 26 | > **Candidate**: "I think, it might be beneficial to support non-UI targets - like saving the image to a file or providing a callback to access the image data when loading is complete." 27 | > **Interviewer**: "Sounds good." 28 | 29 | > **Candidate**: "I would also add a caching support to store downloaded images on the disk to preserve the bandwidth and make it easier to support offline state." 30 | > **Interviewer**: "Sounds good." 31 | 32 | > **Candidate**: "We should also provide view lifecycle management for UI-targets." 33 | > **Interviewer**: "Not sure what you mean." 34 | > **Candidate**: "That means that the library would track view hierarchy lifecycle events to decide when to stop loading and free resources. For example, you should interrupt image loading when the target view becomes detached from the view hierarchy." 35 | > **Interviewer**: "Yes, it's a good feature." 36 | 37 | > **Candidate**: "It would be good to provide thumbnails, customizable placeholders, loading indicators, and transition animation for UI-targets - but we should probably leave it out of scope." 38 | 39 | > **Interviewer**: "What do you mean by `thumbnails`?" 40 | > **Candidate**: "Many backends provide low-resolution versions of the images along with hi-resolution ones. It's useful for metered connections and slow networks. The image loader will prioritize the thumbnail loading over the high-resolution versions - this way, the host app UI would be able to do the initial rendering much faster." 41 | 42 | > **Interviewer**: "Let's leave it out of scope." 43 | 44 | Make sure not to overload the system requirements with unnecessary features. Think in terms of MVP (Minimum Viable Product) and pick features that have the biggest value. You can learn more about requirements gathering [here](https://github.com/weeeBox/mobile-system-design#gathering-requirements). 45 | 46 | ## Functional requirements 47 | - Users should be able to load images from the network, filesystem, and app resources to UI/non-UI targets. 48 | 49 | ## Non-functional requirements 50 | - Cache loaded images in memory and on the disk. 51 | - Image loading respects view hierarchy lifecycle events. 52 | 53 | ## Out of scope 54 | - Custom image loaders. 55 | - Image placeholders, loading indicators, transition animations. 56 | 57 | ## High-Level Diagram 58 | A high-level diagram shows all major system components and their interactions (without implementation details). You can learn more about high-level diagrams [here](https://github.com/weeeBox/mobile-system-design#high-level-diagram). 59 | 60 | ![High-level Diagram](/images/exercise-image-library-high-level-diagram.svg) 61 | 62 | ### Components: 63 | - **Image Request** - encapsulates a single image request; accepted by request manager as input. 64 | - **Request Manager** - accepts and dispatches image requests to corresponding targets. 65 | - **Image Cache** - fast in-memory image storage for quicker access. 66 | - **Image Loader** - coordinates image loading from different source (filesystem, app resources, network). 67 | - **Image Target** - incapsulate image target destination (UI-element, in-memory image data, etc). 68 | 69 | ## Deep Dive 70 | After a high-level discussion, your interviewer might want to discuss some specific components of the system. Make sure to keep your explanation brief and don't overload it with details - let your interviewer guide the conversation and ask questions instead. You can learn more about deep-dive discussions [here](https://github.com/weeeBox/mobile-system-design#deep-dive-tweet-feed-flow). 71 | 72 | ## Deep Dive: Image Cache 73 | > **Interviewer**: "What is the purpose of the `Image Cache` component?" 74 | > **Candidate**: "It's an in-memory LRU cache to keep a subset of loaded images for quicker access." 75 | 76 | > **Interviewer**: "What would you use for the cache key?" 77 | > **Candidate**: "It's a good question. The first option is to use the image URI. However, we might also want to distinguish between images with the same URI but different dimensions and formats. To accomplish this - we can create a new `ImageKey` type to encapsulate image parameters." 78 | 79 | ``` 80 | ImageKey: 81 | + init(source:Uri, width: Int, height: Int, format: ImageFormat) 82 | ``` 83 | 84 | > **Interviewer**: "Why would you need to include image dimensions?" 85 | > **Candidate**: "Mostly to save memory and make drawing more efficient: it's better to re-scale the image to fit its target while loading." 86 | 87 | > **Interviewer**: "What do you mean by `format`?" 88 | > **Candidate**: "The image format describes the bit encoding for every pixel (for example, `RGBA8888`, `RGB888`, `RGB565`, etc). We can optimize our memory usage by reducing the bit depth or discarding the alpha channel." 89 | 90 | > **Interviewer**: "Should this component also handle disk cache?" 91 | > **Candidate**: "The disk cache only makes sense for the images downloaded over the network. I think the network client can handle it better." 92 | 93 | > **Interviewer**: "Why do you think so?" 94 | > **Candidate**: "Most of the modern HTTP clients have built-in disk caching mechanisms that respect `Cache-Control` directives for responses." 95 | > **Candidate**: "This way, we only need to specify the cache size and the client can handle content expiration for us." 96 | 97 | > **Interviewer**: "How would you select the size of the cache?" 98 | > **Candidate**: "I would start with some default value. For example, the Google Drive app has `250`Mb disk cache size by default." 99 | > **Candidate**: "Then I would provide a library setting to override this." 100 | 101 | > **Interviewer**: "That makes sense but I was asking about in-memory cache size 🙂" 102 | > **Candidate**: "This part is harder since we can't make any strong assumptions on the memory usage patterns of the host app." 103 | > **Candidate**: "A simple heuristics could be allocating a chunk proportional to the max device memory. For example, Google [suggests](https://developer.android.com/topic/performance/graphics/cache-bitmap#memory-cache 104 | ) `1/8`th." 105 | > **Candidate**: "We can also use low-memory callbacks to purge the cache when the device runs out of memory." 106 | 107 | > **Interviewer**: "Can you see any drawbacks of using a 3rd party library?" 108 | > **Candidate**: "Depending on a 3rd party library is tricky since it can lead to semantic and binary incompatibilities with the host app. On the other hand, it handles lots of complexity for us and encapsulates the experience of many developers who worked on it. I think it's a trade-off between time-to-the-market and code ownership." 109 | 110 | _NOTE: If you don't have much experience with network stack internals - it's ok to mention a well-known library as a workaround._ 111 | 112 | ## Deep Dive: Image Loader 113 | > **Interviewer**: "Can we talk more about the `Image Loader` component?" 114 | > **Candidate**: "The component is responsible for loading images from different sources such as file system, app resources, and network." 115 | > **Candidate**: "It accepts a `Loader Request` and produces an image in return." 116 | 117 | > **Interviewer**: "What would you include in the request?" 118 | > **Candidate**: "An `ImageKey` and a callback function." 119 | 120 | ``` 121 | LoaderRequest: 122 | + init(key: ImageKey, callback: (key: ImageKey, image: Image?, error: String?) → Void) 123 | ``` 124 | 125 | _NOTE: in this exercise, the API description is purposely left platform/language agnostic to reach a broader audience. During an actual interview, you would most likely design for either iOS or Android and select the API style most suitable for the target platform. For example, you can advocate for coroutines instead of callbacks, suggest Rx-extensions, etc._ 126 | 127 | ![High-level Diagram](/images/exercise-image-library-image-loader.svg) 128 | 129 | > **Candidate**: "I would also decouple image data loading from the decoding. This way, we can easily add new decoders using the [Strategy](https://en.wikipedia.org/wiki/Strategy_pattern) design pattern without changing the loader itself." 130 | 131 | _NOTE: Android engineers might also want to mention a special [Bitmap](https://developer.android.com/reference/android/graphics/Bitmap) cache to avoid excessive garbage collection with recycled bitmaps. For more information check [BitmapPool](https://github.com/bumptech/glide/blob/master/library/src/main/java/com/bumptech/glide/load/engine/bitmap_recycle/BitmapPool.java) from Glide._ 132 | 133 | > **Interviewer**: "I'm a little confused by your caching approach - why there's no cache functionality in the Image Loader component?" 134 | > **Candidate**: "Mostly due to the [Single-responsibility principle](https://en.wikipedia.org/wiki/Single-responsibility_principle): the Image Loader should not know anything about caching - its single purpose is to ~~pass butter~~ load images." 135 | > **Candidate**: "However, the HttpClient component handles the disk cache for downloaded images. This is a trade-off between a 'clean' design and an 'ease of implementation' - might be a good short-term strategy but would scale poorly in a long run." 136 | 137 | > **Interviewer**: "What do you mean by `scaling`? Are we talking about any backend issues?" 138 | > **Candidate**: "No. By `scaling`, I mean library compatibility and long-run support. Depending on the platform, a 3rd-party dependency may have version conflicts with other libraries from the host app." 139 | > **Candidate**: "You also need to have a good plan on supporting and updating the dependency over time: for example, you need to decide on the update schedule (each release, each major release, critical bugfix-only, etc) or if you want to maintain a private fork." 140 | > **Candidate**: "There might be licensing issues for the customers, too." 141 | 142 | _NOTE: See "Chapter 21. Dependency Management" from the "Software Engineering at Google" book._ 143 | 144 | > **Interviewer**: "Sounds tough - why would you want to have the dependency in the first place?" 145 | > **Candidate**: "This a good example of `implement` vs `re-use` trade-off. Personally, I think we should prioritize time-to-market first and then refine in future releases." 146 | > **Interviewer**: "Makes sense." 147 | 148 | _NOTE: The system design interview is not all about technology and engineering - it's always good to keep business needs in mind, too._ 149 | 150 | > **Candidate**: "I forgot to mention this but we can also easily add authentication support on the network client level." 151 | > **Interviewer**: "That's fine - we can leave it out of scope." 152 | 153 | ## Deep Dive: Image Request 154 | 155 | > **Interviewer**: "Tell me more about `ImageRequest`." 156 | > **Candidate**: "It encapsulates the image loading parameters and the image target. We can use fluent chaining to create a cleaner interface." 157 | 158 | ``` 159 | ImageRequest: 160 | + load(source: Uri): ImageRequest 161 | + load(format: ImageFormat): ImageRequest 162 | + into(target: ImageView): ImageRequest // we can overload this to support multiple targets 163 | ``` 164 | 165 | > **Candidate**: "We can register lifecycle callbacks with UI-targets (ImageView, Button, etc) for easier control of resources. For example, we can stop loading when the target is detached from the view hierarchy, becomes invisible, or gets recycled as a part of list scrolling." 166 | 167 | _NOTE: We're not going to discuss the actual implementation here since it greatly depends on the UI framework. Make sure to discuss what mechanisms you would use and how to prevent memory leaks._ 168 | 169 | ## Follow-up Questions 170 | Some interviewers might ask follow-up questions that might change the original design and introduce new requirements. 171 | 172 | > **Interviewer**: "How would you change your design if you can't use the HttpClient caching?" 173 | > **Candidate**: "I would store image metadata in a relational database and store image bytes in the internal cache directory?" 174 | > **Candidate**: "Also, I would set a disk usage limit for the cache and provide a simple LRU eviction policy." 175 | 176 | > **Interviewer**: "Why would you select the internal cache directory? What other options do you have?" 177 | > **Candidate**: "The cache directory can be automatically cleaned up when the device disk space runs low. Also, the cached contents won't be backed up with the app." 178 | > **Candidate**: "I would prefer an internal storage for privacy reasons and to avoid using extra permissions (Android only)." 179 | > **Candidate**: "I don't think if I know a better storage option." 180 | 181 | _NOTE: It's ok to tell the interviewer that you don't have much experience with the storage options. Better be honest than get caught in a lie._ 182 | 183 | > **Interviewer**: "What if you need to store sensitive images?" 184 | > **Candidate**: "I would support this as a flag in the `ImageRequest` and encrypt/decrypt corresponding files using encryption keys from Android Keystore or iOS Keychain." 185 | > **Candidate**: "A **better** option is not to store sensitive images at all." 186 | 187 | ## Major Concerns and Trade-Offs 188 | 189 | ### Balancing Memory/Disk/CPU/Bandwidth usage 190 | The biggest decision to make is how to properly handle system resources utilization: 191 | - Using more memory would make the library more responsive but increases the risk of the host app being killed in the background. 192 | - Using more disk space saves application bandwidth but frequent reads/writes can increase the device temperature and increase the chances for the host app to be uninstalled in low disk space conditions. 193 | - Using more bandwidth may cost the user money and increase the battery drain. 194 | 195 | Since you can't make any assumptions on the specific use-cases - making these parameters configurable might be the preferred approach. If you are not sure what default values to use - learn from the existing applications. 196 | 197 | ### Low Data Mode (iOS) and Data Saver mode (Android) 198 | We might want to respect the device settings for saving cellular data. Possible options might include: 199 | - Block image downloading altogether. 200 | - Introduce the level of priority for each image and only download UI-critical components. See [Quality Of Service](/README.md#quality-of-service) for more information. 201 | - Provide support for hi-quality and low-quality downloads and only fetch low-quality images. 202 | - Prefer cached data instead of fetching it from the network. 203 | 204 | ## Conclusion 205 | Keep this in mind while preparing for a system design interview: 206 | - Don't try to make it perfect - provide a "signal" instead. 207 | - Listen to your interviewer and keep track of the time. 208 | - Try to cover as much ground as possible without digging too much into the implementation details. 209 | 210 | ## Looking for more content? 211 | I’m thinking about creating an in-depth mobile system design course on top of the free articles. Please, fill out this [form](https://forms.gle/KfvmZhPNPMRBE8Jj9) to express your interest! 212 | -------------------------------------------------------------------------------- /images/exercise-file-downloader-library-job-diagram.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /images/exercise-file-downloader-library-high-level-diagram.svg: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /images/exercise-file-downloader-library-deep-dive-download-dispatcher-diagram.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /exercises/caching-library.md: -------------------------------------------------------------------------------- 1 | # Mobile System Design Exercise: Caching Library 2 | The task might be simply defined as "Design Caching Library". The definition is purposely vague and requires some clarification. 3 | 4 | ## Gathering Requirements 5 | You are expected to ask clarifying questions and narrow down the scope of the task. This should not take more than 5 minutes. 6 | 7 | > **Candidate**: "Are we designing a part of an application or a general-purpose library?" 8 | > **Interviewer**: "A general-purpose library." 9 | 10 | > **Candidate**: "What kind of items are we going to cache?" 11 | > **Interviewer**: "Raw bytes." 12 | 13 | > **Candidate**: "Do we have a size limit for each entry and an upper bound for the total number of items?" 14 | > **Interviewer**: "Generally, no. But, for the sake of simplicity, let's assume image-like data up to 10 Mb per item. The max number of items is in order of thousands." 15 | 16 | > **Candidate**: "Do we need to handle sensitive data? Should we support encrypted storage?" 17 | > **Interviewer**: "Great question! What's your recommendation?" 18 | > **Candidate**: "Having access to encrypted storage might be useful for specific use-cases. However, doing it in a general case might lead to an unnecessary overhead (encryption/decryption comes with CPU and memory cost)." 19 | > **Candidate**: "We can allow turning encryption on/off on a library level or for certain specific items and let users decide based on their user-cases." 20 | > **Interviewer**: "Sounds good. Let's leave it out of scope for now." 21 | 22 | > **Candidate**: "How are we going to identify each cached item? Would string keys suffice?" 23 | > **Interviewer**: "String keys are fine." 24 | 25 | > **Candidate**: "Do we need to support an in-memory cache, a disk cache, or a hybrid memory-and-disk cache?" 26 | > **Interviewer**: "Good question. How would the 'hybrid' approach work?" 27 | > **Candidate**: "The cache would use the disk as a primary storage and keep a subset of items in memory for a quicker access." 28 | > **Interviewer**: "Let's use the 'hybrid' approach." 29 | 30 | > **Candidate**: "Should we limit the size of the cache?" 31 | > **Interviewer**: "What's your recommendation?" 32 | > **Candidate**: "I think, we should limit the number of bytes stored in memory and on the disk. Once the specific limit is reached - we should remove a portion of items according to the eviction policy." 33 | 34 | > **Interviewer**: "What eviction policy are you going to use?" 35 | > **Candidate**: "Since we're designing a general-purpose library - we should let the user select an eviction policy based on app-specific needs." 36 | > **Candidate**: "We can start with pre-defined policies: Least Recently Used (LRU), Least Frequently Used (LFU), First In First Out (FIFO), Random, etc." 37 | > **Candidate**: "We can also allow user-defined eviction policies using the [Strategy](https://en.wikipedia.org/wiki/Strategy_pattern) design pattern. Although, we should probably leave it out of scope for now." 38 | > **Interviewer**: "Sounds good. Let's leave user-defined policies out of scope." 39 | 40 | > **Interviewer**: "Do we need to share the library between multiple platforms?" 41 | > **Candidate**: "No. Let's leave it out of scope." 42 | 43 | Make sure not to overload the system requirements with unnecessary features. Think in terms of MVP (Minimum Viable Product) and pick features that have the biggest value. You can learn more about requirements gathering [here](https://github.com/weeeBox/mobile-system-design#gathering-requirements). 44 | 45 | ## Functional requirements 46 | - Users should be able to cache and retrieve raw byte data using strings as keys. 47 | - Users should be able to configure disk and memory usage limits as a part of library initialization. 48 | - Users should be able to configure the cache eviction policy as a part of library initialization. 49 | 50 | ## Non-functional requirements 51 | - The cached data should be persistent on the disk. 52 | - A small subset of items should be kept in memory for quicker access. 53 | - Once the cache is full - a portion of items should be deleted according to the eviction policy. 54 | 55 | ## Out of scope 56 | - User-defined eviction policies. 57 | - Secure item storage. 58 | - Cross-platform support. 59 | 60 | ## Client Public API (Optional) 61 | Your interviewer might want to discuss the client public API. In the simplest form it might look like this: 62 | 63 | ``` 64 | Cache: 65 | + init(config: CacheConfig) 66 | + set(key: String, value: [byte]): CacheTask 67 | + get(key: String): CacheTask 68 | + clear(key: String): CacheTask 69 | + clearAll() 70 | 71 | CacheConfig: 72 | + init(maxMemoryCacheSize: Int, maxDiskCacheSize: Int) 73 | 74 | CacheTask: 75 | + isSuccessful(): Bool 76 | + getCachedData(): [byte]? 77 | + getErrorMessage(): String? 78 | + addOnCompleteCallback(callback: (CacheTask) → Void) 79 | ``` 80 | 81 | > **Interviewer**: "Why do you return a `CacheTask` from `get` and `set` methods?" 82 | > **Candidate**: "`get` and `set` calls may result in blocking I/O operations and should not be invoked on the main thread." 83 | > **Candidate**: "We should make both of them async and let users specify completion callbacks." 84 | 85 | _NOTE_: in this exercise, the API description is purposely left platform/language agnostic to reach a broader audience. During an actual interview, you would most likely design for either iOS or Android, and select the API style most suitable for the target platform. For example, you can advocate for coroutines instead of callbacks, suggest Rx-extensions, etc. 86 | 87 | ## High-Level Diagram 88 | A high-level diagram shows all major system components and their interactions (without implementation details). You can learn more about high-level diagrams [here](https://github.com/weeeBox/mobile-system-design#high-level-diagram). 89 | 90 | ![High-level Diagram](/images/exercise-caching-library-high-level-diagram.svg) 91 | 92 | ### Components: 93 | - **Dispatcher** - coordinates async read/write operations between the client and the library. 94 | - **Journal** - maintains a structural record of metadata for cached items (access count, timestamps, size, etc). 95 | - **In-Memory Cache** - handles fast in-memory item storage for quicker sequential access. 96 | - **Persistent Store** - handles slow persistent disk item storage. 97 | - **Cache Eviction** - manages cache overflow and item eviction. 98 | 99 | ## Deep Dive 100 | After a high-level discussion, your interviewer might want to discuss some specific components of the system. Make sure to keep your explanation brief and don't overload it with details - let your interviewer guide the conversation and ask questions instead. You can learn more about deep-dive discussions [here](https://github.com/weeeBox/mobile-system-design#deep-dive-tweet-feed-flow). 101 | 102 | ## Deep Dive: Dispatcher 103 | > **Interviewer**: "I’m not sure what the Request Dispatcher component does…" 104 | > **Candidate**: "It simplifies handling read/write concurrency." 105 | > **Candidate**: "Using synchronous API calls might be a poor idea since item access might result in a blocking I/O operation. Doing so on the Main thread is not advisable and may result in jank, app termination, and _interview failures_." 106 | > **Candidate**: "To solve this issue, we need to use async API. There might be a few options (depending on the platform and the programming language) but the callback-base approach is the simplest one and can be extended to coroutines, futures, promises, etc." 107 | 108 | ![Dispatcher Sequence Diagram](/images/exercise-caching-library-dispatcher-sequence.svg) 109 | 110 | > **Candidate**: "1) Dispatcher would maintain a _limited_ pool of concurrent workers (via executor, dispatch queue, etc) to avoid creating too many background threads if the client makes a huge number of requests." 111 | > **Candidate**: "2) Each worker would perform a cache access operation and notify its dispatcher via a callback when complete." 112 | > **Candidate**: "3) After a worker completes - the dispatcher would notify the client by invoking the completion callback on the main thread (or user-specified executor/queue)." 113 | 114 | ## Deep Dive: Journal 115 | > **Interviewer**: "Would you explain how the Journal component works?" 116 | > **Candidate**: "The Journal component creates and updates metadata records for cached items." 117 | 118 |
119 | 120 | | name | type | 121 | |---------------|--------| 122 | | key | String | 123 | | size_bytes | Int | 124 | | access_count | Int | 125 | | last_accessed | Date | 126 | 127 |
128 | 129 | > **Candidate**: "Every time a cached item is accessed - the Journal will increase the `access_count` and update the `last_accessed` timestamp. This metadata would allow selecting items for eviction and calculating the total cache size." 130 | > **Candidate**: "The Journal itself could be stored in a text/binary file, property list, or a relational database (ORM). I think a relational database might be the best approach since it allows partial updates, querying, and provides data integrity." 131 | 132 | > **Interviewer**: "So the Journal would only store the metadata - where would the actual bytes be stored?" 133 | 134 | ### First Option: File System 135 | > **Candidate**: "We can write binary files in the app 'cache' directory and use generated [UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)'s as file names. The item key won't work as a filename since the user may pass illegal path characters." 136 | > **Candidate**: "One drawback of such approach - we would need to sync the Journal and the file system separately which might lead to race condition bugs. For example, the host app might crash after the Journal is updated but before the data is written to the disk." 137 | > **Candidate**: "We may overcome this by introducing items states: `CLEAN`, `DIRTY`, `REMOVE`, etc. When the item is being created or updated - it will be marked with `DIRTY` state; once the process completes - it will be marked with `CLEAN` state; same for `REMOVE` state. The library may check the Journal upon initialization and prune dirty items and their files." 138 | 139 |
140 | 141 | | name | type | 142 | |---------------|--------| 143 | | key | String | 144 | | path | String | 145 | | size_bytes | Int | 146 | | access_count | Int | 147 | | last_accessed | Date | 148 | | state | Int | 149 | 150 |
151 | 152 | > **Candidate**: "The introduction of states makes it more reliable but won't solve all potential concurrency issues. We might still run in situations where different threads are trying to write the same item concurrently. We may try to add more synchronization but it would greatly complicate the design." 153 | 154 | > **Candidate**: "The advantage of this option - the app cache (with all binary files) might be cleaned up by the user from the app settings when device storage is low." 155 | 156 | ### Second Option: BLOBs 157 | > **Candidate**: "We can store the binary data in the same database where the Journal data is stored." 158 | 159 |
160 | 161 | | name | type | 162 | |---------------|--------| 163 | | key | String | 164 | | data | BLOB | 165 | | size_bytes | Int | 166 | | access_count | Int | 167 | | last_accessed | Date | 168 | 169 |
170 | 171 | > **Candidate**: "This way we don't need to worry about synchronization and item states. We can also merge the Journal and the Persistent Store components." 172 | 173 | ![High-level Diagram](/images/exercise-caching-library-high-level-diagram-simplified.svg) 174 | 175 | _NOTE: It's ok to change some of your initial statements as the interview progresses - it's more about the thinking process and not pretty diagrams nor "making it right" on the first attempt._ 176 | 177 | > **Candidate**: "The disadvantage of this option - the binary data can't be cleaned from the app settings without removing the database itself and losing all its metadata." 178 | > **Candidate**: "There are lots of discussions around BLOBs vs file system. I don't have enough experience to take any side but I think the second option is an easier one." 179 | 180 | ## Deep Dive: Cache Eviction 181 | 182 | > **Interviewer**: "How does the Cache Eviction component work?" 183 | > **Candidate**: "When a new cache item is created - the component checks the disk and in-memory storage sizes; if any of them exceed the maximum allowed size - the Cache Eviction runs a prepared query to purge a sub-set of items." 184 | 185 | > **Interviewer**: "Why do you need a separate component and won't include the logic directly into the Journal and the In-Memory Cache?" 186 | > **Candidate**: "I'm trying to follow a [Single-responsibility principle](https://en.wikipedia.org/wiki/Single-responsibility_principle): the Journal and the In-Memory Cache components should only perform a single function and should not be aware of cache eviction existence. It's also easier to test different components in isolation." 187 | 188 | > **Interviewer**: "You've mentioned _prepared queries_ - can you provide a bit more details?" 189 | > **Candidate**: "For the Journal component, it could be a prepared SQL statement, a fetch/batch request, or any other ORM mechanism to delete multiple items. Alternatively, we can iterate using a cursor (if the vendor supports it) and delete items until the new size can accommodate an extra item." 190 | > **Candidate**: "The drawback of this approach is that the Cache Eviction components might _know_ too much about other components implementation. At the same time - this is an efficient and a simple-to-build approach. Everything is a trade-off." 191 | 192 | > **Interviewer**: "What method would you choose?" 193 | > **Candidate**: "I think, a SQL cursor might be the most efficient since we don't need to load unnecessary items." 194 | 195 | > **Interviewer**: "Do you know how cursors work under the hood?" 196 | > **Candidate**: "Not really. I only worked with them and don't know the implementation details." 197 | 198 | > **Interviewer**: "How about the In-Memory eviction?" 199 | > **Candidate**: "For the In-Memory component, it could be a custom iterator over the collection which would perform a similar function." 200 | 201 | > **Interviewer**: "What data structure would you use for the in-memory store?" 202 | > **Candidate**: "I think a self-balancing tree with a custom item comparator should work." 203 | 204 | ## Follow-up Questions 205 | Some interviewers might ask follow-up questions that might change the original design and introduce new requirements. 206 | 207 | ### Sensitive Information 208 | > **Interviewer**: "How would you change your design to support handling sensitive information?" 209 | > **Candidate**: "Is every item considered to be sensitive?" 210 | > **Interviewer**: "Not necessary." 211 | > **Candidate**: "In this case, we might want to provide users some flexibility. They can set the whole cache to be encrypted or only secure a subset of the items." 212 | 213 | > **Candidate**: "To my best knowledge, iOS/Android platforms do not provide a built-in secure database at the moment. As a workaround, we could use third-party providers (like [SQLCipher](https://www.zetetic.net/)) to _encrypt the whole database_ or only _encrypt data BLOB storage_ in the database (under a strong assumption that cache keys do not contain any sensitive information)". 214 | > **Candidate**: "For our general-case purpose, I would rather select the BLOB encryption since the vast majority of users won't store any sensitive information. Also, I'd be very cautious before adding a 3rd-party dependency into the library: it might introduce licensing issues and binary/version incompatibility with the host app." 215 | ![High-level Diagram](/images/exercise-caching-library-encryption.svg) 216 | 217 | > **Candidate**: "The encryption key would be generated and stored in the Keystore/Keychain." 218 | > **Candidate**: "We should also store an `encrypted` flag in the database to mark secure items." 219 | 220 |
221 | 222 | | name | type | 223 | |---------------|--------| 224 | | key | String | 225 | | data | BLOB | 226 | | size_bytes | Int | 227 | | access_count | Int | 228 | | last_accessed | Date | 229 | | encrypted | Bool | 230 | 231 |
232 | 233 | > **Candidate**: "As a side note, many applications already have a mature encryption stack. For this case, we might provide them the ability to specify a custom encryption/decryption implementation." 234 | 235 | ``` 236 | CacheEncryption: 237 | + encrypt(data: [Byte]): [Byte] 238 | + decrypt(data: [Byte]): [Byte] 239 | 240 | CacheConfig: 241 | + init(maxMemoryCacheSize: Int, maxDiskCacheSize: Int) 242 | + setCacheEncryption(encryption: CacheEncryption) 243 | ``` 244 | 245 | > **Candidate**: "This way they can better control user data privacy: for example, download encryption key from the backend upon login, etc." 246 | 247 | ### Cross-platform Support 248 | > **Interviewer**: "Imagine, you need to write a cross-platform library that should run on mobile and desktop platforms. How would you change your design?" 249 | > **Candidate**: "Would we need to share the cached storage somehow?" 250 | > **Interviewer**: "What do you mean?" 251 | > **Candidate**: "Should the permanent cache storage be transferred between platforms?" 252 | > **Interviewer**: "No." 253 | > **Candidate**: "In this case, we don't need storage binary compatibility". 254 | > **Candidate**: "I would separate the library into two modules: 1) common code (written in C/C++); 2) platform implementation/adapters (Java/Kotlin/Swift/Objective-C)." 255 | > **Candidate**: "The Journal, the Cache Eviction, and the Database interface would be included in the common module; the rest would be included in the implementation module." 256 | > **Candidate**: "However, I would be very cautious around using native code: 1) hard to debug; 2) hard to read crash logs; 3) need to compile for all supported architectures (arm7, arm64, x86, etc); 4) more likely to crash the host app (instead of just throwing an exception). 5) harder to develop compared to modern languages. 6) iOS/Android developers are more likely to submit pull-requests if the codebase built specifically for the target platform (assuming open-source)." 257 | 258 | ## Major Concerns and Trade-Offs 259 | 260 | ### Library responsiveness vs device resource usage 261 | The biggest decision to make is Dispatcher's worker pool size. Having a larger amount of workers might make the library more responsive but would eventually spawn more threads. 262 | 263 | ### Data security 264 | - No data is considered secure as long as its encryption key is stored on the same device. Some devices provide hardware-backed key storage but we can't guarantee this in a general case. 265 | - Encryption/decryption also comes with a CPU and memory cost: this is especially important for performance-critical cases and low-level devices. 266 | 267 | ## Conclusion 268 | - Keep this in mind while preparing for a system design interview: 269 | - Don't try to make it perfect - provide a "signal" instead. 270 | - Listen to your interviewer and keep track of the time. 271 | - Try to cover as much ground as possible without digging too much into the implementation details. 272 | 273 | ## Looking for more content? 274 | I’m thinking about creating an in-depth mobile system design course on top of the free articles. Please, fill out this [form](https://forms.gle/KfvmZhPNPMRBE8Jj9) to express your interest! 275 | -------------------------------------------------------------------------------- /images/exercise-caching-library-dispatcher-sequence.svg: -------------------------------------------------------------------------------- 1 | --------------------------------------------------------------------------------