├── .gitignore ├── ADRs ├── 001.adr-template.md ├── 002.adr-api-standard.md ├── 003.adr-build-vs-buy.md ├── 004.adr-bff.md ├── 005.adr-choosing-nosql-db.md ├── 006.adr-deployment-method.md ├── 007.adr-arch-style-np-candidate-mvp.md ├── 008.adr-upload-directly-to-document-storage.md ├── 009.adr-idp.md ├── 010.adr-queue-stream.md ├── 011.adr-cloud-providers.md ├── 012.adr-operational-data-storage.md └── README.md ├── README.md ├── design ├── design-artifacts.md └── mobile-flow.png ├── diagrams ├── .DS_Store ├── .gitignore ├── call-flows │ ├── call-flow-candidate-assignment-progress.jpg │ ├── call-flow-course-enrollment.jpg │ └── call-flow-post-notification.jpg ├── component │ └── spotlight-component-diagram.jpg ├── context │ └── spotlight-context-diagram.jpg ├── logical-arch │ └── spotlight-logical-architecture.jpg └── quanta │ ├── Reports-quanta.jpg │ ├── chat-quanta.jpg │ ├── documents-quanta.jpg │ ├── meeting-quanta.jpg │ ├── notifications-quanta.jpg │ ├── np-candidate-quanta.jpg │ ├── np-integration-quanta.jpg │ └── recommendations-quanta.jpg ├── engineering-practices ├── CI-CD.md ├── Continous-Delivery.md ├── Data-compliance-security.md ├── Feature-Toggles.md ├── Monitoring.md ├── sidecar-pattern.md └── twelve-factor.md ├── images ├── aggregate-identification.jpg ├── badge.png ├── candidate-golden-path.jpg ├── cap-pyramid.png ├── cap-theorem.png ├── capacity-planning.jpg ├── chat-quantum-arch-characteristics.jpg ├── chat-quantum-worksheet.jpg ├── cicd.png ├── cloud-pricing-comparision.png ├── document-quantum-arch-characteristics.jpg ├── documents-quantum-worksheet.png ├── meeting-quantum-arch-characteristics.jpg ├── meetings-quantum-worksheet.jpg ├── non-profit-organization-golden-path.jpg ├── notification-quantum-arch-characteristics.jpg ├── notification-quantum-worksheet.jpg ├── np-candidate-quantum-style-worksheet.jpg ├── np-candidate-quantum-worksheet.jpg ├── np-integration-quantum-style-worksheet.jpg ├── npo-integration-quantum-worksheet.jpg ├── presentation-thumbnail.png ├── quanta-identification.jpg ├── recommendations-quantum-style-worksheet.jpg ├── recommendations-quantum-worksheet.jpg ├── report-quantum-arch-characteristics.jpg ├── reports-quantum-worksheet.png └── sidecar-log-collector.png ├── other-artifacts ├── Requirement-analysis.md └── spotlight-platform-team-pegasuz.mp4 ├── other-services ├── auth-service.md ├── bff-service.md ├── infrastructure-services.md └── support-service.md └── quanta ├── chat.md ├── document.md ├── meetings.md ├── notification.md ├── npo-candidate.md ├── npo-integration.md ├── quanta-identification.md ├── recommendation.md └── reports.md /.gitignore: -------------------------------------------------------------------------------- 1 | .vscode 2 | .idea/** 3 | .DS_STORE 4 | diagrams/.DS_STORE 5 | **/.DS_STORE -------------------------------------------------------------------------------- /ADRs/001.adr-template.md: -------------------------------------------------------------------------------- 1 | # Title 2 | 3 | ## Status 4 | 5 | ## Context 6 | 7 | ### Alternatives 8 | 9 | ### PrOACT 10 | 11 | ## Decision 12 | 13 | ## Tradeoffs - Mitigations 14 | -------------------------------------------------------------------------------- /ADRs/002.adr-api-standard.md: -------------------------------------------------------------------------------- 1 | # API Standard 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | 9 | Frontend apps (web and mobile) of the spotlight platform communicate with the platform services to fetch and mutate data. Service APIs determine the interoperability between services and the apps. So, it becomes essential to choose the right API standard to support the use-cases and provide better responsiveness and usability of the apps. 10 | 11 | ### Alternatives 12 | 13 | - REST 14 | - Graphql 15 | - gRPC 16 | 17 | ### PrOACT 18 | 19 | | Criteria | REST | Graphql | gRPC | 20 | | -------------------- | -------------------------- | --------------------------------- | ----------------------------------------- | 21 | | Feasibility | high | high | low | 22 | | Latency | higher | higher | lower | 23 | | Client/app friendly | no | very much | no | 24 | | Maturity,Maintenance | matured | actively improving | actively improving | 25 | | Community | Old and large market users | growing community | growing community | 26 | | Adaptability | easily adaptable | slight friction for new consumers | less likely adaptable as it is fairly new | 27 | 28 | ## Decision 29 | 30 | **Both REST and Graphql**. 31 | 32 | - Graphql APIs solve problems of over-fetching and under-fetching, making it extremely convenient, especially for mobile apps. This will optimize performance slightly due to a smaller payload over mobile networks. Further, since Graphql is schema-based, it can help achieve type-safety with service APIs. 33 | - REST APIs are resource-oriented and have been around for decades. Third-party users or NGOs might feel it easy to integrate with the Spotlight platform using REST. 34 | 35 | ## Tradeoffs - Mitigations 36 | 37 | - Supporting multiple API standards will add an infrastructure overhead. 38 | - Mitigation - Initially, introduce Graphql on just the [backend-for-frontend(BFF)](./004.adr-bff.md) service to support the Spotlight web/mobile app. All the other platform services can start with supporting REST, and the BFF can talk to these services via REST. Long term, the platform services can support Graphql as well the existing infra can be migrated to a Graphql federation design. 39 | - Low performance (compared to gRPC), but this may not impact the use cases as there are not many services and the operations are not 40 | mission-critical (do not affect human life or money). 41 | -------------------------------------------------------------------------------- /ADRs/003.adr-build-vs-buy.md: -------------------------------------------------------------------------------- 1 | # Green field projects - Build vs Buy 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | Greenfield projects are ambiguous and risky. Shorter **feedback loop** and less **go to market** time play a key role. So, it becomes essential to invest the engineering efforts in features/verticals which contribute to the core value proposition of the product. 9 | 10 | ### Alternatives 11 | 12 | * Build in house 13 | * Buy from 3rd party vendors 14 | 15 | Factors contribution to the decision : 16 | * How close to the core value proposition is this functionality? 17 | * Does it provide a competitive advantage? 18 | * What option preserves simplicity? 19 | * What’s the fastest go-live way? 20 | * Does the team have core competency to build this? 21 | * What’s the cost of building and buying? 22 | 23 | 24 | ## Decision 25 | * We plan to buy the services such as auth, document storage, chat, notifications (in-app, push, etc.) and focus on building core features of the platform. 26 | 27 | 28 | ## Tradeoffs - Mitigation Strategies 29 | 30 | * Flexibility and customizability around the features we buy is going to be less. 31 | * A mitigation strategy for the above is having a wrapper service that would serve the missing and needed functionality. 32 | 33 | 34 | -------------------------------------------------------------------------------- /ADRs/004.adr-bff.md: -------------------------------------------------------------------------------- 1 | # Backend for frontend (BFF) with Graphql 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | Following the [API standard decision](./002.adr-api-standard.md), the Spotlight apps would require to consume graphql APIs, but external consumer services would prefer Rest APIs to integrate with the platform. It is an infrastructure overhead for all the microservices to supports both the standards initially. BFF is a service catered for frontend needs and can follow a different API standard (Graphql) from the rest of the system. 9 | 10 | Advantages of BFF pattern with Graphql 11 | 12 | * Solves problems of over fetching and under fetching. 13 | * Solve functional requirements which are different for UI from the backend. 14 | * The frontend client would only need to hit a single service to fetch aggregated data from multiple services, thus reducing the overall network overhead. 15 | * If the frontend uses typescript, this service can be managed by the frontend developers. This gives them greater flexibility to complete the UI sooner without having to rely on the delivery of the underlying application APIs. 16 | 17 | ## Decision 18 | Have a BFF service with Graphql API standard to make use of the above advantages. 19 | 20 | ## Tradeoffs - Mitigation Strategies 21 | 22 | * **Single point of failure** : If BFF is not available the UI apps will not function. 23 | * Mitigation: Deploy BFF in multiple availability zones. 24 | * **Extra service** : This will be an additional service to monitor and another opportunity for bugs in the system. 25 | * **Latency** : Increased latency due to an extra hop to fetch simple (non-aggregated) data. 26 | * Mitigation: Deploy BFF in same cluster as other microservices and communicate to them within the cluster. This will add very little round trip time. Also, BFF pattern is most effective when a page / widget on the app needs aggregated data from multiple services. 27 | 28 | ## References 29 | https://www.infoq.com/presentations/graphql-bff/ 30 | https://www.mobilelive.ca/blog/why-backend-for-frontend-application-architecture#:~:text=Backend%20For%20Frontend%20(BFF)%20architecture,%2C%20and%20native%2Dmobile%20apps. 31 | -------------------------------------------------------------------------------- /ADRs/005.adr-choosing-nosql-db.md: -------------------------------------------------------------------------------- 1 | # Choosing the right NoSQL DB 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | 9 | No SQL Databases offer high flexibility in schema and scale. Hence NoSQL is a better choice over traditional SQL databases for the spotlight platform which is explained in this [ADR](./012.adr-operational-data-storage.md) 10 | 11 | NoSQL databases are classified into 4 types. 12 | 13 | * Document stores 14 | * Graph databases 15 | * Key-value store 16 | * Wide column store 17 | 18 | ![Image](../images/cap-theorem.png) 19 | credits : https://www.researchgate.net/figure/CAP-theorem-with-databases-that-choose-CA-CP-and-AP_fig1_282519669 20 | 21 | 22 | 23 | ## Decision 24 | 25 | - As there won't be too many writes, a document-based store would be a better choice. 26 | - the document data stores offer data partitioning, which would help as the platform expands toward different regions. 27 | - Key-value store would be helpful in use cases of caches, where aggregate queries are not required. 28 | 29 | ## Reference 30 | 31 | - https://www.bmc.com/blogs/cap-theorem/#:~:text=The%20CAP%20theorem%20is%20a,or%20availability%E2%80%94but%20not%20both 32 | -------------------------------------------------------------------------------- /ADRs/006.adr-deployment-method.md: -------------------------------------------------------------------------------- 1 | # Deployment methods - Containers Vs Serverless 2 | Date: 2022-06-21 3 | 4 | ## Status 5 | Proposed 6 | 7 | ## Context 8 | Deployment methods depict how a software service is deployed into a cloud or a machine to serve its users. It is an important aspect of software architecture as it impacts cost, performance, availability of the software. 9 | 10 | ### Alternatives 11 | * Containers 12 | * Serverless 13 | 14 | ### PrOACT 15 | 16 | | Criteria | Containers | Serverless | 17 | | ----------- | ----------- | ----------- | 18 | | Cost | High | low | 19 | | Maintenance | high | low | 20 | | Portability | high, environment agnostic | Coupled with the environment. 21 | | Feasibility(time) | large | quick | 22 | | Scalability | high(with auto scaling pods) | high 23 | | Availability | high | low (cold-start) 24 | | Latency | high | low 25 | | Security | high | low 26 | 27 | 28 | ## Decision 29 | **Use either containers or serverless where necessary**. 30 | * The spotlight platform follows distributed architecture styles (event-driven, micro-services, microkernal) for various architectural quanta. So, both the methods can be used for their respective use-cases. 31 | * Considering the cost and maintenance overhead of virtual containers, some services which do not require high availability can be deployed into serverless architecture. 32 | * Services that need to be highly available and responsive could be in virtualized containers. 33 | 34 | 35 | ## Tradeoffs - Mitigation Strategies 36 | * Going serverless may affect availability of services while going with virtualized containers can add cost and maintenance overhead. And, as identified in [Requirement Analysis](../README.md#requirement-analysis), feasibility (cost) can be a concern. 37 | * Consider using managed deployment services (like AWS ECS or Google Cloud Run, depending on the chosen cloud provider) to mitigate operational overhead of scaling, patching, securing, and managing servers. 38 | * The platform developers would need to have subject knowledge of both the deployment styles. 39 | 40 | ## References 41 | https://www.datadoghq.com/knowledge-center/serverless-architecture/ 42 | -------------------------------------------------------------------------------- /ADRs/007.adr-arch-style-np-candidate-mvp.md: -------------------------------------------------------------------------------- 1 | # Arch style for NP candidate quanta 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Proposed 6 | 7 | ## Context 8 | Np candidate quanta consists of the core services in our architecture 9 | ![Image](../diagrams/quanta/np-candidate-quanta.jpg) 10 | 11 | Spotlight platform is a greenfield project. It comes with ambiguities and risks. 12 | 13 | ### Alternatives 14 | 15 | * Modular Monolith 16 | * Microservice 17 | 18 | ### PrOACT 19 | 20 | | Factor | Modular Monolith | Microservice | 21 | | ----------- | ----------- | ----------- | 22 | | Cost(time+money) | Low | High | 23 | | Maintenance | Low | High | 24 | | Coupling | High | Low 25 | | Scalability | Low | High | 26 | | Extensibility | Low | High 27 | 28 | ## Decision 29 | * **Modular Monolith** if there is a cost constraint. Start with the monolith and later migrate to microservice + event driven after there is a product market fit and a system required to scale. 30 | 31 | 32 | * **Microservice + Event Driven** If there is no cost constraint 33 | 34 | 35 | ## References 36 | 37 | [Monolith vs micorservices](https://www.geeksforgeeks.org/monolithic-vs-microservices-architecture/) -------------------------------------------------------------------------------- /ADRs/008.adr-upload-directly-to-document-storage.md: -------------------------------------------------------------------------------- 1 | # Upload to 3rd party cloud storage directly from the web or mobile application 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | The [document quanta](../quanta/document.md) is responsible for storing and retrieving documents in the spotlight platform. Question arises 9 | 10 | ### Alternatives 11 | * Uploading to cloud storage via document service 12 | * Uploading directly to 3rd party cloud storage provider from frontend 13 | 14 | ### PrOACT 15 | 16 | | Criteria | via Document Service | Directly to cloud storage | 17 | | ----------- | ----------- | ----------- | 18 | | Time to build/integrate | Very high |minimal| 19 | | Cost involved | Build + maintain | pay as you go | 20 | | Reliability | Low - because service memory will bloat un predictably based on size of document uploaded | High| 21 | | Maintenance and monitoring overhead | high | no overhead| 22 | | Latency | high | medium / low | 23 | | Security | high - because authN and authZ would be there on document service | low | 24 | 25 | 26 | ## Decision 27 | Uploading directly to 3rd party cloud storage provider from frontend 28 | 29 | ## Tradeoffs - Mitigation Strategies 30 | * Uploading directly to 3rd party cloud storage provider from frontend may pose security risk of users uploading malicious files (too large, invalid formats, etc.) and retrieving files without sufficient authorizations. 31 | * Mitigation - Use presigned or short-lived URLs to upload/retrieve files to/from 3rd party cloud storage provider. One can specify the file size, format, validity, etc. in the URL. 32 | 33 | 34 | ## References 35 | * https://aws.amazon.com/blogs/compute/uploading-to-amazon-s3-directly-from-a-web-or-mobile-application/ 36 | -------------------------------------------------------------------------------- /ADRs/009.adr-idp.md: -------------------------------------------------------------------------------- 1 | # Identity Provider (IdP) 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | 9 | Identity and access control is a critical piece of the spotlight app since it will be part of every use-case for the user. However, building a robust identity management system comes with its challenges around security, compliance, management, and monitoring. 10 | 11 | ### Alternatives 12 | 13 | - Building an identity service in-house. 14 | - Using open-source self-hosted identity platforms like Keycloak, Apache LDAP 15 | - Using a fully managed 3rd party identity platform like Auth0, Okta. 16 | 17 | ### PrOACT 18 | 19 | | Criteria | Self-built identity service | Self-hosted,pre-built identity platform | 3rd party platform | 20 | | ----------------------------------- | --------------------------- | --------------------------------------- | ------------------ | 21 | | Time to build/integrate | Very high | low | low | 22 | | Cost involved | Build + maintain | maintain | pay as you go | 23 | | Maintenance and monitoring overhead | high | high | no overhead | 24 | | Flexibility | high | medium | low | 25 | 26 | ## Decision 27 | 28 | **Leverage 3rd party fully managed identity solutions** 29 | Building or managing an identity service is expensive and adds the overhead of security and compliance. Using a 3rd party vendor will enable a faster go-to-market and allows engineering teams to focus on the key valye proposition of the product. The pay-as-you-go plans which are available with most of the vendors will save the cost and overhead of developing, monitoring, and maintaining other options. 30 | 31 | ## Tradeoffs - Mitigations 32 | 33 | Choosing a 3rd party vendor will limit us to building for capabilities it does not offer. 34 | 35 | A possible mitigation is to have a wrapper service around this vendor that would 36 | 37 | - behave as an identity facade for the rest of the system 38 | - Build capabilities that the identity provider does not support 39 | -------------------------------------------------------------------------------- /ADRs/010.adr-queue-stream.md: -------------------------------------------------------------------------------- 1 | # Choosing Message queue vs Event stream 2 | 3 | ## Status 4 | Accepted 5 | 6 | ## Context 7 | Spotlight platform being an event driven system, queue/stream becomes a vital component of the architecture. Platform would need a long term reliable solution to scale for its use cases and build an event sourcing system. 8 | 9 | ### Alternatives 10 | * Message queue 11 | * Event streams 12 | 13 | 14 | ### PrOACT 15 | 16 | | Criteria | Message queue | Event stream | 17 | | ----------- | ----------- | ----------- | ----------- 18 | | Reliability | low |High, order is preserved 19 | | Delivery count | once | every consumer have their offset 20 | | Retention | no retention of messages | retained for a certain period | 21 | | Cost | low | high | 22 | Maintenance overhead | high | less| 23 | 24 | 25 | ## Decision 26 | **Event stream** 27 | considering use cases of spotlight platform with use cases of event to be delivered to multiple services (one such example is event candidate_ assignment_updated would be subscribed by candidate service and store the event for analytics ) which would not be offered by a message queue. 28 | 29 | ## Trade-offs 30 | * Cost - Streams are costly compared to queues, however given the advantages of management and reliability it is something that can be lived with. 31 | 32 | ## References 33 | http://docs.eventide-project.org/core-concepts/streams/streams-vs-queues.html -------------------------------------------------------------------------------- /ADRs/011.adr-cloud-providers.md: -------------------------------------------------------------------------------- 1 | # Cloud Provider 2 | Date: 2022-06-23 3 | 4 | ## Status 5 | Accepted 6 | 7 | ## Context 8 | 9 | There is a myriad of cloud providers offering a wide range of services. It’s necessary to evaluate each provider and choose the one that best suits the Spotlight platform's use cases. 10 | 11 | ### Options 12 | 13 | - Amazon Web Services 14 | - Google Cloud Platform 15 | - Microsoft Azure 16 | 17 | ### PrOACT 18 | 19 | ![Image](../images/cloud-pricing-comparision.png) 20 | 21 | ## Decision 22 | 23 | - All three cloud providers are closely placed with the offerings & pricing. AWS has edge over rest being the market leader and having extensive community support. 24 | 25 | ## Reference 26 | 27 | - https://calculator.aws/#/ 28 | - https://cloud.google.com/products/calculator 29 | - https://azure.microsoft.com/en-in/pricing/calculator/ 30 | -------------------------------------------------------------------------------- /ADRs/012.adr-operational-data-storage.md: -------------------------------------------------------------------------------- 1 | # Operational Data Storage 2 | 3 | ## Status 4 | Accepted 5 | 6 | ## Context 7 | 8 | With the growth of Microservices, Cloud, Distributed Applications, Global Scaling, and Semi-Structured Data, it is more vital than ever to select a database since it affects the entire performance and cost of the system. 9 | 10 | ### Alternatives 11 | 12 | - RDBMS 13 | - NoSQL 14 | 15 | ### PrOACT 16 | 17 | | Criteria | RDBMS | NoSQL | 18 | | ------------------------------ | ------ | ----- | 19 | | Schema Flexibility | Low | High | 20 | | Scalability | Medium | High | 21 | | Availability | Medium | High | 22 | | Data Consistency | High | Low | 23 | | Write Performance Requirements | Low | High | 24 | | Cost | High | Low | 25 | 26 | ![Image](../images/cap-theorem.png) 27 | credits : https://www.researchgate.net/figure/CAP-theorem-with-databases-that-choose-CA-CP-and-AP_fig1_282519669 28 | 29 | ![Image](../images/cap-pyramid.png) 30 | credits : https://www.researchgate.net/figure/CAP-theorem-with-databases-that-choose-CA-CP-and-AP_fig1_282519669 31 | 32 | 33 | ## Decision 34 | 35 | - NoSQL over RDBMS since scalability is desired. 36 | - Schema flexibility for a green field project is important due to ambiguity and the evolution of the system in the initial phases. 37 | 38 | 39 | ## Reference 40 | 41 | - https://www.bmc.com/blogs/cap-theorem/#:~:text=The%20CAP%20theorem%20is%20a,or%20availability%E2%80%94but%20not%20both 42 | -------------------------------------------------------------------------------- /ADRs/README.md: -------------------------------------------------------------------------------- 1 | # Architecture Decision Records 2 | 3 | [001 We are using ADRs (template)](./001.adr-template.md) 4 | 5 | [002 API Standard](./002.adr-api-standard.md) 6 | 7 | [003 Green Field Projects - Build vs Buy](003.adr-build-vs-buy.md) 8 | 9 | [004 Backend for frontend (BFF) with Graphql](004.adr-bff.md) 10 | 11 | [005 Choosing the right NoSQL DB](005.adr-choosing-nosql-db.md) 12 | 13 | [006 Deployment methods - Containers Vs Serverless](006.adr-deployment-method.md) 14 | 15 | [007 Arch Style for NP Candidate Quantum](007.adr-arch-style-np-candidate-mvp.md) 16 | 17 | [008 Upload to 3rd party cloud storage directly from frontend](008.adr-upload-directly-to-document-storage.md) 18 | 19 | [009 Identity Provider (IdP)](009.adr-idp.md) 20 | 21 | [010 Choosing Message queue vs Event stream](010.adr-queue-stream.md) 22 | 23 | [011 Cloud Provider](011.adr-cloud-providers.md) 24 | 25 | [012 Operational Data Storage](012.adr-operational-data-storage.md) 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # O'reilly Architectural Katas - Spring 2022 - Spotlight Platform 2 | 3 | ## Update 4 | Thank you judges for the valuable feedback and the entire O'reilly team for organizing the event. It was a great learning experience just to be part of the event and looking at the perspectives of all the participating teams. 5 | 6 | 7 | **Winners (First Place) - O'Reilly Architecture Katas Spring 2022** 8 | 9 | 10 | ## Final Presentation 11 | [![](./images/presentation-thumbnail.png)](https://www.loom.com/share/bd6aaab2875540f68924baab38dabfbd) 12 | 13 | [Loom Link](https://www.loom.com/share/bd6aaab2875540f68924baab38dabfbd) 14 | 15 | 16 | [Prezi Link](https://prezi.com/v/view/QLsX6RaSORYOurrrfhnL/) 17 | 18 | 19 | ## Table of Contents 20 | * [About Team PegasuZ](#about-the-team) 21 | * [Glossary](#glossary) 22 | * [Prelude](#prelude) 23 | * [Non-functional requirements](#non-functional-requirements) 24 | * [Overall Platform Context](#context-diagram) 25 | * [User Experience](#user-experience) 26 | * [Candidate Golden Path](#candidate-flow) 27 | * [NPO Golden Path](#non-profit-flow) 28 | * [Assumptions](#assumptions) 29 | * [User Roles](#user-roles) 30 | * [Identifying Architectural Quanta](#identifying-architectural-quanta) 31 | * [Quanta](#quanta) 32 | * [Other Services](#other-services) 33 | * [Overall Architecture](#overall-architecture) 34 | * [Logical View](#logical-view) 35 | * [Component View](#component-view) 36 | * [Platform Road map](#platform-roadmap) 37 | * [Engineering Practices](#engineering-practices) 38 | * [Resources](#resources) 39 | 40 | 41 | ## About Team PegasuZ 42 | 43 | We are a passionate group of software engineers & product managers from an innovation as a service organization [Zemoso](https://www.zemosolabs.com/). 44 | 45 | * [Pranava Shashank P](https://www.linkedin.com/in/pranavashashank/) (Part of TeamZ which got an honorable mention in Architectural Katas April 2021 - [Repo](https://github.com/z-katas/sysops-squad)) 46 | * [Naveen Chevuru](https://www.linkedin.com/in/naveenchevuru/) 47 | * [Ashish Kumar Das](https://www.linkedin.com/in/das-ashish/) 48 | * [Sai Venkatesh Vemuri](https://www.linkedin.com/in/saivenkateshvemuri/) 49 | * [Vasanth Kumar](https://www.linkedin.com/in/vasanth-kumar-22b9a4109/) 50 | 51 | `PegasuZ = [Pegasu]s + [Z]emoso` 52 | 53 | The name PegasuZ is a combination of the words "Pegasus" (which reflects progression) and "Zemoso" (since the team members are from Zemoso). 54 | 55 | We felt that the Spotlight platform would enable Candidates to progress in their career and hence we went with the mythical creature, "Pegasus". 56 | 57 | 58 | 59 | ## Glossary 60 | * NPO / NP - Both acronyms are used to refer to non-profit organizations. 61 | * NFR - Non-functional requirement 62 | * MVP - Minimum Viable Product 63 | * PII - Personal Identifiable Information 64 | * UI - User Interface 65 | * UX - User Experience 66 | * IA - [Information Architecture](https://en.wikipedia.org/wiki/Information_architecture) 67 | 68 | 69 | 70 | ## Prelude 71 | There are about 1.6 million non-profit organizations [[Source](https://independentsector.org/about/the-charitable-sector/)] and more than 100 million underrepresented people [[Source](https://www.governing.com/archive/gov-nonprofits.html)] in the US. But the problem today is that these NPOs are decentralized and there is no support / framework for active collaboration. Also, lack of visibility of the NPOs creates a barrier of access to the underrepresented demographics. 72 | 73 | Let's imagine a use case - A candidate plans to enrol for a course in an institution in **City A**, away from his home city. He would need to travel to **City A**, find an affordable place to stay, search for places to get groceries and food, etc. Also, after completion of the course, he would need to prepare for interviews and get placed. Without a centralized system and collaboration between NPOs, the candidate may end up spending a lot more than what he can afford and his career path / shift may not be feasible. 74 | 75 | The number of NPOs and underrepresented people mentioned above are quite large and provide tremendous opportunities to build a platform to enable the collaboration. 76 | 77 | Diversity Cyber Council has come forward with a vision to enhance inclusion and representation in the tech industry through training, mentoring, networking, and visibility programs. 78 | 79 | ### Goal of the platform 80 | To establish a sustainable and diverse talent pipeline that extends career equity to underrepresented demographics by providing access to competent training programs that lead to direct employment opportunities. 81 | 82 | 83 | ## Non-Functional Requirements 84 | 85 | After the **[detailed analysis of business requirements](./other-artifacts/Requirement-analysis.md)**, the team has come up with the below NFRs for the platform 86 | 87 | * Workflow 88 | * Evolutionary 89 | * Feasibility 90 | * Scalability 91 | * Usability 92 | * Availability 93 | * Data Integrity 94 | 95 | Since the proposal is a platform, the platform could be composed of several architectural quanta, each with its own architectural style. So, we are not picking a style yet. 96 | 97 | 98 | ## Overall Platform Context 99 | 100 | ![Image](./diagrams/context/spotlight-context-diagram.jpg) 101 | 102 | [For better navigation on the content, use this miro frame](https://miro.com/app/board/uXjVOv-nlBo=/?moveToWidget=3458764528112682339&cot=14) 103 | 104 | 105 | ## User experience 106 | The team went through a design thinking exercise to understand the Candidate and NP user profiles, and empathize their needs and pain points. The following golden paths were considered: 107 | 108 | [Design artifacts](./design/design-artifacts.md) 109 | 110 | 111 | ### Candidate Golden Path 112 | 113 | ![Image](./images/candidate-golden-path.jpg) 114 | 115 | https://www.loom.com/share/7d1540ecfed24c0191f7087d7a747260 116 | 117 | 118 | ### Non-Profit Golden Path 119 | 120 | ![Image](./images/non-profit-organization-golden-path.jpg) 121 | 122 | https://www.loom.com/share/2f3be1ded8bd438e958ffd8cc9595e83 123 | 124 | 125 | ## Assumptions 126 | ### Capacity planning 127 | Going by the no. of NPOs and under represented demographics mentioned in [Prelude](#prelude), we decided to design the system for the following capacity: 128 | 129 | | Time | Candidates | NPOs | Locations | 130 | | ----------- | ----------- | ---------- | ------------------ | 131 | | 3 months | 5k-10k | 500-1k | 1 State - 1 City | 132 | | 6 months | 10k - 50k | 1k - 5k | 1 State - 1 City | 133 | | 1 year | 50k - 100k | 5k - 20k | US 2-3 States (Region) | 134 | | 2 years | ~ 500k | 25k - 50k | US Multiple States (Regions) | 135 | | 3 years | ~1 mil | ~ 100k | US Multiple States (Regions) | 136 | 137 | ### Availability 138 | Since Spotlight is not a public safety or mission critical (dealing with lives or money directly) platform, 5 nines (99.999% - 5.2 minutes of downtime per year) is unreasonable and 4 nines would also be an overkill (99.99% - 52.5 minutes of downtime per year). 139 | 140 | So, we decided to design the system for an availability between 3 nines (for quanta which do not have synchronous dependencies, like reporting and analytics) and 4 nines (for quanta which have synchronous dependencies, like notifications). 141 | 142 | [Reference](https://aws.amazon.com/blogs/publicsector/achieving-five-nines-cloud-justice-public-safety/#:~:text=The%20accepted%20availability%20standard%20for,system%20must%20work%20seamlessly%20together.) 143 | 144 | 145 | ## User Roles 146 | * PLATFORM_ADMIN (associated with the Spotlight Platform) 147 | * COMMUNITY_LEADER (associated with the Spotlight Platform) 148 | * NP_ADMIN (associated with the NPO) 149 | * NP_USER (associated with the NPO) - Can be NPO offering's mentor 150 | * CANDIDATE 151 | 152 | 153 | ## Identifying Architectural Quanta 154 | **Architecture quantum** - *An independently deployable artifact with high functional cohesion and synchronous connascence* 155 | 156 | Quanta identification helps in defining different parts of the platform and the scope of architectural characteristics. We followed a mixture of event storming + actor-action approach to identify aggregates, components and quanta. 157 | 158 | [Click here for more details on the exercise.](./quanta/quanta-identification.md) 159 | 160 | 161 | 162 | ### Quanta 163 | * [NPO - Candidate](./quanta/npo-candidate.md) 164 | * [Reports](./quanta/reports.md) 165 | * [Notification](./quanta/notification.md) 166 | * [Recommendations](./quanta/recommendation.md) 167 | * [Chat](./quanta/chat.md) 168 | * [Meetings](./quanta/meetings.md) 169 | * [Document](./quanta/document.md) 170 | * [NPO Integrations](./quanta/npo-integration.md) 171 | 172 | 173 | 174 | ### Other Services 175 | * [Infrastructure Services](./other-services/infrastructure-services.md) 176 | * [Support](./other-services/support-service.md) 177 | * [BFF](./other-services/bff-service.md) 178 | 179 | 180 | 181 | ## Overall Architecture 182 | 183 | 184 | ### Logical View 185 | ![Image](./diagrams/logical-arch/spotlight-logical-architecture.jpg) 186 | 187 | 188 | 189 | ### Physical View 190 | ![Image](./diagrams/component/spotlight-component-diagram.jpg) 191 | 192 | [For better navigation on the content, use this miro frame](https://miro.com/app/board/uXjVOv-nlBo=/?moveToWidget=3458764528112682338&cot=14) 193 | 194 | 195 | ## Platform Roadmap 196 | 197 | ### MVP 198 | The MVP is envisioned to bring out the unique value proposition of the platform and solve the key business problem / pain points. It should be sufficient for the business to get market feedback and pivot, if needed. 199 | 200 | Areas to be addressed (In the order of priority): 201 | 202 | 1. NPO - Candidate quantum (Modular Monolith) 203 | 2. Base platform infrastructure 204 | * CI / CD pipelines with continuous delivery. 205 | * Logging 206 | * Monitoring 207 | 3. BFF Service 208 | 4. Document quantum 209 | 5. Chat quantum 210 | 6. Notification quantum 211 | 7. Meetings quantum 212 | 8. Reports quantum - Without Analytics 213 | 9. Recommendations quantum - Simple, without model training 214 | 215 | 216 | 217 | ### Long Term 218 | Areas to be addressed: 219 | * NPO - Candidate quantum (migrate to Microservices + Event Driven) 220 | * Reports quantum - Advanced predictive analytics 221 | * Recommendations quantum - with model training 222 | * NPO Integrations 223 | * Support Service 224 | 225 | 226 | 227 | ## Engineering Practices 228 | 229 | * [Continuous Integrations and Continuous Deployment ](./engineering-practices/CI-CD.md) 230 | * [Continuous Delivery](./engineering-practices/Continous-Delivery.md) 231 | * [Data compliance and Security](./engineering-practices/Data-compliance-security.md) 232 | * [Twelve Factor app](./engineering-practices/twelve-factor.md) 233 | * [Sidecar Pattern](./engineering-practices/sidecar-pattern.md) 234 | * [Monitoring and Observability](./engineering-practices/Monitoring.md) 235 | * [Feature Toggles](./engineering-practices/Feature-Toggles.md) 236 | 237 | 238 | 239 | ## Resources 240 | * [Design Thinking](https://miro.com/app/board/uXjVOv-nLZE=/?share_link_id=379517243423) 241 | * [Arch Katas Artifacts](https://miro.com/app/board/uXjVOv-nlBo=/?share_link_id=836537483968) 242 | -------------------------------------------------------------------------------- /design/design-artifacts.md: -------------------------------------------------------------------------------- 1 | ## Design Artifacts 2 | 3 | ### Candidate 4 | 5 | Figma - https://www.figma.com/file/mVJ0b61exFSiuDktdLuYHm/Spotlight?node-id=124%3A12582 6 | 7 | Prototype - https://www.figma.com/proto/mVJ0b61exFSiuDktdLuYHm/Spotlight?node-id=205%3A20859&scaling=scale-down&page-id=124%3A12582&starting-point-node-id=205%3A20859 8 | 9 | Loom - https://www.loom.com/share/7d1540ecfed24c0191f7087d7a747260 10 | 11 | 12 | 13 | ### NPO 14 | 15 | Figma - https://www.figma.com/file/mVJ0b61exFSiuDktdLuYHm/Spotlight?node-id=213%3A4805 16 | 17 | Prototype - https://www.figma.com/proto/mVJ0b61exFSiuDktdLuYHm/Spotlight?node-id=213%3A4807&scaling=scale-down&page-id=213%3A4805&starting-point-node-id=213%3A4807 18 | 19 | Loom - https://www.loom.com/share/2f3be1ded8bd438e958ffd8cc9595e83 -------------------------------------------------------------------------------- /design/mobile-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/design/mobile-flow.png -------------------------------------------------------------------------------- /diagrams/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/.DS_Store -------------------------------------------------------------------------------- /diagrams/.gitignore: -------------------------------------------------------------------------------- 1 | .vscode -------------------------------------------------------------------------------- /diagrams/call-flows/call-flow-candidate-assignment-progress.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/call-flows/call-flow-candidate-assignment-progress.jpg -------------------------------------------------------------------------------- /diagrams/call-flows/call-flow-course-enrollment.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/call-flows/call-flow-course-enrollment.jpg -------------------------------------------------------------------------------- /diagrams/call-flows/call-flow-post-notification.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/call-flows/call-flow-post-notification.jpg -------------------------------------------------------------------------------- /diagrams/component/spotlight-component-diagram.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/component/spotlight-component-diagram.jpg -------------------------------------------------------------------------------- /diagrams/context/spotlight-context-diagram.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/context/spotlight-context-diagram.jpg -------------------------------------------------------------------------------- /diagrams/logical-arch/spotlight-logical-architecture.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/logical-arch/spotlight-logical-architecture.jpg -------------------------------------------------------------------------------- /diagrams/quanta/Reports-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/Reports-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/chat-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/chat-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/documents-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/documents-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/meeting-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/meeting-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/notifications-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/notifications-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/np-candidate-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/np-candidate-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/np-integration-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/np-integration-quanta.jpg -------------------------------------------------------------------------------- /diagrams/quanta/recommendations-quanta.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/diagrams/quanta/recommendations-quanta.jpg -------------------------------------------------------------------------------- /engineering-practices/CI-CD.md: -------------------------------------------------------------------------------- 1 | # CICD practices 2 | 3 | Continuous integration and continuous deployment must be an integral part of engineering culture. 4 | 5 | 6 | ![alt text](../images/cicd.png) 7 | credits : src : https://www.synopsys.com/glossary/what-is-cicd.html 8 | 9 | 10 | ## Continuous Integration Practices 11 | 12 | * Linting and format checks for code consistency. 13 | * Code should be built ensuring no static and compile-time errors. 14 | * Unit tested to ensure there are no functional bugs. 15 | * Code quality checks to ensure bad code stays away from the main branch. 16 | 17 | ## Continuous Deployment Practices 18 | 19 | * Code ships automatically when merged to the main branch 20 | 21 | 22 | ## References 23 | https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment -------------------------------------------------------------------------------- /engineering-practices/Continous-Delivery.md: -------------------------------------------------------------------------------- 1 | # Continuous Delivery 2 | Continuous delivery is an extension of continuous integration since it automatically deploys all code changes to a testing and/or production environment after the build stage. 3 | 4 | 5 | ## Continuous Delivery Practices 6 | 7 | * Features are smoke tested(automation tests) in lower environments before promoting to higher environments. 8 | * Code is built only once and deployed to many environments ensuring the same build is shipped, reducing disparity in environments. 9 | * Feature flags are present to hide features that are not intended to be released in an environment. 10 | 11 | 12 | ## References 13 | https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment -------------------------------------------------------------------------------- /engineering-practices/Data-compliance-security.md: -------------------------------------------------------------------------------- 1 | # Data compliance and security 2 | 3 | The system deals with PII data(emails, contact details) which are sensitive. Compliance and security become a core part of the data requirements. 4 | 5 | ## Measures to be followed 6 | 7 | * Store no PII data in our system and store them entirely in a 3rd party system that is compliant. 8 | * If the system needs to store any such information, use a data-compliant database and cloud provider. 9 | * Encrypt the data at rest and transit. 10 | * Ensure regular security scans and alerts in the system. -------------------------------------------------------------------------------- /engineering-practices/Feature-Toggles.md: -------------------------------------------------------------------------------- 1 | # Feature Toggles(Feature flags) 2 | 3 | It is a technique that turns a functionality on/off at runtime. This toggle does not need code changes and deployment, giving code owners more control and flexibility to iterate. 4 | 5 | ## Benefits: 6 | 7 | * A/B testing 8 | * Production testing 9 | * Canary Release 10 | * Rollback switch 11 | * Trunk based deployment 12 | * Cross service switches - One switch toggles the feature for all the services 13 | 14 | References : 15 | 16 | https://www.optimizely.com/optimization-glossary/feature-rollout/ -------------------------------------------------------------------------------- /engineering-practices/Monitoring.md: -------------------------------------------------------------------------------- 1 | # Observability and Monitoring 2 | 3 | Each service in the system should produce appropriate data to support detection and alerting. Each service should be observable. 4 | 5 | It is the responsibility of each service to 6 | 7 | - Perform health checks 8 | - Generate Metrics 9 | - Log entries 10 | - Request tracing 11 | 12 | To achieve the above it is recommended to follow the [sidecar pattern](./sidecar-pattern.md) 13 | 14 | ## References 15 | 16 | https://cloud.ibm.com/docs/java?topic=cloud-native-observability-cn 17 | -------------------------------------------------------------------------------- /engineering-practices/sidecar-pattern.md: -------------------------------------------------------------------------------- 1 | #Sidecar Pattern 2 | 3 | ## Context 4 | Applications and services often require related functionality, such as monitoring, logging, etc. These peripheral tasks can be implemented as separate components or services. This means they are not well isolated, and an outage in one of these components can affect other components or the entire application. Also, they usually need to be implemented using the same language as the parent application. As a result, the component and the application have close interdependence on each other. 5 | 6 | ## Sidecar Pattern 7 | 8 | * The sidecar is a component of the application that is attached to a parent application and provides supporting features. 9 | * The sidecar also shares the same lifecycle as the parent application, is created and retired alongside the parent. 10 | 11 | ![alt text](../images/sidecar-log-collector.png) 12 | credits: https://dev.to/peterj/sidecar-container-pattern-314 13 | 14 | ## Usecases 15 | 16 | * Your primary application uses a heterogeneous set of languages and frameworks. 17 | * A component is owned by a remote team or a different organization. 18 | * A component or feature must be co-located on the same host as the application 19 | 20 | 21 | 22 | ##References 23 | https://www.oreilly.com/library/view/designing-distributed-systems/9781491983638/ch02.html 24 | https://netflixtechblog.com/how-netflix-uses-ebpf-flow-logs-at-scale-for-network-insight-e3ea997dca96 -------------------------------------------------------------------------------- /engineering-practices/twelve-factor.md: -------------------------------------------------------------------------------- 1 | # Twelve-factor app 2 | 3 | The twelve-factor methodology helps us follow best practices and build applications that are portable and resilient. It talks about 12 key things that must be present in a Saas system 4 | 5 | ## References 6 | https://12factor.net/ 7 | -------------------------------------------------------------------------------- /images/aggregate-identification.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/aggregate-identification.jpg -------------------------------------------------------------------------------- /images/badge.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/badge.png -------------------------------------------------------------------------------- /images/candidate-golden-path.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/candidate-golden-path.jpg -------------------------------------------------------------------------------- /images/cap-pyramid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/cap-pyramid.png -------------------------------------------------------------------------------- /images/cap-theorem.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/cap-theorem.png -------------------------------------------------------------------------------- /images/capacity-planning.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/capacity-planning.jpg -------------------------------------------------------------------------------- /images/chat-quantum-arch-characteristics.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/chat-quantum-arch-characteristics.jpg -------------------------------------------------------------------------------- /images/chat-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/chat-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/cicd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/cicd.png -------------------------------------------------------------------------------- /images/cloud-pricing-comparision.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/cloud-pricing-comparision.png -------------------------------------------------------------------------------- /images/document-quantum-arch-characteristics.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/document-quantum-arch-characteristics.jpg -------------------------------------------------------------------------------- /images/documents-quantum-worksheet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/documents-quantum-worksheet.png -------------------------------------------------------------------------------- /images/meeting-quantum-arch-characteristics.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/meeting-quantum-arch-characteristics.jpg -------------------------------------------------------------------------------- /images/meetings-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/meetings-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/non-profit-organization-golden-path.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/non-profit-organization-golden-path.jpg -------------------------------------------------------------------------------- /images/notification-quantum-arch-characteristics.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/notification-quantum-arch-characteristics.jpg -------------------------------------------------------------------------------- /images/notification-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/notification-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/np-candidate-quantum-style-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/np-candidate-quantum-style-worksheet.jpg -------------------------------------------------------------------------------- /images/np-candidate-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/np-candidate-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/np-integration-quantum-style-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/np-integration-quantum-style-worksheet.jpg -------------------------------------------------------------------------------- /images/npo-integration-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/npo-integration-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/presentation-thumbnail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/presentation-thumbnail.png -------------------------------------------------------------------------------- /images/quanta-identification.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/quanta-identification.jpg -------------------------------------------------------------------------------- /images/recommendations-quantum-style-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/recommendations-quantum-style-worksheet.jpg -------------------------------------------------------------------------------- /images/recommendations-quantum-worksheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/recommendations-quantum-worksheet.jpg -------------------------------------------------------------------------------- /images/report-quantum-arch-characteristics.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/report-quantum-arch-characteristics.jpg -------------------------------------------------------------------------------- /images/reports-quantum-worksheet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/reports-quantum-worksheet.png -------------------------------------------------------------------------------- /images/sidecar-log-collector.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/images/sidecar-log-collector.png -------------------------------------------------------------------------------- /other-artifacts/Requirement-analysis.md: -------------------------------------------------------------------------------- 1 | ## Requirement Analysis 2 | The following analyses follow the format: 3 | *Given business requirement* 4 | * Initial Thoughts (if any) 5 | * NFRs (if any) 6 | * NFR 1 7 | * NFR 2 8 | 9 | 10 | 11 | 1. *The Platform must establish a way to incentivize engagement such as sharing of resources, collaboration, networking, facilitating introductions, and partnerships* 12 | * NPOs or candidates can be rewarded with points for various activities they perform using the system. [Google Maps Local Guides](https://maps.google.com/localguides/) is a good example which rewards with points on posting reviews, images, etc. on places. The Spotlight platform could also use something similar to award points to the NPOs and candidates on engagement. 13 | * NFRs 14 | * **Configurability** - The platform must be configurable to introduce or update new incentive rules with minimum or no engineering effort to reward engagement. 15 | 2. *The Platform must categorize/tag nonprofit support services to match candidate needs identified in the onboarding assessment to include but not limited to Resume Writing Services, Interview Prep, Free Business Attire, Apprenticeship Program Registration, Training Program Registration, College & University Registration, Free Grocery & Meal Services, Discounted Rent & Housing Services, Daycare/Child Care Services, Mentorship/Career Advocate Services* 16 | * Allow tagging categories on NPO, NPO offerings, candidate profiles, etc. 17 | * NFRs 18 | * **Performance** - The platform must be performant to provide faster matches between NPO offerings and Candidates and provide relevant recommendations. Indexing categories on various data entities may help in this regard. 19 | 3. *End-Use Ease of Use is a hard requirement* 20 | * The underrepresented demographics may not be tech-savvy. So, the Spotlight App's IA should be simple and intuitive to improve engagement. The App could have tour guides, FAQs, etc. (Long-term) The Platform could push recommendations and notifications to enable the user to take the next step on the app. 21 | * To enable NPOs to integrate with the Spotlight platform, platform needs to have well defined and documented APIs. 22 | * NFRs 23 | * **Usability** 24 | * **Interoperability** 25 | 4. *Tracking candidate progress is a hard requirement.* 26 | * For NPOs which have their own APIs for their offerings, data must be regularly pulled from their APIs to update candidate progress and particular NPO offerings in the platform. 27 | * This warrants data integrity in the system to accurately show candidate progress as and when his course progress is updated. 28 | * NFRs 29 | * **Workflow** 30 | * **Data Integrity** 31 | 5. *Tracking engagement is a hard requirement* 32 | * Any activity (attending meetings, posts, subscribing to the offerings, etc.) performed by the users must be tracked and recorded in the system. This will also help to incentivize engagement. 33 | * NFRs 34 | * **Workflow** 35 | * **Data Integrity** 36 | 6. *The Platform must provide a way to allow Non-Profits to publicize offerings to the platform that can provide some level of automatic matching for Candidate requests.* 37 | * Recommendations based on the candidate's profile, interests and activity in the system. 38 | 7. *The Platform allows offerings to contain rich text, links, and downloadable readable content such as PDFs, but no other downloads.* 39 | * Since the files/documents are essential for any NGO to operate, they must be safely stored and recoverable (in case of disaster). 40 | * NFRs 41 | * **Recoverability** 42 | * **Fault Tolerance** 43 | 8. *Each offering must support a certain list of properties (defined by the platform), such as name, organization description, website, unique identifier (assigned by the Administrators) and other identification information.* 44 | 9. *The Platform must provide both operational reports (number of candidate matches / period, number of offerings / region, and so on) and analytical reports (projections of future desirable career paths, Offering gaps in a region based on demand, and so on) for use by Administrators.* 45 | * System should reliably store user activity and data coming from various services in the platform. 46 | * Big data store to train models for better predictions on recommendations. Consider **OLAP solution** to perform analytics. 47 | 10. *Reminder to think critically about the nonprofit and candidate experience, anticipate these users needs while developing the use case and user stories. Consider what can offer these users maximum value to fulfill the intent of logging on the app.* 48 | * Apply design thinking to understand user personas and their needs. Come up with a user experience that is intuitive. 49 | * NFRs 50 | * **Usability** 51 | 52 | ### Further Analysis and Considerations 53 | #### Green Field Project 54 | There are several factors which affect the success of any new business or green field project. 55 | * Viability of the idea - Are there any buyers for the idea? 56 | * Funding - Are there any investors for the idea? 57 | * Time - An idea may be viable but it may take a long time to develop / implement. And, an idea which is viable today may not be that attractive after certain period of time. 58 | 59 | Also, _Why should the business invest to build a fortress when it is not sure if anyone would be staying in it or using it._ 60 | 61 | So, shorter iterations to get market feedback and pivoting, if necessary, becomes important. So, the architecture should be in such a way that it can evolve with the growth in the business. 62 | 63 | **NFRs** 64 | * **Feasibility** 65 | * **Evolutionary** 66 | 67 | #### Funding 68 | The team researched about NPOs, 501 (c3), underrepresented demographics in the US. It was found that fund raising has been difficult during COVID times ([Source](https://morweb.org/post/5-challenges-facing-nonprofits-in-2021)) due to limited in-person fundraising opportunities. 69 | 70 | However, there were also articles stating that the Americans are generous ([Source](https://independentsector.org/about/the-charitable-sector/)) and donate to non-profits if their vision is sticky. Also, there are several grants for NPOs to raise money without taking out traditional business loans ([Source](https://www.fundera.com/blog/grants-for-nonprofits)). So, funding should not be a problem, at least in the long-term, if the Spotlight platform markets their idea and MVP faster. 71 | 72 | **NFRs** 73 | * **Feasibility** 74 | 75 | #### Security 76 | Sensitive data stored in the system includes - Candidate career profiles, PII (email, phone, address, ethnicity, SSN, etc.). 77 | As per [GDPR](https://gdpr.eu/right-to-be-forgotten/), the system should be able to erase the PII if a user wants to be forgotten. 78 | 79 | Consider not leaving footprints of PII in various services and logs of the system, which might make it harder to erase the PII. Use just one place where the data can be stored and retrieved. Ephemeral storages, such as cache, may be used for faster retrievals but should have mechanisms in place to erase the data 80 | 81 | 82 | #### No. of users and Regions 83 | 84 | Going by the no. of NPOs ([Source](https://independentsector.org/about/the-charitable-sector/)) and under-represented demographics ([Source](https://www.governing.com/archive/gov-nonprofits.html)), there is a tremendous potential for active collaboration and could be an exponential growth in no. of users and NPOs in the platform once the idea is viral. Even if we assume 0.5% adoption in 1~2 years, there could be 100k NPOs and 500k under-represented folks using the platform, which is huge. 85 | 86 | To support NPOs and demographics in multiple regions, consider multi-zone deployment in the longer-term, for improving availability. 87 | 88 | **NFRs** 89 | * Scalability 90 | * Availability 91 | 92 | 93 | ### Summary 94 | 95 | 96 | | Given Requirements | Analysis | NFRs | 97 | | ------------------ | -------- | ---- | 98 | | The Platform must establish a way to incentivize engagement such as sharing of resources, collaboration, networking, facilitating introductions, and partnerships| To introduce or update new incentive rules and rewards systems with no or minimal engineering effort | Configurability | 99 | | The Platform must categorize/tag nonprofit support services to match candidate needs identified in the onboarding assessment | The platform must be performant to provide faster matches between NPO offerings and Candidates and provide relevant recommendations. Indexing categories on various data entities may help in this regard.| Performance | 100 | | End-Use Ease of Use is a hard requirement | The underrepresented demographics may not be tech-savvy. So, the Spotlight App's Information Architecture should be simple and intuitive to improve engagement. To enable NPOs to integrate with the Spotlight platform, platform needs to have well defined and documented APIs. | Usability, Interoperability | 101 | | Tracking candidate progress is a hard requirement. | For NPOs which have their own APIs for their offerings, data must be regularly pulled from their APIs to update candidate progress and particular NPO offerings in the platform. This warrants data integrity in the system to accurately show candidate progress as and when his course progress is updated.| Workflow, Data Integrity | 102 | | Tracking engagement is a hard requirement | Any activity (attending meetings, posts, subscribing to the offerings, etc.) performed by the users must be tracked and recorded in the system. This will also help to incentivize engagement. | Workflow, Data Integrity | 103 | | The Platform must provide a way to allow Non-Profits to publicize offerings to the platform that can provide some level of automatic matching for Candidate requests. | Recommendations based on the candidate's profile, interests and activity in the system. | NA | 104 | | The Platform allows offerings to contain rich text, links, and downloadable readable content such as PDFs, but no other downloads. | Since the files/documents are essential for any NGO to operate, they must be safely stored and recoverable (in case of disaster). | Recoverability, Fault tolerance | 105 | | The Platform must provide both operational reports (number of candidate matches / period, number of offerings / region, and so on) and analytical reports (projections of future desirable career paths, Offering gaps in a region based on demand, and so on) for use by Administrators. | System should reliably store user activity and data coming from various services in the platform. Big data store to train models for better predictions on recommendations. Consider **OLAP solution** to perform analytics.| NA | 106 | | Green Field Project | Green field projects need shorter iterations to get market feedback and pivoting, if necessary, becomes important. So, the architecture should allow faster introduction or removal of features. Also, raising funds has been difficult during COVID times.| Evolutionary, Feasibility | 107 | | [Derived] Sensitive data stored in the system includes - Candidate career profiles, PII (email, phone, address, ethnicity, SSN, etc.). | As per [GDPR](https://gdpr.eu/right-to-be-forgotten/), the system should be able to erase the PII if a user wants to be forgotten. | Security | 108 | | [Derived] No. of users | To support exponential growth in no. of users and NPOs in the platform once the idea is viral. Even if we assume 0.5% adoption (Reference - [NPOs](https://independentsector.org/about/the-charitable-sector/) and [Under-represented demographics](https://www.governing.com/archive/gov-nonprofits.html)) in 1-2 years, there could be 100k NPOs and 500k under-represented folks using the platform, which is huge.| Scalability | 109 | | [Derived] Multi region support | To support NPOs and demographics in multiple regions, consider multi-zone deployment in the longer-term| Availability | 110 | -------------------------------------------------------------------------------- /other-artifacts/spotlight-platform-team-pegasuz.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/z-katas/arch-katas-dcc/1ef2c660866a82d9a9d693d7dc1f52dcce42a190/other-artifacts/spotlight-platform-team-pegasuz.mp4 -------------------------------------------------------------------------------- /other-services/auth-service.md: -------------------------------------------------------------------------------- 1 | ## Auth Service 2 | 3 | This service handles identity management and access control in the platform. 4 | 5 | ### Responsibilities 6 | 7 | 1. Listen to NPO and Candidate registrations in the system to create corresponding users. 8 | 2. User management APIs - CRUD 9 | 1. Create/update a user profile, metadata (npo_id, etc) 10 | 2. Update user avatars 11 | 3. Reset & Change Password 12 | 3. Roles and permissions management - CRUD 13 | 4. Send emails (publish to an email topic) to users on successful creation, reset/change password, etc. 14 | 15 | ### Driving Architectural Characteristics 16 | 17 | #### Top 3 18 | 19 | - **Feasibility (time)** - Since all the users and many services in the platform rely on auth service for identity and access control, this service becomes core to the platform and should be developed the earliest. 20 | - **Availability** - Since this is a core service, this service needs to be highly available to support features in the platform. 21 | - **Data integrity** - Auth service will not be able to function as expected if the data it stores is not accurate or consistent. So, data integrity is important for this service. 22 | 23 | #### Other Driving Characteristics 24 | 25 | - **Scalability** - Since this is a core service, traffic on this service is expected to quickly rise with the increase in no. of NPOs and candidates. 26 | - **Responsiveness** - Since this service is a core service, it should be able to respond to requests as quickly as possible. 27 | - **Security** - Since this service stores PII, data life cycle needs to follow certain compliance rules. 28 | 29 | ### Architectural Style Preferred 30 | 31 | Hybrid - Microservices and Event-Driven 32 | 33 | ### Relevant ADRs 34 | 35 | - [External IdP](../ADRs/009.adr-idp.md) -------------------------------------------------------------------------------- /other-services/bff-service.md: -------------------------------------------------------------------------------- 1 | ## Backend for frontend Service 2 | 3 | This service handles all the frontend client interactions with the platform services. 4 | 5 | ### Responsibilities 6 | 1. Expose a graphql interface for all the frontend use cases. 7 | 2. Route Graphql API requests from UI to respective microservices translating them into rest requests. 8 | 3. Address the functional requirements which are different for UI from the backend. 9 | 4. Stream system events to UI incase of failures in async commits. 10 | 5. Merge data from different microservices and supply to the UI. (this helps when there is low network bandwidth) 11 | 6. Completely stateless 12 | 13 | ### Driving Architectural Characteristics 14 | 15 | #### Top 3 16 | * **Fault tolerance (time)** - This will be a single point of failure for the UI apps. Hence this has to be highly fault-tolerant. 17 | * **Scalability** - Service should be highly scalable as 18 | * **Testability** - This needs to be testable and maintainable as we introduce more APIs to this service as the platform keeps growing, thus adding more room for bugs. 19 | 20 | 21 | ### Relevant ADRs 22 | * [API standard](../ADRs/002.adr-api-standard.md) 23 | * [Backend for frontend](../ADRs/004.adr-bff.md) 24 | 25 | -------------------------------------------------------------------------------- /other-services/infrastructure-services.md: -------------------------------------------------------------------------------- 1 | ## Infrastructure Components 2 | 3 | Components falling under this group are 4 | * API Gateway 5 | * Sidecar logger 6 | * Metrics collector 7 | * Monitoring and Alerting systems 8 | 9 | These services handle the infrastructure and operational needs of the platform. 10 | 11 | 12 | 13 | ### API Gateway Responsibilities 14 | 1. Route the traffics to respective microservices in the systems. 15 | 2. Authenticate the incoming requests. 16 | 3. Add a request-id header to incoming requests for traceability. 17 | 18 | ### Sidecar Logger Responsibilities 19 | 1. Collects logs from the pods and push them to a central log repository 20 | 21 | ### Metrics collector Responsibilities 22 | 1. Collect metrics for the entire platform, these metrics include 23 | * No. of API requests 24 | * Success and failure rates 25 | * Response times 26 | 2. Store the metrics in a time-series database 27 | 28 | ### Monitoring and Alerting systems 29 | 30 | 1. Dashboards for monitoring the services. 31 | 2. Alerting mechanisms in case of service downtime/failures 32 | 33 | 34 | ### References 35 | https://medium.datadriveninvestor.com/simple-way-to-stream-kubernetes-logs-with-sidecar-pattern-5240ee88e8f0 36 | 37 | -------------------------------------------------------------------------------- /other-services/support-service.md: -------------------------------------------------------------------------------- 1 | ## Support Service 2 | 3 | This service enables users to raise tickets (technical / general) and track the status. It provides support team access to all tickets and current progress. 4 | 5 | ### Responsibilities 6 | 1. Let users / NPOs raise support tickets. 7 | 2. View raised tickets and status. 8 | 3. Let support team view all tickets and status. 9 | 4. Notify users / support team any update on the ticket. 10 | 5. Multichannel support (Optional) - Chat, Voice, Email. 11 | 12 | ### Driving Architectural Characteristics 13 | 14 | #### Top 3 15 | * **Availability** - Support service must be highly available (24x7) to raise any issues. 16 | * **Workflow** - The tickets follow progress across statuses in a defined workflow. 17 | * **Data integrity** - Data captured in tickets is of most importance - overall accuracy, completeness, of data is necessary. 18 | 19 | ### Architectural Style Preferred 20 | Microservices 21 | 22 | ### Relevant ADRs 23 | NA 24 | 25 | -------------------------------------------------------------------------------- /quanta/chat.md: -------------------------------------------------------------------------------- 1 | ## Chat Quantum 2 | 3 | ![Image](../diagrams/quanta/chat-quanta.jpg) 4 | 5 | ### Context 6 | 7 | Having in-app chat support on the platform is a great way to engage with the community. Also, a candidate doesn't have to leave the app to engage with the mentors (say, over email). 8 | 9 | However, Building a reliable chat service within the platform would take time and is not the core value proposition. (See [ADR](../ADRs/003.adr-build-vs-buy.md)). Recommended to use a 3rd party calendar service. 10 | 11 | ### Responsibilities 12 | 13 | 1. Initiate a new conversation with users. 14 | 2. View and reply to messages. 15 | 3. Support read receipts and last active status. 16 | 4. Integrate with 3rd party chat service for the functionalities. 17 | 5. [Long Term] Remind users through an email if a chat remains unread. 18 | 19 | ### Driving Architectural Characteristics 20 | 21 | ![Image](../images/chat-quantum-worksheet.jpg) 22 | 23 | #### Top 3 24 | 25 | ##### Driving Architectural Characteristics 26 | 27 | * **Extensibility** - Should support addition of platform specific features not provided by the 3rd party meetings provider. 28 | * **Abstraction** - The chat service acts like a facade to the 3rd party chat provider. 29 | * **Testability** 30 | 31 | ##### Characteristics that we do not need as we offloaded to 3rd party vendors 32 | * **Security** - Security for messages during storage and transit. 33 | * **Scalability** - Service highly depends on third party service for scale 34 | * **Availability** - Service highly depends on third party service for uptime 35 | * **Responsiveness** - The messages have to be sent and received with minimal latency 36 | 37 | ### Architectural Style Preferred 38 | 39 | ![Image](../images/chat-quantum-arch-characteristics.jpg) 40 | Microservices 41 | 42 | ### Relevant ADRs 43 | 44 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) -------------------------------------------------------------------------------- /quanta/document.md: -------------------------------------------------------------------------------- 1 | ## Document 2 | 3 | ![Image](../diagrams/quanta/documents-quanta.jpg) 4 | This service stores and retrieves files. The files could be images (avatars, NPO favicons, etc.) or documents (candidate resumes, NPO offering details, reports - PDF / Spreadsheet, etc.), which are not expected to be too large. NP admin wants to engage with his/her community with the help of a post involving an image. 5 | 6 | ### Responsibilities 7 | 8 | 1. Document management - create, read, delete 9 | 10 | ### Driving Architectural Characteristics 11 | 12 | ![Image](../images/documents-quantum-worksheet.png) 13 | 14 | #### Top 3 15 | 16 | - **Recoverability** - In case of disasters, the service is expected to lose storage. Regular backups need to be taken. 17 | - **Responsiveness** - Respond to document requests as quickly as possible. Probably, cache the frequently used files using CDNs. 18 | - **Scalability** - Since this is integral to the business to store resumes, NPO offerings, etc., traffic on the service is expected to quickly rise with the increase in no. of NPOs and candidates. 19 | 20 | #### Other Driving Characteristics 21 | 22 | - **Availability** - Since documents, such as resumes and NPO offerings, form the core of the business, it is expected that documents service The service should be highly available. 23 | 24 | ### Architectural Style Preferred 25 | 26 | Microservices 27 | 28 | ![Image](../images/document-quantum-arch-characteristics.jpg) 29 | 30 | ### Relevant ADRs 31 | 32 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) 33 | - [Data route to storage](../ADRs/008.adr-upload-directly-to-document-storage.md) -------------------------------------------------------------------------------- /quanta/meetings.md: -------------------------------------------------------------------------------- 1 | ## Meetings 2 | 3 | ![Image](../diagrams/quanta/meeting-quanta.jpg) 4 | 5 | Engagements are a key part of the Spotlight platform and meetings are a way to engage with the community. However, Building a reliable calendar service within the platform would take time and is not the core value proposition. (See [ADR](../ADRs/003.adr-build-vs-buy.md)). Recommended to use a 3rd party calendar service. 6 | 7 | ### Responsibilities 8 | 9 | 1. Let NPOs create & manage events. 10 | 2. Maintain RSVP - invited, accepted, rejected, Maybe. 11 | 3. Send notifications to respective participants regarding any update to event. 12 | 4. Collect feedback, post an event. 13 | 5. Archive event post completion and disable further updates. 14 | 15 | ### Driving Architectural Characteristics 16 | 17 | ![Image](../images/meetings-quantum-worksheet.jpg) 18 | 19 | #### Top 3 20 | 21 | * **Abstraction** - The meetings service acts like a facade to the 3rd party calendar service. 22 | 23 | - **Scalability**: Should support the increase in no. of NPOs and Candidates adopting to the platform. 24 | 25 | * **Extensibility** - Should support addition of platform specific features not provided by the 3rd party meetings provider. 26 | 27 | 28 | ##### Characteristics that we do not need as we offloaded to 3rd party vendors 29 | - **Feasibility** - Meetings is the core value proposition of the platform to enable better engagement between the user, especially Candidates and Mentors. So, building it as part of the MVP is inevitable and feasibility becomes important. One can consider going for a 3rd party chat provider, rather than building in-house. 30 | - **Recoverability** - Service must recover from any failures fast and no loss of data. 31 | - **Availability** - The service must be available at all times since other services are dependent on it. 32 | 33 | 34 | ### Architectural Style Preferred 35 | 36 | Microservices 37 | 38 | ![Image](../images/meeting-quantum-arch-characteristics.jpg) 39 | 40 | ### Relevant ADRs 41 | 42 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) -------------------------------------------------------------------------------- /quanta/notification.md: -------------------------------------------------------------------------------- 1 | ## Notification Quantum 2 | ![Image](../diagrams/quanta/notifications-quanta.jpg) 3 | 4 | ### Responsibilities 5 | 1. Send in-app and push notifications to users, this will increase the usability of the app. 6 | 2. Saves users' preferences about the type and frequency of notifications they would like to be subscribed to. 7 | 3. Notify users via emails. 8 | 4. Integrate with 3rd party notification and email providers. 9 | 4. Enable users to send and notify user(s) through emails and notifications 10 | 11 | 12 | ### Driving Architectural Characteristics 13 | 14 | ![Image](../images/notification-quantum-worksheet.jpg) 15 | #### Top 3 16 | ##### Driving Architectural Characteristics 17 | 1. **Scalability** - The quantum is highly scalable and can be scaled to the growing NPO + Candidate community adopting to the platform. 18 | 2. **Extensibility** - Should be able to add new notification channel easily. 19 | 3. **Availability** - Notifications for all the events in the system depend on this service. So this service needs to be highly available 20 | 21 | ##### Characteristics that we do not need as we offloaded to 3rd party vendors 22 | * **Security** - Security around emailing service will be the responsibility of the emailing vendor. 23 | 24 | 25 | ### Architectural Style Preferred 26 | Hybrid - Microservices + Event Driven 27 | 28 | ![Image](../images/notification-quantum-arch-characteristics.jpg) 29 | 30 | ### Relevant ADRs 31 | 32 | -------------------------------------------------------------------------------- /quanta/npo-candidate.md: -------------------------------------------------------------------------------- 1 | ## NPO-Candidate Quantum 2 | ![Image](../diagrams/quanta/np-candidate-quanta.jpg) 3 | 4 | ### Components and Responsibilities 5 | * **NP Service** 6 | * Manages NPO registration, profile, capability assessment, posts, comments, offerings, NP User / Mentor profiles, npo partnerships, offering feedback. 7 | * Publishes events when posts are created, comments are added, there is a need to notify users, NPO is assessed, feedback is provided on offering, NPO Admin / User is registered. 8 | * **Candidate Service** 9 | * Manages candidate registration, career assessment, profile, testimonials, road map, incentives 10 | * Publishes events when candidate is assessed, testimonials are added, candidate registration is made, there is a need to notify users. 11 | * Listens to assignment updates to incentivize candidate. 12 | * Acts as an interface to **LinkedIn** to fetch candidate career profile. 13 | * **Assignment Service** 14 | * Manages candidate and NPO assignments, feedback on assignments. 15 | * Listens to NPO assessed and Candidate assessed events to create corresponding assignments. 16 | * Listens to assignment created event to search for mentors to be tagged to the assignment. 17 | * Publishes events when assignments are updated and there is a need to notify users. 18 | * **Auth Service** 19 | * Enables user management, access control. More details can be found in [Auth Service](../other-services/auth-service.md) 20 | 21 | 22 | 23 | ### Driving Architectural Characteristics 24 | ![Image](../images/np-candidate-quantum-worksheet.jpg) 25 | 26 | #### Top 3 27 | * **Feasibility (cost & time, in MVP)** - Since it is a green field project and quick market validation of the idea is paramount. Also, initial funding may be hard to come by. 28 | 29 | * **Extensibility (Evolvability)** - Since it is a green field project and there is ambiguity at the beginning, the architecture should be flexible to introduce new features or change existing ones. 30 | 31 | * **Scalability** - The components in this quantum form the core operational aspect of the platform and must be scalable to accommodate the capacity planned for the system. 32 | 33 | * **Availability (Post MVP)** - The components in this quantum form the core operational aspect of the platform and must be available in multiple zones across USA. 34 | 35 | #### Other Driving Characteristics 36 | 37 | * **Testability** - The components in this quantum hold policies and rules (such as incentive programs, registrations, mentor assignment rules, etc.) and must be testable easily. 38 | 39 | * **Interoperability** - The service APIs in this quantum would be used by external NPOs to integrate with the platform. So the API documentation becomes important. 40 | 41 | * **Recoverability** - The operational data captured by the components in this quantum used by the system for analytics and reports. So, it must be recoverable in case of disasters. 42 | 43 | * **Data integrity** - Tracking candidate progress is a hard requirement and the operational data captured by the components in this quantum used by the system for analytics and reports. So, accuracy becomes important. 44 | 45 | ### Architectural Style 46 | ![Image](../images/np-candidate-quantum-style-worksheet.jpg) 47 | * MVP - Modular monolith (see [ADR - NP Candidate Arch style MVP](../ADRs/007.adr-arch-style-np-candidate-mvp.md)) 48 | * Long-Term - Hybrid - Microservices and Event-Driven 49 | 50 | ### Trade-offs - Mitigation strategies 51 | * Event driven + microservices is a costly architecture. Setting up the infrastructure (CI/CD pipelines, continuous delivery, aggregated logging and monitoring, etc.) becomes really important. 52 | * Mitigation - Given the advancements in the devops world and so much guidance out on the internet, this should not be a concern. Also, setting up the infrastructure at the beginning of the project might reap benefits later on. 53 | * Data is not consistent in an event driven system. It would be eventually consistent. 54 | * Mitigation - This trade-off may be acceptable as the system is not dealing with mission critical information (such as affecting human life / money). 55 | * Having a microservices architecture may reduce the performance and reliability of the services due to network hops. 56 | * Mitigation - Having an event-driven system internally with a reliable message queue would increase the reliability and performance of the system. 57 | 58 | 59 | ### Call Flow diagrams 60 | 61 | #### Course enrollment for a course call flow 62 | ![Image](../diagrams/call-flows/call-flow-course-enrollment.jpg) 63 | 64 | ### Post notification call flow 65 | ![Image](../diagrams/call-flows/call-flow-post-notification.jpg) 66 | 67 | ### NPO user updating an excel sheet to update candidate assignment progress call flow 68 | ![Image](../diagrams/call-flows/call-flow-candidate-assignment-progress.jpg) 69 | 70 | ### Relevant ADRs 71 | 72 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) 73 | - [Identity Provider](../ADRs/009.adr-idp.md) -------------------------------------------------------------------------------- /quanta/npo-integration.md: -------------------------------------------------------------------------------- 1 | ## NPO Integration [Long Term] Quantum 2 | 3 | ![Image](../diagrams/quanta/np-integration-quanta.jpg) 4 | 5 | The spotlight platform's requirements state that _Tracking candidate progress_ and _End-Use Ease of Use_ are hard requirements. Without an automated system in place, NP Users (admins, mentors, tutors, etc.) would have to manually update candidates' progress on assignments by logging into the spotlight app, which may leave room for error or a gap in updating candidates' progress. Similarly, some offerings may need manual interventions before accepting candidate enrollments. This is unavoidable if the NPOs do not have a system or an API (outside of the Spotlight platform) of their own to manage their offerings and candidates. 6 | 7 | What if they do have such a developed system? There are two ways to integrate such systems: 8 | 9 | 1. The NPO builds a service to **push** data to a reliable stream provided by the Spotlight platform. 10 | - This is straightforward and can be solved with services and streams mentioned in the [NPO-Candidate quantum](npo-candidate.md) 11 | 2. Or Spotlight platform pulls the information from the NPO's API 12 | - This requires a sophisticated system that can be built in the longer term. 13 | 14 | This document talks about the second use case. 15 | 16 | ### Requirements 17 | 18 | Some of the requirements for the NPO integration: 19 | 20 | - NPOs should be able to provide their API details (endpoints, credentials, etc.) to the Spotlight platform at the time of NPO assessment. 21 | - Have independent functions to correctly transform the data to/from the NPO's API and the Spotlight platform. 22 | - Should be able to introduce new NPO integrations without affecting the existing ones. 23 | - The platform should be able to reliably transfer the data from the NPO's API to the Spotlight platform. 24 | 25 | ### Components and Responsibilities 26 | 27 | - NPO Integration Service: 28 | - Manages serverless function endpoint for each NPO offering and NPO 29 | - Manages API credentials of the NPO 30 | - Trigger Enrollment (Job): 31 | - Listens to candidate assignment event and triggers NPO's corresponding enrolment function to transform and send data to the NPO service. 32 | - Enrolment Function (Job): 33 | - Transforms Spotlight platform's assignment data to the format accepted by the respective NPO's enrolment offering API 34 | - Posts the above-transformed data to that service 35 | - NPO Offering / Candidate Assignment Extraction Function (Job): 36 | - Fetches data from the corresponding NPO's API and stores it in the NPO integration store as is (maybe with an additional field to check processed or not) 37 | - NPO Extraction Scheduler) 38 | - Periodically queries all the active NPO offerings present on the NPO Integration service and triggers corresponding functions to fetch updates from the corresponding NPO's API. 39 | - Data Ingestion Scheduler 40 | - Periodically queries the integration store and triggers corresponding functions to transform offerings and candidate assignments. 41 | - Assignment Ingestion Function (Job) 42 | - Transforms candidate assignment data to the format recognized by the spotlight platform and publishes an event to assignment update topic on the event stream. 43 | - Offering Ingestion Function (Job) 44 | - Transforms NPO offering data to the format recognized by the spotlight platform and publishes an event to offering create / update topic on the event stream. 45 | 46 | ### Driving Architectural Characteristics 47 | 48 | ![Image](../images/npo-integration-quantum-worksheet.jpg) 49 | 50 | #### Top 3 51 | 52 | - **Workflow**: Workflow is core to the NPO integration quantum to enable reliable ingestion into spotlight platform and post data to the NPO's API. 53 | - **Configurability**: Integration of the platform with a new NPO should not warrant a redeployment of the entire quantum. 54 | - **Feasibility**: The architecture should support quick integration with a new NPO 55 | 56 | #### Other Driving Characteristics 57 | 58 | - **Scalability**: Should support the increase in no. of NPOs wanting to integrate with the platform. 59 | - **Testability**: Since this system needs to be highly configurable, a supporting framework should be bug-free. 60 | 61 | ### Architectural Style Preferred 62 | ![Image](../images/np-integration-quantum-style-worksheet.jpg) 63 | - _Event-driven_ for enabling workflow in data ingestion into the spotlight platform and pushing data to NPO APIs 64 | - _Microkernal_ for introducing new NPO integrations without affecting the existing ones. 65 | 66 | ### Tradeoffs - Mitigation Strategies 67 | 68 | - Data consistency and integrity - Since the design is pulling data from NPO's API periodically (asynchronously), the data in both systems is not guaranteed to be consistent. 69 | - Mitigation - Data can be made eventually consistent, based on the frequency of the scheduler, by using a reliable streaming framework. 70 | - Testing an event-driven system is tough. 71 | - Mitigation - The good news is that the challenges are well-known. Aggregated logging and distributed tracing across the system can be used to make it easier for the testers. 72 | -------------------------------------------------------------------------------- /quanta/quanta-identification.md: -------------------------------------------------------------------------------- 1 | ## Quanta Identification 2 | 3 | The team collaboratively performed an event storming + actor action exercise to identify the aggregates. 4 | 5 | ![Image](../images/aggregate-identification.jpg) 6 | 7 | [For better navigation on the content, use this miro frame](https://miro.com/app/board/uXjVOv-nlBo=/?moveToWidget=3458764528112831067&cot=14) 8 | 9 | The aggregates were then grouped to identify services. Services with synchronous dependencies were identified to identify quanta. 10 | 11 | ![Image](../images/quanta-identification.jpg) 12 | 13 | 14 | ### Quanta - Services 15 | 16 | * NPO-Candidate 17 | * NPO Service 18 | * Candidate Service 19 | * Assignment Service 20 | * Assignment Scheduler 21 | * Auth Service 22 | * Chat 23 | * Chat Service 24 | * Notification 25 | * Notification service 26 | * Email Service 27 | * 3rd Party Notification systems (for push and in-app notifications) 28 | * SMS 29 | * Meeting 30 | * NPO-Integration 31 | * Recommendation 32 | * Documents 33 | * Chat -------------------------------------------------------------------------------- /quanta/recommendation.md: -------------------------------------------------------------------------------- 1 | ## Recommendations Quantum 2 | Intelligently predicting and providing relevant recommendations regarding NPO offerings to the candidate adds value and fulfills the intent of the candidate of logging on the app. And, the matching can only become better over time with model training with more data captured by the platform. 3 | ![Image](../diagrams/quanta/recommendations-quanta.jpg) 4 | 5 | ### Responsibilities 6 | 1. Recommends communities and NPO's to the candidates based on offerings and testimonials. 7 | 2. Recommends programs that candidates can enroll into 8 | 3. The service learns based on the feedback of previous recommendations and continues to improve as the system grows. 9 | 10 | ### Services 11 | * Recommendations Service 12 | * Model Training 13 | * Training Scheduler 14 | * Matching Trigger Service 15 | * Matching Engine 16 | 17 | ### Driving Architectural Characteristics 18 | ![Image](../images/recommendations-quantum-worksheet.jpg) 19 | #### Top 3 20 | 1. Adaptability - The system continues to adapt to growing requirements and varieties of users/communities it onboard. 21 | 2. Data integrity - The accuracy of matches is really crucial in increasing the user engagement and value proposition of the platform 22 | 3. Scalability - Scale the matching algorithm and workflow to ever-growing platform users. 23 | 24 | #### Other Driving Characteristics 25 | * Workflow 26 | * Testability 27 | 28 | ### Architectural Style Preferred 29 | ![Image](../images/recommendations-quantum-style-worksheet.jpg) 30 | Microservices 31 | 32 | ### Relevant ADRs 33 | 34 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) -------------------------------------------------------------------------------- /quanta/reports.md: -------------------------------------------------------------------------------- 1 | ## Reports and Analytics Quantum 2 | 3 | ![Image](../diagrams/quanta/Reports-quanta.jpg) 4 | 5 | This service enables users to generate various reports from analytics readily available in data warehouse 6 | 7 | ### Responsibilities 8 | 9 | 1. Let users generate and download reports on demand in requisite formats - PDF, CSV etc. 10 | 2. Scheduler to send reports - EOD, Weekly, Monthly etc. 11 | 3. Provide most commonly used reports by default 12 | 4. Enable the creation of custom reports (using custom query language) 13 | 14 | ### Driving Architectural Characteristics 15 | 16 | ![Image](../images/reports-quantum-worksheet.png) 17 | 18 | #### Top 3 19 | 20 | - **Performance** - Since reports can be fetched for varying time periods, the amount of data to be processed might be high - necessary optimizations must be done to ensure decent performance. 21 | - **Recoverability** - Service must recover from any failures fast and with minimal loss of data. 22 | - **Configurability** - Users can create and configure the way reports need to be generated. 23 | 24 | ### Architectural Style Preferred 25 | 26 | Hybrid - Event-driven (for processing periodic reports) + Microservices 27 | 28 | ![Image](../images/report-quantum-arch-characteristics.jpg) 29 | 30 | ### Relevant ADRs 31 | - [Build vs buy](../ADRs/003.adr-build-vs-buy.md) 32 | --------------------------------------------------------------------------------