5 |
6 | Comment [DRB]:
7 | This should be an introduction only, with plenty of references to
8 | external resources.
9 |
10 |
11 |
12 | This chapter is intended to provide some context for the main subject of
13 | this guide: a discussion of the Command Query Responsibility Segregation
14 | (CQRS) pattern. It is useful to understand some of the origins of the
15 | CQRS pattern and some of the terminology you will encounter both in this
16 | guide and in other material that discusses the CQRS pattern. It is
17 | particularly important to understand that the CQRS pattern is not
18 | intended for use as the top-level architecture of your system; rather it
19 | should be applied to those sub-systems that will gain specific benefits
20 | from the application of the pattern.
21 |
22 | Before we look at the issues surrounding the use of different
23 | architectures within a complex application, we need to introduce some of
24 | the terminology that we will use in this chapter, and in subsequent
25 | chapters of this reference guide. Much of this terminology comes from an
26 | approach known as Domain-driven Design (DDD). There are a few important
27 | points to note about our use of this terminology:
28 |
29 | - There are other approaches that tackle the same problems that DDD
30 | tackles, with similar concepts, but with their own specific
31 | terminologies.
32 |
33 | - We are using the DDD terminology because many CQRS practitioners also
34 | use this terminology, and it is used in much of the existing CQRS
35 | literature.
36 |
37 | - While it is clear that using a DDD approach sometimes leads naturally
38 | to an adoption of the CQRS pattern, we are not saying that the DDD
39 | approach always leads to the use of the CQRS pattern, or that the DDD
40 | approach is a prerequisite for using the CQRS pattern.
41 |
42 | - You may question our interpretation of some of the concepts of DDD.
43 | The intention of this guide is to take what is useful from DDD to help
44 | us explain the CQRS pattern and related concepts, not to provide
45 | guidance on how to use the DDD approach.
46 |
47 | To learn more about DDD, you should read the book "Domain-Driven Design:
48 | Tackling Complexity in the Heart of Software" by Eric Evans.
49 |
50 | For a summary of the key points in Eric Evans' book, you should read the
51 | free book, "[Domain-Driven Design Quickly][dddquickly]" by Abel Avram
52 | and Floyd Marinescu.
53 |
54 | # What is Domain-Driven Design?
55 | As previously stated, DDD is an approach to developing software systems,
56 | and in particular systems that are complex, that have ever-changing
57 | business rules, and that you expect to last for the long-term within the
58 | enterprise.
59 |
60 | The core of the DDD approach uses a set of techniques to analyze your
61 | domain and to construct a conceptual model that captures the results of
62 | that analysis: you can then use that model as the basis of your
63 | solution. The analysis and model in the DDD approach are especially well
64 | suited to designing solutions for large and complex domains. DDD also
65 | extends its influence to other aspects of the software development
66 | process as a part of the attempt to help you manage complexity:
67 |
68 | - In focusing on the domain, DDD focuses on the area where the business
69 | and the development team must be able to communicate with each other
70 | clearly, but where in practice they often misunderstand each other.
71 | The domain models that DDD uses should capture detailed, rich business
72 | knowledge, but should also be very close to the code that is actually
73 | written.
74 | - Domain models are also useful in the longer term if they are
75 | maintained. By capturing valuable domain knowledge, they facilitate
76 | maintaining and enhancing the system in the future.
77 | - DDD offers guidance on how large problem domains can be effectively
78 | divided up, enabling multiple teams to work in parallel, and enabling
79 | you to target appropriate resources to those parts of the system with
80 | the greatest business value.
81 |
82 | The DDD approach is appropriate for large, complex systems that are
83 | expected to have a long life span. You are unlikely to see a return on
84 | your investment in DDD if you use it on small, simple, or short-term
85 | projects.
86 |
87 |
88 |
89 | Comment [DRB]:
90 | Look for a suitable quote to use for the previous paragraph.
91 |
92 |
93 |
94 | # Domain-Driven Design: Concepts and Terminology
95 | As previously stated, this guide is not intended to provide guidance on
96 | using the DDD methodology. However, it is useful to understand some of
97 | the concepts and terminology from DDD because they are useful when we
98 | describe some aspects of CQRS pattern implementations. These are not
99 | official or rigorous definitions; they are intended to be useful,
100 | working definitions for the purposes of this guide.
101 |
102 | ## Domain Model
103 | At the heart of DDD lies the concept of the domain model. This model is
104 | built by the team responsible for developing the system in question, and
105 | that team consists of both domain experts from the business and software
106 | developers. The domain model serves several functions:
107 |
108 | - It captures all of the relevant domain knowledge from the domain
109 | experts.
110 | - It enables the team to determine the scope and verify the consistency
111 | of that knowledge.
112 | - The model is expressed in code by the developers.
113 | - It is constantly maintained to reflect evolutionary changes in the
114 | domain.
115 |
116 |
117 | DDD focuses on the domain because it contains business value. An
118 | enterprise derives its competitive advantage and generates business
119 | value from its core domains. The role of the domain model is to capture
120 | what is valuable or unique to the business.
121 |
122 | Much of the DDD approach focuses on how to create, maintain, and use
123 | these domain models. Domain models are typically composed of elements
124 | such as Entities, Value Objects, Aggregates, and described using terms
125 | from a Ubiquitous Language.
126 |
127 | ## Ubiquitous Language
128 | The concept of a ubiquitous language is very closely related to that of
129 | the domain model. One of the functions of the domain model is to foster
130 | a common understanding of the domain between the domain experts and the
131 | developers. If both the domain experts and the developers use the same
132 | terms for things and actions within the domain (for example, conference,
133 | chair, attendee, reserve, waitlist), there is less possibility for
134 | confusion or misunderstanding. More specifically, if everyone uses the
135 | same language there are less likely to be misunderstandings that result
136 | from translations between languages. For example, if a developer has to
137 | think, "if the domain expert talks about a delegate, he is really
138 | talking about an attendee in the software," then eventually something
139 | will go wrong as a result of a misunderstanding or mistranslation.
140 |
141 | ## Entities, Value Objects, and Services
142 | DDD uses these terms to identify some of the internal artifacts that
143 | will make up the domain model.
144 |
145 | ### Entities
146 | Entities are objects that are defined by their identity, and that
147 | identity continues through time. For example, a conference in a
148 | conference management system will be an entity; many of its attributes
149 | could change over time (such as its status, size, and even name), but
150 | within the system each particular conference will have its own unique
151 | identity. The object that represents the conference may not always exist
152 | within the system's memory; at times it may be persisted to a database,
153 | only to be re-instantiated at some point in the future.
154 |
155 | ### Value Objects
156 | Not all objects are defined by their identity: for some objects what is
157 | important are the values of their attributes. For example, within our
158 | conference management system we do not need to assign an identity to
159 | every attendee's address (for a start all of the attendee's from a
160 | particular organization may share the same address). All we are
161 | concerned with are the values of the attributes of an address: street,
162 | city, state, etc. Value objects are usually immutable.
163 |
164 | ### Services
165 | You cannot always model everything as an object. For example, in the
166 | conference management system it makes sense to model a third-party
167 | payment processing system as a service: the system can pass the
168 | parameters required by the payment processing service and then receive a
169 | result back from the service. Notice that a common characteristic of a
170 | service is that it is stateless (unlike entities and value objects).
171 |
172 | > **Note:** Services are usually implemented as regular class libraries
173 | that expose a collection of stateless methods. A service in a DDD
174 | domain model is not a Web service; these are two different concepts.
175 |
176 | ##Aggregates and Aggregate Roots
177 | Whereas entities, value objects, and services are terms for the elements
178 | that DDD uses to describe things in the domain model, the terms
179 | aggregate and aggregate root relate specifically to the life cycle and
180 | grouping of those elements.
181 |
182 | When you design a system that allows multiple users to work on shared
183 | data, you will have to evaluate the trade-off between consistency and
184 | usability. At one extreme, when a user needs to make a change to some
185 | data, you could lock the entire database to ensure that the change is
186 | consistent. However, the system would be unusable for all other users
187 | for the duration of the update. At the other extreme, you could decide
188 | not enforce any locks at all, allowing any user to edit any piece of
189 | data at any time, but you would soon end up with inconsistent or corrupt
190 | data within the system. Choosing the correct granularity for locking to
191 | balance the demands of consistency and usability requires detailed
192 | knowledge of the domain:
193 |
194 | - You need to know which set of entities and value objects each
195 | transaction can potentially affect. For example, are there
196 | transactions in the system that can update attendee, conference, and
197 | room objects?
198 | - You need to know how far the relationships from one object extend
199 | through other entities and value objects within the domain, and where
200 | you must define the consistency boundaries. For example, if you delete
201 | a room object, should you also delete a projector object, or a set of
202 | attendee objects?
203 |
204 | DDD uses the term aggregate to define a cluster of related entities and
205 | value objects that form a consistency boundary within the system. That
206 | consistency boundary is usually based on transactional consistency.
207 |
208 | An aggregate root (also known as a Root Entity) is the gatekeeper object
209 | for the aggregate: all access to the objects within the aggregate must
210 | be done through the aggregate root, external entities are only permitted
211 | to hold a reference to the aggregate root, and all invariants should be
212 | checked by the aggregate root.
213 |
214 | In summary, aggregates are the mechanism that DDD uses to manage the
215 | complex set of relationships that typically exist between the many
216 | entities and value objects that exist within a typical domain model.
217 |
218 | # Bounded Contexts
219 | So far, the DDD concepts and terminology that we have briefly introduced
220 | are related to creating, maintaining, and using a domain model. For a
221 | large system, it may not be practical to maintain a single domain model;
222 | it is too large and complex to make it feasible to keep it coherent and
223 | consistent. To manage this scenario, DDD introduces the concepts of
224 | bounded contexts and multiple models. Within a system, you might choose
225 | to use multiple smaller models rather than a single large model, each
226 | one focusing on some aspect or grouping of functionality within the
227 | overall system. A bounded context is the context for one, particular
228 | domain model. Similarly, each bounded context (if implemented following
229 | the DDD approach) has its own ubiquitous language, or at least its own
230 | dialect of the domain's ubiquitous language.
231 |
232 | ![Figure 1][fig1]
233 |
234 | **Bounded contexts within a large, complex system**
235 |
236 | Figure 1 shows an example, drawn from a conference management system, of
237 | a system that is divided into multiple bounded contexts. In practice,
238 | there are likely to be more bounded contexts than the three shown in the
239 | diagram.
240 |
241 | There are no hard and fast rules that specify how big a bounded context
242 | should be. Ultimately it's a pragmatic issue that is determined by your
243 | requirements and the constraints on your project:
244 |
245 | > "Favoring Larger Bounded Contexts
246 | >
247 | > - Flow between user tasks is smoother when more is handled with a
248 | > unified model.
249 | > - It is easier to understand one coherent model than two distinct
250 | > ones plus mappings.
251 | > - Translation between two models can be difficult (sometimes
252 | > impossible).
253 | > - Shared language fosters clear team communication.
254 | >
255 | > Favoring Smaller Bounded Contexts
256 | >
257 | > - Communication overhead between developers is reduced.
258 | > - Continuous Integration is easier with smaller teams and code bases.
259 | > - Larger contexts may call for more versatile abstract models,
260 | > requiring skills that are in short supply.
261 | > - Different models can cater to special needs or encompass the jargon
262 | > of specialized groups of users, along with specialized dialects of
263 | > the Ubiquitous Language."
264 | >
265 | > Eric Evans, "Domain-Driven Design," p383.
266 |
267 | ## Bounded Contexts and Multiple Architectures
268 | A bounded context typically represents a slice of the overall system
269 | with clearly defined boundaries separating it from other bounded
270 | contexts within the system. If a bounded context is implemented by
271 | following the DDD approach, the bounded context will have its own domain
272 | model and its own ubiquitous language. Bounded contexts are also
273 | typically vertical slices through the system, so the implementation of a
274 | bounded context will include everything from the data store, right up to
275 | the UI.
276 |
277 | The same domain concept can exist in multiple bounded contexts: for
278 | example, the concept of an attendee in a conference management system
279 | might exist in the bounded context that deals with bookings, in the
280 | bounded context that deals with badge printing, and in the bounded
281 | context that deals with hotel reservations. From the perspective of the
282 | domain expert, these different versions of the attendee may require
283 | different behaviors and attributes. For example, in the bookings bounded
284 | context the attendee is associated with a registrant who makes the
285 | bookings and payments. Information about the registrant is not relevant
286 | in the hotel reservations bounded context where information such as
287 | dietary requirements or smoking preferences is important.
288 |
289 | One important consequence of this split is that you can use different
290 | implementation architectures in different bounded contexts. For example,
291 | one bounded context might be implemented using a DDD layered
292 | architecture, another might use a two-tier CRUD architecture, and
293 | another might use an architecture derived from the CQRS pattern. Figure
294 | 2 illustrates a system with multiple bounded contexts each using a
295 | different architectural style. It also highlights that each bounded
296 | context is typically end-to-end, from the persistence store through to
297 | the UI.
298 |
299 | ![Figure 2][fig2]
300 |
301 | **Multiple architectural styles within a large, complex application**
302 |
303 | This highlights another benefit of dividing the system into bounded
304 | contexts in addition to managing complexity. You can use an appropriate
305 | technical architecture for different parts of the system that addresses
306 | the specific characteristics of that part: is it a complex part of the
307 | system; does it contain core domain functionality; what is its expected
308 | lifetime?
309 |
310 | ## Bounded Contexts and Multiple Development Teams
311 | Clearly separating out the different bounded contexts, working with
312 | separate domain models and ubiquitous languages also makes it possible
313 | to parallelize the development work by using separate teams for each
314 | bounded context. This relates to the idea of using different technical
315 | architectures for different bounded contexts because the different
316 | development teams might have different skill sets and skill levels.
317 |
318 | ## Maintaining Multiple Bounded Contexts
319 | Although bounded contexts help to manage the complexity of large systems
320 | by dividing them up into more manageable pieces, it is unlikely that
321 | each bounded context will exist in isolation. Bounded contexts will need
322 | to exchange data with each other, and this exchange of data will be
323 | complicated if you need to translate between the different definitions
324 | of the same thing that exist in the different domain models. In our
325 | conference management system, we may need to move information about
326 | attendees between the bounded contexts that deal with conference
327 | bookings, badge printing, and hotel reservations. The DDD methodology
328 | offers a number of approaches for handling the interactions between
329 | multiple models in multiple bounded contexts such as using
330 | anti-corruption layers, or using shared kernels.
331 |
332 | > **Note:** At the technical implementation level, communication between
333 | bounded contexts is often handled asynchronously using events and a
334 | messaging infrastructure.
335 |
336 | For more information about how DDD deals with large systems and complex
337 | models, you should read "Part IV: Strategic Design" in the book
338 | "Domain-Driven Design: Tackling Complexity in the Heart of Software" by
339 | Eric Evans.
340 |
341 | # CQRS and DDD
342 | As stated in the introduction to this chapter, it is useful to
343 | understand a little of the terminology and concepts from DDD when you
344 | are learning about the CQRS pattern.
345 |
346 | Many of the ideas that informed the CQRS pattern arose from issues that
347 | DDD practitioners faced when applying the DDD approach to real-world
348 | problems. As such, if you decide to use the DDD approach, you may find
349 | that the CQRS pattern is a very natural fit for some of the bounded
350 | contexts that you identify within your system, and that it is relatively
351 | straightforward to move from your domain model to the physical
352 | implementation of the CQRS pattern.
353 |
354 | However, many people can point to projects where they have seen real
355 | benefits from implementing the CQRS pattern, but where they have not
356 | used the DDD approach for the domain analysis and model design.
357 |
358 |
359 |
360 | Comment [DRB]:
361 | Look for a suitable quote to use for the previous paragraph.
362 |
363 |
364 |
365 | In summary, the DDD approach is not a pre-requisite for implementing the
366 | CQRS pattern, but they do often go together.
367 |
368 |
369 |
370 | [dddquickly]: http://www.infoq.com/minibooks/domain-driven-design-quickly
371 |
372 | [fig1]: images/Reference_01_BCs.png?raw=true
373 | [fig2]: images/Reference_01_Archs.png?raw=true
--------------------------------------------------------------------------------
/Reference_02_CQRSIntroduction.markdown:
--------------------------------------------------------------------------------
1 | ## Chapter 2
2 | # Introducing the Command Query Responsibility Segregation Pattern (Chapter Title)
3 | This chapter introduces the topic of Command Query Responsibility
4 | Segregation (CQRS): it describes the CQRS pattern that lies at the heart
5 | of almost everything discussed in this guidance and shows how applying
6 | this pattern facilitates an approach to architecting elements of your
7 | enterprise application. It is important to understand that CQRS is not a
8 | silver bullet for all the problems that you encounter when you design
9 | and build enterprise applications, and that it is not a top-level
10 | architectural approach. You will see the full benefits of applying the
11 | CQRS pattern when you apply it selectively to key portions of your
12 | system. The chapter [Decomposing the Domain][j_chapter2] in "A CQRS
13 | Journey" describes how Contoso divided up the Conference Management
14 | System into bounded contexts and identified which bounded contexts would
15 | benefit from using the CQRS pattern.
16 |
17 | Subsequent chapters provide a more in depth guidance on how to apply the
18 | CQRS pattern and other related patterns to your implementation.
19 |
20 | # What is CQRS?
21 | In his book "[Object Oriented Software Construction][meyerbook],"
22 | Betrand Meyer introduced the term "[command query separation][cqsterm]"
23 | to describe the principle that an object's methods should be either
24 | commands, or queries. A query returns data and does not alter the state
25 | of the object, a command changes the state of an object but does not
26 | return any data. The benefit is improved clarity in relation to
27 | understanding what does, and what does not, change the state in your
28 | system.
29 |
30 | CQRS takes this principal a step further to define a simple pattern:
31 |
32 | > "CQRS is simply the creation of two objects where there was previously
33 | only one. The separation occurs based upon whether the methods are a
34 | command or a query (the same definition that is used by Meyer in
35 | Command and Query Separation, a command is any method that mutates
36 | state and a query is any method that returns a value)."
37 | Greg Young,
38 | [CQRS, Task Based UIs, Event Sourcing agh!][gregyoungcqrs]
39 |
40 | What is important and interesting about this simple pattern is the "how,
41 | where, and why" of using it when you build enterprise systems. Using
42 | this simple pattern enables you to meet a wide range of architectural
43 | challenges such as achieving scalability, managing complexity, and
44 | managing changing business rules in some portions of your system.
45 |
46 | ## Read and Write Sides
47 | Figure 1 shows a typical application of the CQRS pattern to a portion of
48 | an enterprise system. This approach shows how, when you apply the CQRS
49 | pattern, you can separate the read and write sides in this portion of
50 | the system.
51 |
52 | ![Figure 1][fig1]
53 |
54 | **A possible architectural implementation of the CQRS pattern**
55 |
56 | In figure 1, you can see how this portion of the system is split into a
57 | read-side, and a write-side. The object or objects or the read-side
58 | contain only query methods, and the objects on the write-side contain
59 | only command methods.
60 |
61 | There are several motivations for this segregation including:
62 |
63 | - In many business systems, there is a large imbalance between the
64 | number of reads and the number of writes. A system may process
65 | thousands of reads for every write. Segregating the two sides enables
66 | you to optimize the two sides independently. For example, you can
67 | scale out the read-side to support the larger number of read
68 | operations independently of the write-side.
69 | - Typically, commands involve complex business logic to ensure that the
70 | system writes correct and consistent data to the data store. Read
71 | operations are often much simpler than the write operations. A single
72 | conceptual model that tries to encapsulate both read and write
73 | operations may do neither well. Segregating the two sides ultimately
74 | results in simpler, more maintainable, and more flexible models.
75 | - Segregation can also occur in the data store. The write-side may use a
76 | database schema that is close to third normal form (3NF) and optimized
77 | for data modification, while the read-side uses a denormalized
78 | database that is optimized for fast query operations.
79 |
80 | > **Note:** Although figure 1 shows two data stores, applying the CQRS
81 | pattern does not mandate you to split the data store, or to use any
82 | particular technology such as a relational database, NoSQL store, or
83 | event store. You should view CQRS as a pattern that facilitates
84 | splitting the data store and enabling you to select from a range of
85 | storage mechanisms.
86 |
87 | There are some questions that might occur to you about the
88 | practicalities of adopting architecture such as the one shown in
89 | figure 1.
90 |
91 | - Although the individual models on the read-side and write-side might
92 | be simpler than a single compound model, the overall architecture is
93 | more complex than a traditional approach with a single model and a
94 | single data store. Haven't we just shifted the complexity?
95 | - How should we manage propagating the changes in the data store on the
96 | write-side to the read-side?
97 | - What if there is a delay while the updates on the write-side are
98 | propagated to the read-side?
99 | - What exactly do we mean when we talk about models?
100 |
101 | The remainder of this chapter will begin to address these questions and
102 | to explore the motivations for using the CQRS pattern in more detail.
103 | Later chapters will explore these issues in more depth.
104 |
105 | # CQRS and Domain-Driven Design
106 | The previous chapter "[CQRS in Context][r_chapter1]," introduced some
107 | of the concepts and terminology from Domain-Driven Design (DDD) that
108 | are relevant to the implementation of the CQRS pattern. Two areas are
109 | particularly significant to the CQRS pattern.
110 |
111 | > **Note:** "CQRS is an architectural style that is often enabling of
112 | DDD."
113 | Eric Evans, tweet February 2012.
114 |
115 | The first is the concept of the model:
116 |
117 | > "The model is a set of concepts built up in the heads of people on
118 | the project, with terms and relationships that reflect domain
119 | insight. These terms and interrelationships provide the semantics of
120 | a language that is tailored to the domain while being precise enough
121 | for technical development."
122 | Eric Evans, "Domain-Driven Design," p23.
123 |
124 |
125 | In his book "Domain-Driven Design," Eric Evans provides the following
126 | list of ingredients for effective modeling. This list helps to capture
127 | the idea of a model, but is no substitute for reading the book to gain
128 | a deeper understanding of the concept:
129 |
130 | - Models should be bound to the implementation.
131 | - You should cultivate a language based on the model.
132 | - Models should be knowledge rich.
133 | - You should brainstorm and experiment to develop the model.
134 |
135 | In figure 1, the implementation of the model exists on the write-side;
136 | it encapsulates the complex business logic in this portion of the
137 | system. The read-side is a simpler, read-only view of the system state
138 | accessed through queries.
139 |
140 | The second important concept is the way that DDD divides large,
141 | complex systems into more manageable units known as bounded contexts.
142 | A bounded context defines the context for a model:
143 |
144 | > "Explicitly define the context within which a model applies.
145 | Explicitly set boundaries in terms of team organization, usage
146 | within specific parts of the application, and physical manifestations
147 | such as code bases and database schemas. Keep the model strictly
148 | consistent within these bounds, but don't be distracted or confused
149 | by issues outside."
150 | Eric Evans, "Domain-Driven Design," p335.
151 |
152 | > **Note:** Other design approaches may use different terminology: for
153 | example in Event-Driven SOA, the concept of a service is roughly
154 | equivalent to a bounded context in DDD.
155 |
156 | When we say that you should apply the CQRS pattern to a portion of a
157 | system, we mean that you should implement the CQRS pattern within a
158 | bounded context.
159 |
160 | The reasons for identifying context boundaries for your domain models,
161 | are not necessarily the same reasons for choosing the portions of the
162 | system that should use the CQRS pattern. In DDD, a bounded context
163 | defines the context for a model and the scope of a ubiquitous language.
164 | You should implement the CQRS pattern to gain certain benefits for your
165 | application such as scalability, simplicity, and maintainability.
166 | Because of these differences, it may make sense to think about applying
167 | the CQRS pattern to business components rather than bounded contexts.
168 |
169 | > "A given Bounded Context should be divided into Business Components,
170 | where these Business Components have full UI through DB code, and are
171 | ultimately put together in composite UI's and other physical
172 | pipelines to fulfill the system's functionality.
173 | A Business Component can exist in only one Bounded Context.
174 | CQRS, if it is to be used at all, should be used within a Business
175 | Component."
176 | Udi Dahan, Udi & Greg Reach CQRS Agreement.
177 |
178 | It is quite possible that your bounded contexts map exactly on to your
179 | business components.
180 |
181 | > **Note:** For the remainder of this guide, we will choose to use the
182 | term *bounded context* in preference to the term *business
183 | component* to refer to the context within which we are implementing
184 | the CQRS pattern.
185 |
186 | In summary, you should **not** apply the CQRS pattern to the top-level
187 | of your system. You should clearly identify the different portions of
188 | your system that you can design and implement largely independently of
189 | each other, and then only apply the CQRS pattern to those portions
190 | where there are clear business benefits in doing so.
191 |
192 | # Introducing Commands, Events, and Messages
193 | DDD is an analysis and design approach that encourages you to use models
194 | and a ubiquitous language to bridge the gap between the business and the
195 | development team by fostering a common understanding of the domain. Of
196 | necessity, the DDD approach is oriented towards analyzing behavior
197 | rather than just data in the business domain, and this leads to a focus
198 | on modeling and implementing behavior in the software. A natural way to
199 | implement the domain model in code is to use commands and events. This
200 | is particularly true of applications that use a task-oriented UI.
201 |
202 | > **Note:** DDD is not the only approach where it is common to see
203 | tasks and behavior specified in the domain model implemented using
204 | commands and events. However, many advocates of the CQRS pattern are
205 | also strongly influenced by the DDD approach so it is common to see
206 | DDD terminology in use whenever there is a discussion of CQRS.
207 |
208 | *Commands* are imperatives; they are requests for the system to
209 | perform a task or action. For example, "book two places on conference X"
210 | or "allocate speaker Y to room Z." Commands are usually processed just
211 | once, by a single recipient.
212 |
213 | *Events* are notifications; they report something that has already
214 | happened to other interested parties. For example, "the customer's
215 | credit card has been billed $200" or "ten seats have been booked on
216 | conference X." Events can be processed multiple times, by multiple
217 | consumers.
218 |
219 | Both commands and events are types of message that are used to exchange
220 | data between objects. In DDD terms, these messages represent business
221 | behaviors and therefore help the system capture the business intent
222 | behind the message.
223 |
224 | One possible implementation of the CQRS pattern may use separate data
225 | stores for the read-side and the write-side, each data store optimized
226 | for the use cases that it supports. Events provide the basis of a
227 | mechanism for synchronizing the changes on the write-side (that result
228 | from processing commands) with the read-side. If the write-side raises
229 | an event whenever the state of the application changes, the read-side
230 | should respond to that event and update the data that is used by its
231 | queries and views. Figure 2 shows how commands and events could be used
232 | if you implement the CQRS pattern.
233 |
234 | ![Figure 2][fig2]
235 |
236 | **Commands and Events in the CQRS pattern**
237 |
238 | We also require some infrastructure to handle commands and events, and
239 | we will explore this in more detail in later chapters.
240 |
241 | > **Note:** Events are not the only way to manage the push
242 | synchronization of updates from the write-side to the read-side.
243 |
244 | # Why Should I Use CQRS?
245 | Stepping back from CQRS for a moment, one of the benefits of dividing
246 | the domain into bounded contexts in DDD is to enable you to identify and
247 | focus on those portions (bounded contexts) of the system that are more
248 | complex, subject to ever-changing business rules, or offer functionality
249 | that is key business differentiator.
250 |
251 | You should consider applying the CQRS pattern within a specific bounded
252 | context only when it provides identifiable business benefits, not
253 | because it is the default pattern that you consider.
254 |
255 | The most common business benefits that you might gain from applying the
256 | CQRS pattern are scalability, simplifying a complex aspect of your
257 | domain, and adding flexibility to your solution.
258 |
259 | ## Scalability
260 | In many enterprise systems, the number of reads vastly exceeds the
261 | number of writes, so your scalability requirements will be different for
262 | each side. By separating the read-side and the write-side into separate
263 | models within the bounded context, you now have the ability to scale
264 | each one of them independently. For example, if you are hosting
265 | applications in Windows Azure, you can use a different role for each
266 | side and then scale them independently by adding a different number of
267 | role instances to each.
268 |
269 | > **Note:** Scalability should not be the only reason why you choose to
270 | implement the CQRS pattern in a specific bounded context:
271 | "In a non-collaborative domain, where you can horizontally add more
272 | database servers to support more users/requests/data at the same time
273 | you're adding web servers - there is no real scalability problem
274 | (caveat, until you're Amazon/Google/Facebook scale).
275 | Database servers can be cheap - if using MySQL/SQL Express/others."
276 | Udi Dahan, When to avoid CQRS.
277 |
278 | ## Managing Complexity
279 | In complex areas of your domain, designing and implementing objects that
280 | are responsible for both reading and writing data can exacerbate the
281 | complexity. In many cases, the complex business logic is only applied
282 | when the system is handling updates and transactional operations; in
283 | comparison, read logic is often much simpler. When the business logic
284 | and read logic are mixed together in the same model, it becomes much
285 | harder to deal with difficult issues such as multiple-users, shared
286 | data, performance, transactions, consistency, and stale data. Separating
287 | the read logic and business logic into separate models makes it easier
288 | to separate out and address these complex issues. However, in many cases
289 | it may require some effort to disentangle and understand the existing
290 | model in the domain.
291 |
292 | > **Note:** Separation of concerns is the key motivation behind
293 | Bertrand Meyer's Command Query Separation Principle:
294 | "The really valuable idea in this principle is that it's extremely
295 | handy if you can clearly separate methods that change state from those
296 | that don't. This is because you can use queries in many situations
297 | with much more confidence, introducing them anywhere, changing their
298 | order. You have to be more careful with modifiers."
299 | Martin Fowler, CommandQuerySeparation
300 |
301 | Like many patterns, you can view the CQRS pattern as a mechanism for
302 | shifting some of the complexity inherent in your domain into something
303 | that is well known, well understood, and that offers a standard approach
304 | to solving certain categories of problem.
305 |
306 | Another potential benefit of simplifying the bounded context by
307 | separating out the read logic and the business logic is that it can make
308 | testing easier.
309 |
310 | ## Flexibility
311 | The flexibility of a solution that uses the CQRS pattern largely derives
312 | from the separation into the read-side and the write-side models. It
313 | becomes much easier to make changes on the read-side, for example adding
314 | a new query to support a new report screen in the UI, when you can be
315 | confident that you won't have any impact on the behavior of the business
316 | logic. On the write-side, having a model that concerns itself solely
317 | with the core business logic in the domain means that you have a simpler
318 | model to deal with than a model that includes read logic as well.
319 |
320 | In the longer term, a good, useful model that accurately describes your
321 | core domain business logic will become a valuable asset. It will enable
322 | you to be more agile in the face of a changing business environment and
323 | competitive pressures on your organization.
324 |
325 | This is related to the concept of Continuous Integration in DDD:
326 |
327 | > "Continuous integration means that all work within the context is
328 | being merged and made consistent frequently enough that when splinters
329 | happed they are caught and corrected quickly."
330 | Eric Evans, "Domain-Driven Design," p342.
331 |
332 | In some cases, it may be possible to have different development teams
333 | working on the write-side and the read-side, although in practice this
334 | will probably depend on how large the particular bounded context is.
335 |
336 | ## Focus on the Business
337 | If you use an approach like CRUD, then the technology tends to shape the
338 | solution. Adopting the CQRS pattern helps you to focus on the business
339 | and to build task-oriented UIs. A consequence of separating the
340 | different concerns into the read-side and the write-side is a solution
341 | that is more adaptable in the face of changing business requirements.
342 | This results in lower development and maintenance costs in the
343 | longer-term.
344 |
345 | # When Should I Use CQRS?
346 | Although we have outlined some of the reasons why you might decide to
347 | apply the CQRS pattern to some of the bounded contexts in your system,
348 | it is helpful to have some rules of thumb to help identify the bounded
349 | contexts that might benefit from applying the CQRS pattern.
350 |
351 | In general, applying the CQRS pattern may provide most value in those
352 | bounded contexts that are complex, include ever changing business rules,
353 | and deliver a significant competitive advantage to the business.
354 | Analyzing the business requirements, building a useful model, expressing
355 | it in code, and implementing it using the CQRS pattern for such a
356 | bounded context all take time and cost money. You should expect this
357 | investment to pay dividends in the medium to long-term. It is probably
358 | not worth making this investment if you don't expect to see returns such
359 | as increased adaptability and flexibility in the system, or reduced
360 | maintenance costs.
361 |
362 | ## Collaborative Domains
363 | Both Udi Dahan and Greg Young identify collaboration as the
364 | characteristic of a bounded context that provides the best indicator
365 | that you may see benefits from applying the CQRS pattern.
366 |
367 | > "In a collaborative domain, an inherent property of the domain is that
368 | multiple actors operate in parallel on the same set of data. A
369 | reservation system for concerts would be a good example of a
370 | collaborative domain – everyone wants the 'good seats'"
371 | Udi Dahan,
372 | [Why you should be using CQRS almost everywhere…][dahaneverywhere]
373 |
374 | The CQRS pattern is particularly useful where the collaboration involves
375 | complex decisions about what the outcome should be when you have
376 | multiple actors operating on the same, shared data. For example, does
377 | the rule "last one wins" capture the expected business outcome for your
378 | scenario, or do you need something more sophisticated? It's important to
379 | note that actors are not necessarily people; they could be other parts
380 | of the system that can independently operate on the same data.
381 |
382 | > **Note:** Collaborative behavior is a *good indicator* that there
383 | will be benefits from applying the CQRS pattern; this is not a hard
384 | and fast rule!
385 |
386 | Such collaborative portions of the system are often the most complex,
387 | fluid, and significant bounded contexts. However, this characteristic is
388 | only a guide: not all collaborative domains benefit from the CQRS
389 | pattern, and some non-collaborative domains do benefit from the CQRS
390 | pattern.
391 |
392 | ## Stale Data
393 | In a collaborative environment where multiple users can operate on the
394 | same data simultaneously, you also encounter the issue of stale data: if
395 | one user is viewing a piece of data while another user changes it, then
396 | the first user's view of the data is stale.
397 |
398 | Whatever architecture you choose, you must address this problem; for
399 | example, by using a particular locking scheme in your database, or
400 | defining the refresh policy for the cache from which your users read
401 | data.
402 |
403 | The CQRS pattern helps you address the issue of stale data explicitly at
404 | the architectural level. Changes to data happen on the write-side, users
405 | view data by querying the read-side. Whatever mechanism you chose to use
406 | to push the changes from the write-side to the read-side is also the
407 | mechanism that controls when the data on the read-side becomes stale,
408 | and for how long. This differs from other architectures, where
409 | management of stale data is more of an implementation detail that is not
410 | always addressed in a standard or consistent manner.
411 |
412 | > "Standard layered architectures don't explicitly deal with either of
413 | these issues. While putting everything in the same database may be one
414 | step in the direction of handling collaboration, staleness is usually
415 | exacerbated in those architectures by the use of caches as a
416 | performance-improving afterthought."
417 | Udi Dahan talking about collaboration and staleness,
418 | [Clarified CQRS][dahanclarify].
419 |
420 | In the chapter "[A CQRS/ES Deep Dive][r_chapter4]," we will look at how
421 | the synchronization mechanism between write-side and the read-side
422 | determines how you manage the issue of stale data in your application.
423 |
424 | ## Moving to the Cloud
425 | Moving an application to the cloud or developing an application for the
426 | cloud is not a sufficient reason for choosing to implement to CQRS
427 | pattern. However, many of the drivers for using the cloud such as
428 | requirements for scalability, elasticity, and agility are also drivers
429 | for adopting the CQRS pattern. Furthermore, many of the services
430 | typically offered as part of a PaaS cloud-computing platform are well
431 | suited for building the infrastructure for a CQRS implementation: for
432 | example, highly scalable data stores, messaging services, and caching
433 | services.
434 |
435 | #When Should I Avoid CQRS?
436 | Simple, static, non-core bounded contexts are less likely to warrant the
437 | upfront investment in detailed analysis, modeling, and complex
438 | implementation. Again, non-collaborative bounded contexts are less
439 | likely to see benefits from applying the CQRS pattern.
440 |
441 | In most systems, the majority of bounded contexts will probably not
442 | benefit from using the CQRS pattern. You should only use the pattern
443 | when you can identify clear business benefits from so doing.
444 |
445 | # Summary
446 | The CQRS pattern is an enabler for building individual portions (bounded
447 | contexts) in your system. Identifying where to use the CQRS pattern
448 | requires you to analyze the trade-offs between the initial cost and
449 | overhead of implementing the pattern and the future business benefits.
450 | Useful heuristics for identifying where you might apply the CQRS pattern
451 | are to look for components that are complex, involve fluid business
452 | rules, deliver competitive advantage to the business, and are
453 | collaborative.
454 |
455 | The next chapters will discuss how to implement the CQRS pattern in more
456 | detail, for example explaining specific class-pattern implementations,
457 | exploring how to synchronize the data between the write-side and
458 | read-side, and describing different options for storing data.
459 |
460 |
461 |
462 | [j_chapter2]: Journey_02_BoundedContexts.markdown
463 | [r_chapter1]: Reference_01_CQRSContext.markdown
464 | [r_chapter4]: Reference_04_DeepDive.markdown
465 |
466 | [meyerbook]: http://www.amazon.com/gp/product/0136291554
467 | [cqsterm]: http://martinfowler.com/bliki/CommandQuerySeparation.html
468 | [gregyoungcqrs]: http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/
469 | [dahaneverywhere]:http://www.udidahan.com/2011/10/02/why-you-should-be-using-cqrs-almost-everywhere%E2%80%A6/
470 | [dahanclarify]: http://www.udidahan.com/2009/12/09/clarified-cqrs/
471 |
472 | [fig1]: images/Reference_02_Arch_01.png?raw=true
473 | [fig2]: images/Reference_02_Messages.png?raw=true
--------------------------------------------------------------------------------
/Reference_03_DeepDive.markdown:
--------------------------------------------------------------------------------
1 | ## Chapter 4
2 |
3 | # A CQRS/ES Deep Dive (Chapter Title)
4 |
5 | # The Domain Model
6 |
7 |
13 |
14 | # Commands and CommandHandlers
15 |
16 | # Events and EventHandlers
17 |
18 | # Embracing Eventual Consistency
19 |
20 | Maintaining the consistency of business data is a key requirement in all
21 | enterprise systems. One of the first things that many developers learn
22 | in relation to database systems is the ACID properties of transactions:
23 | transactions must ensure that the stored data is consistent as well
24 | transactions being atomic, isolated, and durable. Developers also become
25 | familiar with complex concepts such as pessimistic and optimistic
26 | concurrency, and their performance characteristics in particular
27 | scenarios. They may also need to understand the different isolation
28 | levels of transactions: serializable, repeatable reads, read committed,
29 | and read uncommitted.
30 |
31 | In a distributed computer system, there are some additional factors that
32 | are relevant to consistency. The CAP theorem states that it is
33 | impossible for a distributed computer system to provide the following
34 | three guarantees simultaneously:
35 |
36 | 1. Consistency. A guarantee that all the nodes in the system see the
37 | same data at the same time.
38 | 2. Availability. A guarantee that the system can continue to operate
39 | even if a node is unavailable.
40 | 3. Partition tolerance. A guarantee that the system continues to operate
41 | despite the nodes being unable to communicate.
42 |
43 | For more information about the [CAP theorem][captheorem], see CAP
44 | theorem on Wikipedia.
45 |
46 | The concept of *eventual consistency* offers to way to make these three
47 | guarantees simultaneously by relaxing the guarantee of consistency. In
48 | the CAP theorem, the consistency guarantee specifies that all the nodes
49 | should see the same data *at the same time*; if instead, we state that
50 | all the nodes will eventually see the same data, we can make the
51 | consistency guarantee compatible with the other two. Another way of
52 | viewing this is to say that we will accept that at any given time, some
53 | of the data seen by users of the system could be stale. For many
54 | business scenarios, this turns out to be perfectly acceptable: a
55 | business user will accept that the information they are seeing on a
56 | screen may be a few seconds, or even minutes out of date. Depending on
57 | the details of the scenario, the business user can refresh the display a
58 | bit later on to see what has changed, or simply accepts that what they
59 | see is always slightly out of date. There are some scenarios where this
60 | delay is unacceptable, but they tend to be the exception rather than the
61 | rule.
62 |
63 | # Eventual Consistency and CQRS
64 |
65 | How does the concept of eventual consistency relate to the CQRS pattern?
66 | A typical implementation of the CQRS pattern is a distributed system
67 | made up of one node for the write-side, and one or more nodes for the
68 | read-side. Your implementation must provide some mechanism for
69 | synchronizing data between these two sides. This is not a complex
70 | synchronization task because all of the changes take place on the
71 | write-side, so the synchronization process only needs to push changes
72 | from the write-side to the read-side.
73 |
74 | If you decide that the two sides must always be consistent, then you
75 | will need to introduce a distributed transaction that spans both sides
76 | as shown in figure 1.
77 |
78 | ![Figure 1][fig1]
79 |
80 | **Using a distributed transaction to maintain consistency**
81 |
82 | The problems that may result from this approach relate to performance
83 | and availability. Firstly, both sides will need to hold locks until both
84 | sides are ready to commit, in other words the transaction can only
85 | complete as fast as the slowest participant can.
86 |
87 | This transaction may include more than two participants. If we are
88 | scaling the read-side by adding multiple instances, the transaction must
89 | span all of those instances.
90 |
91 | Secondly, if one node fails for any reason or does not complete the
92 | transaction, the transaction cannot complete. In terms of the CAP
93 | theorem, by guaranteeing consistency, we cannot guarantee the
94 | availability of the system.
95 |
96 | If you decide to relax your consistency constraint and specify that your
97 | read-side only needs to be eventually consistent with the write-side,
98 | you can change the scope of your transaction. Figure 2 shows how you can
99 | make the read-side eventually consistent with the write-side by using a
100 | reliable messaging transport to propagate the changes.
101 |
102 | ![Figure 2][fig2]
103 |
104 | **Using a reliable message transport**
105 |
106 | In this example, you can see that there is still a transaction. The
107 | scope of this transaction includes saving the changes to the data store
108 | on the write-side, and placing a copy of the change onto the queue that
109 | pushes the change to the read-side.
110 |
111 | This solution does not suffer from the potential performance problems
112 | that you saw in the original solution if you assume that the messaging
113 | infrastructure allows you to quickly add messages to a queue. This
114 | solution is also no longer dependent on all of the read-side nodes being
115 | constantly available because the queue acts as a buffer for the messages
116 | addressed to the read-side nodes.
117 |
118 | > **Note:** In practice, the messaging infrastructure is likely to use a
119 | publish/subscribe topology rather than a queue to enable multiple
120 | read-side nodes to receive the messages.
121 |
122 | This third example in figure z shows a way that you can do away with the
123 | need for a distributed transaction.
124 |
125 | ![Figure 3][fig3]
126 |
127 | **No distributed transactions**
128 |
129 | This example depends on functionality in the write-side data store: it
130 | must be able to send a message in response to every update that the
131 | write-side model makes to the data. This approach lends itself
132 | particularly well to the scenario where you combine CQRS with event
133 | sourcing. If the event store can send a copy of every event that it
134 | saves onto a message queue, then you can make the read-side eventually
135 | consistent by using this infrastructure feature.
136 |
137 |
138 |
139 | Comment [DRB]:
140 | [Add a reference to an event store that offers this functionality.]
141 |
142 |
143 |
144 | Queries
145 |
146 | # Optimizing the Read Side
147 |
148 | # Optimizing the Write Side
149 |
150 |
151 |
152 | Comment [DRB]:
153 | parallelizing command handling, events, etc.
154 | Snapshots in the event store
155 |
156 |
163 |
164 | Comment [DRB]:
165 | Discuss role of messaging infrastructure and CQRS. Transport for commands and events. Topologies: queues and pub/sub.
166 |
167 |
172 |
173 | Comment [DRB]:
174 | Problems that arise with out of order messages, duplicate messages, lost messages. Strategies for dealing with these issues.
175 | Need to discuss issues of dropped, multiple, out of order delivery here.
176 | Multiple delivery of messages or lost messages. This is a general messaging issueIt is possible that due to a problem in the messaging infrastructureIf it is possible that the read-side could receive multiple copies of a single event, for example because of a problem in the messaging infrastructure, you must ensure that your system state remains correct. For example, an individual subscriber could receive multiple copies introducing a temporary error that you could fix by replaying the events. For example, if a reservation event is delivered twice, then you may end up with an inaccurate figure for the total number of bookings for a conference. Designing event messages to be idempotent or assigning messages unique identifiers are possible strategies to help address this problem. Similar problems can result from subscribers not receiving messages.
177 | Out of order delivery. In some scenarios, the specific ordering of a set of event messages may be significant. Again, it may be that your messaging infrastructure does not guarantee ordered delivery. Assigning sequence numbers to messages, using batches, or using sagas are all possible strategies for addressing this problem. The chapter "A CQRS/ES Deep Dive" includes a discussion of sagas.
178 |
179 |
185 |
186 | Comment [DRB]:
187 | What to expect from an infrastructure messaging service.
188 | Reference Windows Azure Service Bus section below, but mention other products.
189 |
190 |
191 |
192 | # Task-based UIs
193 |
194 | # Taking Advantage of Windows Azure
195 |
196 | In the chapter "[Introducing the Command Query Responsibility
197 | Segregation Pattern][r_chapter2]," we suggested that the motivations for
198 | hosting an application in the cloud were similar to the motivations for
199 | implementing the CQRS pattern: scalability, elasticity, and agility.
200 | This section describes in more detail how a CQRS implementation might
201 | use some of specific features of the Windows Azure platform to provide
202 | some of the infrastructure that you typically need when you implement
203 | the CQRS pattern.
204 |
205 | ## Scaling Out Using Multiple Role Instances
206 |
207 | When you deploy an application to Windows Azure, you deploy the
208 | application to roles in your Windows Azure environment; a Windows Azure
209 | application typically consists of multiple roles. Each role has
210 | different code and performs a different function within the application.
211 | In CQRS terms, you might have one role for the implementation of the
212 | write-side model, one role for the implementation of the read-side
213 | model, and another role for the UI elements of the application.
214 |
215 | After you deploy the roles that make up your application to Windows
216 | Azure, you can specify (and change dynamically) the number of running
217 | instances of each role. By adjusting the number of running instances of
218 | each role, you can elastically scale your application in response to
219 | changes in levels of activity. One of the motivations for using the CQRS
220 | pattern is the ability to scale the read-side and the write-side
221 | independently given their typically different usage patterns. For
222 | information about how to automatically scale roles in Windows Azure, see
223 | "[The Autoscaling Application Block][aab]" on MSDN.
224 |
225 | ## Implementing an Event Store Using Windows Azure Table Storage
226 |
227 | ## Implementing a Messaging Infrastructure Using the Windows Azure Service Bus
228 |
229 |
230 | [r_chapter1]: Reference_01_CQRSContext.markdown
231 | [r_chapter2]: Reference_02_CQRSIntroduction.markdown
232 | [r_chapter4]: Reference_04_DeepDive.markdown
233 |
234 |
235 | [captheorem]: http://en.wikipedia.org/wiki/CAP_theorem
236 | [aab]: http://msdn.microsoft.com/en-us/library/hh680892(PandP.50).aspx
237 |
238 |
239 | [fig1]: images/Reference_04_Consistency_01.png?raw=true
240 | [fig2]: images/Reference_04_Consistency_02.png?raw=true
241 | [fig3]: images/Reference_04_Consistency_03.png?raw=true
--------------------------------------------------------------------------------
/Reference_03_ESIntroduction.markdown:
--------------------------------------------------------------------------------
1 | ## Chapter 3
2 | # Introducing Event Sourcing (Chapter Title)
3 |
4 | Event sourcing and Command Query Responsibility Segregation (CQRS) are
5 | frequently mentioned together. Although neither one necessarily implies
6 | the other, you will see that they do complement each other. This chapter
7 | introduces the key concepts that underlie event sourcing, with some
8 | pointers to the potential relationship with the CQRS pattern. This
9 | chapter is an introduction; the chapter "[A CQRS/ES Deep
10 | Dive][r_chapter4]" explores event sourcing and its relationship with
11 | CQRS in more depth.
12 |
13 | Before going any further, we should have a basic definition of events
14 | that captures their essential characteristics:
15 |
16 |
17 | - Events happen in the past. For example, "the speaker was booked,"
18 | "the seat was reserved," "the cash was dispensed." Notice how we
19 | describe these events using the past tense.
20 | - Events are immutable. Because events happen in the past, they cannot
21 | be changed or undone. Subsequent events may alter or negate the
22 | effects of earlier events. For example, "the reservation was
23 | cancelled."
24 | - Events are one-way messages. Events have a single source (publisher)
25 | that fires the event. One or more recipients (subscribers) may receive
26 | events.
27 | - Typically, events include parameters that provide additional
28 | information about the event. For example, "Seat E23 was booked by
29 | Alice."
30 | - In the context of event sourcing, events should describe business
31 | intent. For example, "Seat E23 was booked by Alice" describes in
32 | business terms what has happened and is more descriptive in business
33 | terms than, "In the bookings table, the row with key E23 had the name
34 | field updated with the value Alice."
35 |
36 | We will also assume that the events being discussed in this chapter are
37 | associated with aggregates; see the chapter "[CQRS in
38 | Context][r_chapter1]" for a descriptions of the DDD terms: aggregates,
39 | aggregate roots, and entities. The features of aggregates that are
40 | relevant to events and event sourcing are:
41 |
42 | - Aggregates define consistency boundaries for groups of related
43 | entities; therefore, you can use an event raised by an aggregate to
44 | notify interested parties that a transaction (consistent set of
45 | updates) has taken place on that group of entities.
46 | - Every aggregate has a unique ID; therefore, you can use that ID to
47 | record which aggregate in the system was the source of a particular
48 | event.
49 |
50 | For the remainder of this chapter, we will use the term aggregate to
51 | refer to a cluster of associated objects that are treated as a unit for
52 | the purposes of data changes. This does mean that event sourcing is
53 | directly related to the DDD approach; we are simply using the
54 | terminology from DDD to try to maintain some consistency in our
55 | terminology.
56 |
57 | # What is Event Sourcing?
58 |
59 | Event sourcing is a way of storing your application's state by storing
60 | the history that determines the current state of your application. For
61 | example, a conference management system needs to track the number of
62 | completed bookings for a conference so it can check whether there are
63 | still seats available when someone tries to make a new booking. The
64 | system could store the total number of bookings for a conference in two
65 | ways:
66 |
67 | - It could store the total number of bookings for a particular
68 | conference and adjust this number whenever someone makes or cancels a
69 | booking. You can think of the number of bookings as being an integer
70 | value stored in a specific column of a table that has a row for each
71 | conference in the system.
72 | - It could store all the booking and cancellation events for each
73 | conference and then calculate the current number of bookings by
74 | replaying the events associated with the conference for which you
75 | wanted to check the current total number of bookings.
76 |
77 | ![Figure 1][fig1]
78 |
79 | **Using an ORM mapping layer**
80 |
81 | Figure 1 illustrates the first approach to storing the total number of
82 | reservations. The following list of steps corresponds to the numbers in
83 | the diagram:
84 |
85 |
86 | 1. A command is issued, probably from the UI, to book two attendees onto
87 | a conference with an ID of 157.
88 | 2. A booking aggregate executes some domain logic and then issues a
89 | command to the aggregate for conference 157.
90 | 3. If necessary, the aggregate is populated with data through the ORM
91 | layer that issues a query against the table (or tables) that hold the
92 | required data. This includes the existing number of reservations for
93 | the conference.
94 | 4. The conference aggregate performs its domain logic, which includes
95 | calculating the new number of reservations for the conference.
96 | 5. The information in the conference aggregate is persisted to the data
97 | store. The ORM layer constructs the necessary update (or updates)
98 | that must be executed.
99 |
100 | This diagram provides a deliberately simplified view of the process. In
101 | practice, the mapping performed by the ORM will be significantly more
102 | complex. You will also need to consider exactly when the load and save
103 | operations must happen to balance the demands of consistency,
104 | reliability, scalability, and performance.
105 |
106 |
107 | ![Figure 2][fig2]
108 |
109 | **Using event sourcing**
110 |
111 | Figure 2 illustrates the second approach, using event sourcing in place
112 | of an ORM layer and an RDMS.
113 |
114 |
115 | > **Note:** You might decide to implement the event store using an
116 | RDBMS, but it will have a much simpler schema than in the first
117 | approach. You can also use a custom event store.
118 |
119 | The following list of steps corresponds to the numbers in the diagram.
120 | Note that steps one, two, and four are the same.
121 |
122 |
123 | 1. A command is issued, probably from the UI, to book two attendees
124 | onto a conference with an ID of 157.
125 | 2. A booking aggregate executes some domain logic and then issues a
126 | command to the aggregate for conference 157.
127 | 3. If necessary, the aggregate is populated by querying for all of the
128 | events belonging to conference aggregate 157.
129 | 4. The conference aggregate performs its domain logic, which includes
130 | calculating the new number of reservations for the conference.
131 | 5. The aggregate appends the event that records making two new bookings
132 | to the list of events associated with the aggregate in the event
133 | store.
134 |
135 | This second approach is simpler because it dispenses with the ORM layer,
136 | and replaces a complex relational schema in the data store with a much
137 | simpler one. The data store only needs to support querying for events by
138 | aggregate ID and appending new events. You will still need to consider
139 | performance and scalability optimizations for reading from and writing
140 | to the store, but the impact of these optimizations on reliability and
141 | consistency should be much easier to understand.
142 |
143 | > **Note:** Some optimizations to consider are using snapshots so you
144 | don't need to query the full list of events to obtain the current
145 | state of the aggregate, and maintaining cached copies of aggregates in
146 | memory.
147 |
148 | You must also ensure that you have a mechanism that enables an aggregate
149 | to rebuild its state by querying for its list of historical events.
150 |
151 | What you have also gained with the second approach is a complete
152 | history, or audit trail, of the bookings and cancellations for a
153 | conference.
154 |
155 | In some domains, such as accounting, event sourcing is the natural,
156 | well-established approach: accounting systems store individual
157 | transactions from which it always possible to reconstruct the current
158 | state of the system. Event sourcing can bring a number of benefits to
159 | other domains.
160 |
161 | # Why Should I Use Event Sourcing?
162 |
163 | So far, the only justification we have offered for event sourcing is the
164 | fact that it stores a complete history of the events associated with the
165 | aggregates in your domain. This is a vital feature in some domains such
166 | as accounting where you need a complete audit trail of the financial
167 | transactions, and where events must be immutable. Once a transaction has
168 | happened, you cannot delete or change it, although you can create a new
169 | corrective or reversing transaction if necessary.
170 |
171 | The following list describes some of the additional benefits that you
172 | can derive from using event sourcing. The significance of the individual
173 | benefits will vary depending on the domain you are working in.
174 |
175 | - **Performance.** Because events are immutable, you can use an
176 | append-only when you save them. Events are also simple, stand-alone
177 | objects. Both these factors can lead to better performance and
178 | scalability for the system than approaches that use complex relational
179 | storage models.
180 | - **Simplification.** Events are simple objects that describe what has
181 | happened in the system. By simply saving events, you are avoiding the
182 | complications associated with saving complex domain objects to a
183 | relational store: the object-relational impedance mismatch.
184 | - **Audit Trail.** Events are immutable and store the full history of
185 | the state of the system. As such, they can provide a detailed audit
186 | trail of what has taken place within the system.
187 | - **Integration with other sub-systems.** Events provide a useful way of
188 | communicating with other sub-systems. Your event store can publish
189 | events to notify other interested sub-systems of changes to the
190 | application's state. Again, the event store provides a complete
191 | record of all the events that it published to other systems.
192 | - **Deriving additional business value from event history.** By storing
193 | events, you have the ability to determine what the state of the system
194 | was at any previous point in time by querying the events associated
195 | with a domain object up to that point in time. This enables you to
196 | answer historical questions from the business about the system. In
197 | addition, you cannot predict what questions the business might want to
198 | ask about the information stored in a system. If you store your
199 | events, you are not discarding information that may prove to be
200 | valuable in the future.
201 | - **Production troubleshooting.** You can use the event store to
202 | troubleshoot problems in a production system by taking a copy of the
203 | production event store and replaying it in a test environment. If you
204 | know the time that an issue occurred in the production system then you
205 | can easily replay the event stream up to that point to observe exactly
206 | what was happening.
207 | - **Fixing errors.** You might discover a coding error that results in
208 | the system calculating an incorrect value. Rather than fixing the
209 | coding error and performing a risky manual adjustment on a stored item
210 | of data, you can fix the coding error and replay the event stream so
211 | that the system calculates the value correctly based on the new
212 | version of the code.
213 |
214 | The chapter "[A CQRS/ES Deep Dive][r_chapter4]" discusses these benefits
215 | in more detail. There are also many illustrations of these benefits in
216 | these chapters in the section "A CQRS Journey."
217 |
218 | # Event Sourcing Concerns
219 |
220 | The previous section described some of the benefits that you might
221 | obtain if you decide to use event sourcing in your system. However,
222 | there may be some concerns that you will need to address if you decide
223 | to use event sourcing in your system:
224 |
225 | - **Performance.** Although event sourcing typically improves the
226 | performance of updates, you may need to consider the time it takes to
227 | load domain object state by querying the event store for all of the
228 | events that relate to the state of an aggregate. Using snapshots may
229 | enable you limit the amount of data that you need to load: you can go
230 | back to the latest snapshot and replay the events from that point
231 | forward. See the chapter "A CQRS/ES Deep Dive" for more information
232 | about snapshots.
233 | - **Versioning.** You may find it necessary to change the definition of
234 | a particular event type or aggregate at some point in the future. You
235 | must consider how your system will be able to handle multiple versions
236 | of an event type and aggregates.
237 |
238 | # CQRS/ES
239 | The chapter "Introducing Command Query Responsibility Segregation"
240 | suggested that events can form the basis of the push synchronization of
241 | the applications state from the data store on the write-side to the data
242 | store on the read-side. Remember that typically the read-side data-store
243 | contains denormalized data that is optimized for the queries that are
244 | run against your data, for example to display information in your
245 | application's UI.
246 |
247 | You can use the events that you persist in your event store to propagate
248 | all the updates made on the write-side to the read-side. The read-side
249 | can use the information contained in the events to maintain whatever
250 | de-normalized data you require on the read-side to support your queries.
251 |
252 | ![Figure 3][fig3]
253 |
254 | **CQRS and Event Sourcing**
255 |
256 | Notice how the write-side publishes events after it persists them to the
257 | event store. This avoids the need to use a two-phase commit, which would
258 | be the case if the aggregate were responsible for saving the event to
259 | the even store and publishing the event to the read-side.
260 |
261 | Normally, these events will enable you to keep the data on the read-side
262 | up to date in close to real-time; there will be some delay due to the
263 | transport mechanism, and the chapter "[A CQRS/ES Deep Dive][r_chapter4]"
264 | discusses the possible consequences of this delay.
265 |
266 | You can also rebuild the data on the read-side from scratch at any time
267 | by replaying the events from your event-store on the write-side. You
268 | might need to do this if the read-side data store got out of
269 | synchronization for some reason, or because you needed to modify the
270 | structure of the read-side data store to support a new query.
271 |
272 | You need to be careful replaying the events from the event store to
273 | rebuild the read-side data store if other bounded contexts also
274 | subscribe to the same events. It might be easy to empty the read-side
275 | data store before replaying the events, it might not be so easy to
276 | ensure the consistency of another bounded context if it sees a duplicate
277 | stream of events.
278 |
279 | Remember that the CQRS pattern does not mandate you to use different
280 | stores on the read-side and the write-side. You could decide to use a
281 | single relational store with a schema in third normal form and a set of
282 | denormalized views over that schema. However, replaying events is a very
283 | convenient mechanism for re-synchronizing the read-side data store with
284 | the write-side data store.
285 |
286 | # Standalone Event Sourcing
287 |
288 | You can use event sourcing without also applying the CQRS pattern. The
289 | ability to rebuild the application state, to mine the event history for
290 | new business data, and to simplify the data storage part of the
291 | application are all valuable in some scenarios.
292 |
293 |