├── README.md
├── _config.yml
├── assets
├── MONGO_SE1.png
├── RDBMS_MongoDB_Mapping.jpg
├── StorageEngine.png
├── column-database.png
├── document-database.png
├── graph-database.png
├── horizontal-vs-vertical-scaling.png
├── journaling.png
├── key-value-database.png
├── mongoSplitting.svg
├── mongodb-compass.png
├── objectid.png
├── replica-set.png
├── sharding.png
└── star.png
├── mongodb-code.md
├── mongodb-commands.md
└── nodemongodb
├── ReadMe.md
├── controllers
└── crud.js
├── index.js
├── models
└── todo.js
├── package-lock.json
├── package.json
├── routes
└── todo.js
└── utils
└── dbconfig.js
/README.md:
--------------------------------------------------------------------------------
1 | # MongoDB Basics
2 |
3 | > *Click ★ if you like the project. Your contributions are heartily ♡ welcome.*
4 |
5 |
6 |
7 | ## Table of Contents
8 |
9 | * *[MongoDB Commands](mongodb-commands.md)*
10 | * *[MongoDB Coding Practice](mongodb-code.md)*
11 |
12 |
13 |
14 | ## Q. What are the different types of NoSQL databases?
15 |
16 | NoSQL is a non-relational DBMS, that does not require a fixed schema, avoids joins, and is easy to scale. The purpose of using a NoSQL database is for distributed data stores with humongous data storage needs. NoSQL is used for Big data and real-time web apps.
17 |
18 | **Types of NoSQL Databases:**
19 |
20 | * Document databases
21 | * Key-value stores
22 | * Column-oriented databases
23 | * Graph databases
24 |
25 | **1. Document databases:**
26 |
27 | A document database stores data in JSON, BSON , or XML documents. In a document database, documents can be nested. Particular elements can be indexed for faster querying.
28 |
29 | Documents can be stored and retrieved in a form that is much closer to the data objects used in applications, which means less translation is required to use the data in an application. SQL data must often be assembled and disassembled when moving back and forth between applications and storage.
30 |
31 | **Example:** Amazon SimpleDB, CouchDB, MongoDB, Riak, Lotus Notes are popular Document originated DBMS systems.
32 |
33 |
34 |
35 |
36 |
37 | **2. Key-value Stores:**
38 |
39 | Data is stored in key/value pairs. It is designed in such a way to handle lots of data and heavy load. Key-value pair storage databases store data as a hash table where each key is unique, and the value can be a JSON, BLOB(Binary Large Objects), string, etc.
40 |
41 | **Example:** of key-value stores are Redis, Voldemort, Riak, and Amazon\'s DynamoDB.
42 |
43 |
44 |
45 |
46 |
47 | **3. Column-Oriented Databases:**
48 |
49 | Column-oriented databases work on columns and are based on BigTable paper by Google. Every column is treated separately. The values of single column databases are stored contiguously.
50 |
51 | They deliver high performance on aggregation queries like SUM, COUNT, AVG, MIN, etc. as the data is readily available in a column.
52 |
53 | **Example:** Column-based NoSQL databases are widely used to manage data warehouses, business intelligence, CRM, Library card catalogs, HBase, Cassandra, HBase, Hypertable are examples of a column-based database.
54 |
55 |
56 |
57 |
58 |
59 | **4. Graph Databases:**
60 |
61 | A graph type database stores entities as well the relations amongst those entities. The entity is stored as a node with the relationship as edges. An edge gives a relationship between nodes. Every node and edge has a unique identifier.
62 |
63 | Compared to a relational database where tables are loosely connected, a Graph database is a multi-relational in nature. Traversing relationships as fast as they are already captured into the DB, and there is no need to calculate them.
64 |
65 | Graph base databases mostly used for social networks, logistics, spatial data.
66 |
67 | **Example:** Neo4J, Infinite Graph, OrientDB, FlockDB are some popular graph-based databases.
68 |
69 |
76 |
77 | ## Q. What is MongoDB?
78 |
79 | **MongoDB** is a document-oriented NoSQL database used for high volume data storage. Instead of using tables and rows as in the traditional relational databases, MongoDB makes use of collections and documents. Documents consist of key-value pairs which are the basic unit of data in MongoDB. Collections contain sets of documents and function which is the equivalent of relational database tables.
80 |
81 | **Key Features:**
82 |
83 | * Document Oriented and NoSQL database.
84 | * Supports Aggregation
85 | * Uses BSON format
86 | * Sharding (Helps in Horizontal Scalability)
87 | * Supports Ad Hoc Queries
88 | * Schema Less
89 | * Capped Collection
90 | * Indexing (Any field in MongoDB can be indexed)
91 | * MongoDB Replica Set (Provides high availability)
92 | * Supports Multiple Storage Engines
93 |
94 | **Key Components:**
95 |
96 | **1. _id**: The `_id` field represents a unique value in the MongoDB document. The `_id` field is like the document\'s primary key. If you create a new document without an `_id` field, MongoDB will automatically create the field.
97 |
98 | **2. Collection**: This is a grouping of MongoDB documents. A collection is the equivalent of a table which is created in any other RDMS such as Oracle.
99 |
100 | **3. Cursor**: This is a pointer to the result set of a query. Clients can iterate through a cursor to retrieve results.
101 |
102 | **4. Database**: This is a container for collections like in RDMS wherein it is a container for tables. Each database gets its own set of files on the file system. A MongoDB server can store multiple databases.
103 |
104 | **5. Document**: A record in a MongoDB collection is basically called a document. The document, in turn, will consist of field name and values.
105 |
106 | **6. Field**: A name-value pair in a document. A document has zero or more fields. Fields are analogous to columns in relational databases.
107 |
108 | **Example:**
109 |
110 | Connecting MongoDB Cloud using MongoDB Compass
111 |
112 |
121 |
122 | ## Q. What are Indexes in MongoDB?
123 |
124 | Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
125 |
126 | Indexes are special data structures that store a small portion of the collection\'s data set in an easy to traverse form. The index stores the value of a specific field or set of fields, ordered by the value of the field. The ordering of the index entries supports efficient equality matches and range-based query operations. In addition, MongoDB can return sorted results by using the ordering in the index.
127 |
128 | **Example:**
129 |
130 | The `createIndex()` method only creates an index if an index of the same specification does not already exist. The following example ( using Node.js ) creates a single key descending index on the name field:
131 |
132 | ```js
133 | collection.createIndex( { name : -1 }, function(err, result) {
134 | console.log(result);
135 | callback(result);
136 | }
137 | ```
138 |
139 |
142 |
143 | ## Q. What are the types of Indexes available in MongoDB?
144 |
145 | MongoDB supports the following types of the index for running a query.
146 |
147 | **1. Single Field Index:**
148 |
149 | MongoDB supports user-defined indexes like single field index. A single field index is used to create an index on the single field of a document. With single field index, MongoDB can traverse in ascending and descending order. By default, each collection has a single field index automatically created on the `_id` field, the primary key.
150 |
151 | **Example:**
152 |
153 | ```js
154 | {
155 | "_id": 1,
156 | "person": { name: "Alex", surname: "K" },
157 | "age": 29,
158 | "city": "New York"
159 | }
160 | ```
161 |
162 | We can define, a single field index on the age field.
163 |
164 | ```js
165 | db.people.createIndex( {age : 1} ) // creates an ascending index
166 |
167 | db.people.createIndex( {age : -1} ) // creates a descending index
168 | ```
169 |
170 | With this kind of index we can improve all the queries that find documents with a condition and the age field, like the following:
171 |
172 | ```js
173 | db.people.find( { age : 20 } )
174 | db.people.find( { name : "Alex", age : 30 } )
175 | db.people.find( { age : { $gt : 25} } )
176 | ```
177 |
178 | **2. Compound Index:**
179 |
180 | A compound index is an index on multiple fields. Using the same people collection we can create a compound index combining the city and age field.
181 |
182 | ```js
183 | db.people.createIndex( {city: 1, age: 1, person.surname: 1 } )
184 | ```
185 |
186 | In this case, we have created a compound index where the first entry is the value of the city field, the second is the value of the age field, and the third is the person.name. All the fields here are defined in ascending order.
187 |
188 | Queries such as the following can benefit from the index:
189 |
190 | ```js
191 | db.people.find( { city: "Miami", age: { $gt: 50 } } )
192 | db.people.find( { city: "Boston" } )
193 | db.people.find( { city: "Atlanta", age: {$lt: 25}, "person.surname": "Green" } )
194 | ```
195 |
196 | **3. Multikey Index:**
197 |
198 | This is the index type for arrays. When creating an index on an array, MongoDB will create an index entry for every element.
199 |
200 | **Example:**
201 |
202 | ```js
203 | {
204 | "_id": 1,
205 | "person": { name: "John", surname: "Brown" },
206 | "age": 34,
207 | "city": "New York",
208 | "hobbies": [ "music", "gardening", "skiing" ]
209 | }
210 | ```
211 |
212 | The multikey index can be created as:
213 |
214 | ```js
215 | db.people.createIndex( { hobbies: 1} )
216 | ```
217 |
218 | Queries such as these next examples will use the index:
219 |
220 | ```js
221 | db.people.find( { hobbies: "music" } )
222 | db.people.find( { hobbies: "music", hobbies: "gardening" } )
223 | ```
224 |
225 | **4. Geospatial Index:**
226 |
227 | GeoIndexes are a special index type that allows a search based on location, distance from a point and many other different features. To query geospatial data, MongoDB supports two types of indexes – `2d indexes` and `2d sphere indexes`. 2d indexes use planar geometry when returning results and 2dsphere indexes use spherical geometry to return results.
228 |
229 | **5. Text Index:**
230 |
231 | It is another type of index that is supported by MongoDB. Text index supports searching for string content in a collection. These index types do not store language-specific stop words (e.g. "the", "a", "or"). Text indexes restrict the words in a collection to only store root words.
232 |
233 | **Example:**
234 |
235 | Let\'s insert some sample documents.
236 |
237 | ```js
238 | var entries = db.people("blogs").entries;
239 | entries.insert( {
240 | title : "my blog post",
241 | text : "i am writing a blog. yay",
242 | site: "home",
243 | language: "english" });
244 | entries.insert( {
245 | title : "my 2nd post",
246 | text : "this is a new blog i am typing. yay",
247 | site: "work",
248 | language: "english" });
249 | entries.insert( {
250 | title : "knives are Fun",
251 | text : "this is a new blog i am writing. yay",
252 | site: "home",
253 | language: "english" });
254 | ```
255 |
256 | Let\'s define create the text index.
257 |
258 | ```js
259 | var entries = db.people("blogs").entries;
260 | entries.ensureIndex({title: "text", text: "text"}, { weights: {
261 | title: 10,
262 | text: 5
263 | },
264 | name: "TextIndex",
265 | default_language: "english",
266 | language_override: "language" });
267 | ```
268 |
269 | Queries such as these next examples will use the index:
270 |
271 | ```js
272 | var entries = db.people("blogs").entries;
273 | entries.find({$text: {$search: "blog"}, site: "home"})
274 | ```
275 |
276 | **6. Hashed Index:**
277 |
278 | MongoDB supports hash-based sharding and provides hashed indexes. These indexes are the hashes of the field value. Shards use hashed indexes and create a hash according to the field value to spread the writes across the sharded instances.
279 |
280 |
283 |
284 | ## Q. Explain Index Properties in MongoDB?
285 |
286 | **1. TTL Indexes:**
287 |
288 | TTL ( **Time To Live** ) is a special option that we can apply only to a single field index to permit the automatic deletion of documents after a certain time.
289 |
290 | During index creation, we can define an expiration time. After that time, all the documents that are older than the expiration time will be removed from the collection. This kind of feature is very useful when we are dealing with data that don\'t need to persist in the database ( eg. `session data` ).
291 |
292 | **Example:**
293 |
294 | ```js
295 | db.sessionlog.createIndex( { "lastUpdateTime": 1 }, { expireAfterSeconds: 1800 } )
296 | ```
297 |
298 | In this case, MongoDB will drop the documents from the collection automatically once half an hour (1800 seconds) has passed since the value in **lastUpdateTime** field.
299 |
300 | **Restrictions:**
301 |
302 | * Only single field indexes can have the TTL option
303 | * the `_id` single field index cannot support the TTL option
304 | * the indexed field must be a date type
305 | * a capped collection cannot have a TTL index
306 |
307 | **2. Partial indexes:**
308 |
309 | A partial index is an index that contains only a subset of the values based on a filter rule. They are useful in cases where:
310 |
311 | * The index size can be reduced
312 | * We want to index the most relevant and used values in the query conditions
313 | * We want to index the most selective values of a field
314 |
315 | **Example:**
316 |
317 | ```js
318 | db.people.createIndex(
319 | { "city": 1, "person.surname": 1 },
320 | { partialFilterExpression: { age : { $lt: 30 } } }
321 | )
322 | ```
323 |
324 | We have created a compound index on city and person.surname but only for the documents with age less than 30.
325 | In order for the partial index to be used the queries must contain a condition on the age field.
326 |
327 | ```js
328 | db.people.find( { city: "New Tork", age: { $eq: 20} } )
329 | ```
330 |
331 | **3. Sparse indexes:**
332 |
333 | Sparse indexes are a subset of partial indexes. A sparse index only contains elements for the documents that have the indexed field, even if it is null.
334 |
335 | Since MongoDB is a schemaless database, the documents in a collection can have different fields, so an indexed field may not be present in some of them.
336 |
337 | **Example:**
338 |
339 | To create such an index use the sparse option:
340 |
341 | ```js
342 | db.people.createIndex( { city: 1 }, { sparse: true } )
343 | ```
344 |
345 | In this case, we are assuming there could be documents in the collection with the field city missing. Sparse indexes are based on the existence of a field in the documents and are useful to reduce the size of the index.
346 |
347 | **4. Unique indexes:**
348 |
349 | MongoDB can create an index as unique. An index defined this way cannot contain duplicate entries.
350 |
351 | **Example:**
352 |
353 | ```js
354 | db.people.createIndex( { city: 1 }, { unique: true } )
355 | ```
356 |
357 | Uniqueness can be defined for compound indexes too.
358 |
359 | ```js
360 | db.people.createIndex( { city: 1, person.surname: 1}, { unique: true } )
361 | ```
362 |
363 | By default, the index on `_id` is automatically created as unique.
364 |
365 |
368 |
369 | ## Q. How many indexes does MongoDB create by default for a new collection?
370 |
371 | By default MongoDB creates a unique index on the `_id` field during the creation of a collection. The `_id` index prevents clients from inserting two documents with the same value for the `_id` field.
372 |
373 |
376 |
377 | ## Q. Can you create an index in an array field in MongoDB?
378 |
379 | Yes, To index a field that holds an array value, MongoDB creates an index key for each element in the array. Multikey indexes can be constructed over arrays that hold both scalar values (e.g. strings, numbers) and nested documents. MongoDB automatically creates a multikey index if any indexed field is an array.
380 |
381 | **Syntax:**
382 |
383 | ```js
384 | db.collection.createIndex( { : < 1 or -1 > } )
385 | ```
386 |
387 | For example, consider an inventory collection that contains the following documents:
388 |
389 | ```js
390 | { _id: 10, type: "food", item: "aaa", ratings: [ 5, 8, 9 ] }
391 | { _id: 11, type: "food", item: "bbb", ratings: [ 5, 9 ] }
392 | { _id: 12, type: "food", item: "ccc", ratings: [ 9, 5, 8, 4, 7 ] }
393 | ```
394 |
395 | The collection has a multikey index on the ratings field:
396 |
397 | ```js
398 | db.inventory.createIndex( { ratings: 1 } )
399 | ```
400 |
401 | The following query looks for documents where the ratings field is the array [ 5, 9 ]:
402 |
403 | ```js
404 | db.inventory.find( { ratings: [ 5, 9 ] } )
405 | ```
406 |
407 | MongoDB can use the multikey index to find documents that have 5 at any position in the ratings array. Then, MongoDB retrieves these documents and filters for documents whose ratings array equals the query array [ 5, 9 ].
408 |
409 |
502 |
503 | ## Q. What is "Namespace" in MongoDB?
504 |
505 | MongoDB stores BSON (Binary Interchange and Structure Object Notation) objects in the collection. The concatenation of the collection name and database name is called a namespace
506 |
507 |
510 |
511 | ## Q. What is Replication in Mongodb?
512 |
513 | Replication exists primarily to offer data redundancy and high availability. It maintain the durability of data by keeping multiple copies or replicas of that data on physically isolated servers. Replication allows to increase data availability by creating multiple copies of data across servers. This is especially useful if a server crashes or hardware failure.
514 |
515 | With MongoDB, replication is achieved through a **Replica Set**. Writer operations are sent to the primary server (node), which applies the operations across secondary servers, replicating the data. If the primary server fails (through a crash or system failure), one of the secondary servers takes over and becomes the new primary node via election. If that server comes back online, it becomes a secondary once it fully recovers, aiding the new primary node.
516 |
517 |
520 |
521 | ## Q. What is Replica Set in MongoDB?
522 |
523 | It is a group of mongo processes that maintain same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments. A replica set contains a primary node and multiple secondary nodes.
524 |
525 | The primary node receives all write operations. A replica set can have only one primary capable of confirming writes with `{ w: "majority" }` write concern; although in some circumstances, another mongod instance may transiently believe itself to also be primary.
526 |
527 | The secondaries replicate the primary\'s oplog and apply the operations to their data sets such that the secondaries\' data sets reflect the primary\'s data set. If the primary is unavailable, an eligible secondary will hold an election to elect itself the new primary.
528 |
529 |
536 |
537 | ## Q. How does MongoDB ensure high availability?
538 |
539 | **High Availability (HA)** refers to the improvement of system and app availability by minimizing the downtime caused by routine maintenance operations (planned) and sudden system crashes (unplanned).
540 |
541 | **Replica Set:**
542 |
543 | The replica set mechanism of MongoDB has two main purposes:
544 |
545 | * One is for data redundancy for failure recovery. When the hardware fails, or the node is down for other reasons, you can use a replica for recovery.
546 | * The other purpose is for read-write splitting. It routes the reading requests to the replica to reduce the reading pressure on the primary node.
547 |
548 | MongoDB automatically maintains replica sets, multiple copies of data that are distributed across servers, racks and data centers. Replica sets help prevent database downtime using native replication and automatic failover.
549 |
550 | A replica set consists of multiple replica set members. At any given time, one member acts as the primary member, and the other members act as secondary members. If the primary member fails for any reason (e.g., hardware failure), one of the secondary members is automatically elected to primary and begins to process all reads and writes.
551 |
552 |
555 |
556 | ## Q. What is an Embedded MongoDB Document?
557 |
558 | An embedded, or nested, MongoDB Document is a normal document that is nested inside another document within a MongoDB collection. Embedding connected data in a single document can reduce the number of read operations required to obtain data. In general, we should structure our schema so that application receives all of its required information in a single read operation.
559 |
560 | **Example:**
561 |
562 | In the normalized data model, the address documents contain a reference to the patron document.
563 |
564 | ```js
565 | // patron document
566 | {
567 | _id: "joe",
568 | name: "Joe Bookreader"
569 | }
570 |
571 | // address documents
572 | {
573 | patron_id: "joe", // reference to patron document
574 | street: "123 Fake Street",
575 | city: "Faketon",
576 | state: "MA",
577 | zip: "12345"
578 | }
579 |
580 | {
581 | patron_id: "joe",
582 | street: "1 Some Other Street",
583 | city: "Boston",
584 | state: "MA",
585 | zip: "12345"
586 | }
587 | ```
588 |
589 | Embedded documents are particularly useful when a **one-to-many** relationship exists between documents. In the example shown above, we see that a single customer has multiple addresses associated with him. The nested document structure makes it easy to retrieve complete address information about this customer with just a single query.
590 |
591 |
594 |
595 | ## Q. How can you achieve primary key - foreign key relationships in MongoDB?
596 |
597 | The primary key-foreign key relationship can be achieved by embedding one document inside the another. As an example, a department document can have its employee document(s).
598 |
599 |
602 |
603 | ## Q. When should we embed one document within another in MongoDB?
604 |
605 | You should consider embedding documents for:
606 |
607 | * *contains* relationships between entities
608 | * One-to-many relationships
609 | * Performance reasons
610 |
611 |
614 |
615 | ## Q. How is data stored in MongoDB?
616 |
617 | In MongoDB, Data is stored in BSON documents (short for `Binary JSON`). These documents are stored in MongoDB in JSON (JavaScript Object Notation) format. JSON documents support embedded fields, so related data and lists of data can be stored with the document instead of an external table. Documents contain one or more fields, and each field contains a value of a specific data type, including arrays, binary data and sub-documents. Documents that tend to share a similar structure are organized as collections.
618 |
619 | JSON is formatted as name/value pairs. In JSON documents, field names and values are separated by a colon, field name and value pairs are separated by commas, and sets of fields are encapsulated in "curly braces" ({}).
620 |
621 | **Example:**
622 |
623 | ```js
624 | {
625 | "name": "notebook",
626 | "qty": 50,
627 | "rating": [ { "score": 8 }, { "score": 9 } ],
628 | "size": { "height": 11, "width": 8.5, "unit": "in" },
629 | "status": "A",
630 | "tags": [ "college-ruled", "perforated"]
631 | }
632 | ```
633 |
634 |
637 |
638 | ## Q. What are the differences between MongoDB and SQL-SERVER?
639 |
640 | * The MongoDB store the data in documents with JSON format but SQL store the data in Table format.
641 | * The MongoDB provides high performance, high availability, easy scalability etc. rather than SQL Server.
642 | * In the MongoDB, we can change the structure simply by adding, removing column from the existing documents.
643 |
644 | **MongoDB and SQL Server Comparision Table:**
645 |
646 | |Base of Comparison | MS SQL Server | MongoDB |
647 | |--------------------|--------------------|---------------------|
648 | |Storage Model |RDBMS |Document-Oriented |
649 | |Joins |Yes |No |
650 | |Transaction |ACID |Multi-document ACID Transactions with snapshot isolation |
651 | |Agile practices |No |Yes |
652 | |Data Schema |Fixed |Dynamic |
653 | |Scalability |Vertical |Horizontal |
654 | |Map Reduce |No |Yes |
655 | |Language |SQL query language |JSON Query Language |
656 | |Secondary index |Yes |Yes |
657 | |Triggers |Yes |Yes |
658 | |Foreign Keys |Yes |No |
659 | |Concurrency |Yes |yes |
660 | |XML Support |Yes |No |
661 |
662 |
669 |
670 | ## Q. How can you achieve transaction and locking in MongoDB?
671 |
672 | In MongoDB (4.2), an operation on a single document is atomic. For situations that require atomicity of reads and writes to multiple documents (in a single or multiple collections), MongoDB supports multi-document transactions. With distributed transactions, transactions can be used across multiple operations, collections, databases, documents, and shards.
673 |
674 | MongoDB allows multiple clients to read and write the same data. In order to ensure consistency, it uses locking and other concurrency control measures to prevent multiple clients from modifying the same piece of data simultaneously.
675 |
676 | MongoDB uses **multi-granularity locking** that allows operations to lock at the global, database or collection level, and allows for individual storage engines to implement their own concurrency control below the collection level (e.g., at the document-level in WiredTiger). MongoDB uses reader-writer locks that allow concurrent readers shared access to a resource, such as a database or collection.
677 |
678 | The lock modes are represented as follows:
679 |
680 | |Lock Mode |Description |
681 | |------------|----------------------------|
682 | |R |Represents Shared (S) lock. |
683 | |W |Represents Exclusive (X) lock.|
684 | |r |Represents Intent Shared (IS) lock.|
685 | |w |Represents Intent Exclusive (IX) lock.|
686 |
687 | **Example:**
688 |
689 | The following example highlights the key components of the transactions API
690 |
691 | ```js
692 | const client = new MongoClient(uri);
693 | await client.connect();
694 |
695 | // Prereq: Create collections.
696 |
697 | await client.db('mydb1').collection('foo').insertOne({ abc: 0 }, { w: 'majority' });
698 |
699 | await client.db('mydb2').collection('bar').insertOne({ xyz: 0 }, { w: 'majority' });
700 |
701 | // Step 1: Start a Client Session
702 | const session = client.startSession();
703 |
704 | // Step 2: Optional. Define options to use for the transaction
705 | const transactionOptions = {
706 | readPreference: 'primary',
707 | readConcern: { level: 'local' },
708 | writeConcern: { w: 'majority' }
709 | };
710 |
711 | // Step 3: Use withTransaction to start a transaction, execute the callback, and commit (or abort on error)
712 | // Note: The callback for withTransaction MUST be async and/or return a Promise.
713 | try {
714 | await session.withTransaction(async () => {
715 | const coll1 = client.db('mydb1').collection('foo');
716 | const coll2 = client.db('mydb2').collection('bar');
717 |
718 | // Important:: You must pass the session to the operations
719 |
720 | await coll1.insertOne({ abc: 1 }, { session });
721 | await coll2.insertOne({ xyz: 999 }, { session });
722 | }, transactionOptions);
723 | } finally {
724 | await session.endSession();
725 | await client.close();
726 | }
727 | ```
728 |
729 |
732 |
733 | ## Q. When to Use MongoDB Rather than MySQL?
734 |
735 | **1. MongoDB:**
736 |
737 | MongoDB is one of the most popular document-oriented databases under the banner of NoSQL database. It employs the format of key-value pairs, here called document store. Document stores in MongoDB are created is stored in BSON files which are, in fact, a little-modified version of JSON files and hence all JS are supported.
738 |
739 | It offers greater efficiency and reliability which in turn can meet your storage capacity and speed demands. The schema-free implementation of MongoDB eliminates the prerequisites of defining a fixed structure. These models allow hierarchical relationships representation and facilitate the ability to change the structure of the record.
740 |
741 | **Pros:**
742 |
743 | * MongoDB has a lower latency per query & spends less CPU time per query because it is doing a lot less work (e.g. no joins, transactions). As a result, it can handle a higher load in terms of queries per second.
744 | * MongoDB is easier to shard (use in a cluster) because it doesn\'t have to worry about transactions and consistency.
745 | * MongoDB has a faster write speed because it does not have to worry about transactions or rollbacks (and thus does not have to worry about locking).
746 | * It supports many Features like automatic repair, easier data distribution, and simpler data models make administration and tuning requirements lesser in NoSQL.
747 | * NoSQL databases are cheap and open source.
748 | * NoSQL database support caching in system memory so it increases data output performance.
749 |
750 | **Cons:**
751 |
752 | * MongoDB does not support transactions.
753 | * In general, MongoDB creates more work (e.g. more CPU cost) for the client server. For example, to join data one has to issue multiple queries and do the join on the client.
754 | * No Stored Procedures in mongo dB (NoSQL database).
755 |
756 | **Reasons to Use a NoSQL Database:**
757 |
758 | * **Storing large volumes of data without structure**: A NoSQL database doesn\'t limit storable data types. Plus, you can add new types as business needs change.
759 | * **Using cloud computing and storage**: Cloud-based storage is a great solution, but it requires data to be easily spread across multiple servers for scaling. Using affordable hardware on-site for testing and then for production in the cloud is what NoSQL databases are designed for.
760 | * **Rapid development**: If you are developing using modern agile methodologies, a relational database will slow you down. A NoSQL database doesn\'t require the level of preparation typically needed for relational databases.
761 |
762 | **2. MySQL:**
763 |
764 | MySQL is a popular open-source relational database management system (RDBMS) that is developed, distributed and supported by Oracle Corporation. MySQL stores data in tables and uses structured query language (SQL) for database access. It uses Structured Query Language SQL to access and transfer the data and commands such as 'SELECT', 'UPDATE', 'INSERT' and 'DELETE' to manage it.
765 |
766 | Related information is stored in different tables but the concept of JOIN operations simplifies the process of correlating it and performing queries across multiple tables and minimize the chances of data duplication. It follows the ACID (Atomic, Consistent, Isolated and Durable) model. This means that once a transaction is complete, the data remains consistent and stable on the disc which may include distinct multiple memory locations.
767 |
768 | **Pros:**
769 |
770 | * SQL databases are table based databases.
771 | * Data store in rows and columns
772 | * Each row contains a unique instance of data for the categories defined by the columns.
773 | * Provide facility primary key, to uniquely identify the rows.
774 |
775 | **Cons:**
776 |
777 | * Users have to scale relational database on powerful servers that are expensive and difficult to handle. To scale relational database, it has to be distributed on to multiple servers. Handling tables across different servers is difficult.
778 | * In SQL server\'s data has to fit into tables anyhow. If your data doesn\'t fit into tables, then you need to design your database structure that will be complex and again difficult to handle.
779 |
780 |
783 |
784 | ## Q. How MongoDB supports ACID transactions and locking functionalities?
785 |
786 | ACID stands that any update is:
787 |
788 | * **Atomic:** it either fully completes or it does not
789 | * **Consistent:** no reader will see a "partially applied" update
790 | * **Isolated:** no reader will see a "dirty" read
791 | * **Durable:** (with the appropriate write concern)
792 |
793 | MongoDB, has always supported ACID transactions in a single document and, when leveraging the document model appropriately, many applications don\'t need ACID guarantees across multiple documents.
794 |
795 | MongoDB is a document based NoSQL database with a flexible schema. Transactions are not operations that should be executed for every write operation since they incur a greater performance cost over a single document writes. With a document based structure and denormalized data model, there will be a minimized need for transactions. Since MongoDB allows document embedding, you don\'t necessarily need to use a transaction to meet a write operation.
796 |
797 | MongoDB version 4.0 provides **multi-document transaction** support for replica set deployments only and probably the version 4.2 will extend support for sharded deployments.
798 |
799 | **Example:** Multi-Document ACID Transactions in MongoDB
800 |
801 | These are multi-statement operations that need to be executed sequentially without affecting each other. For example below we can create two transactions, one to add a user and another to update a user with a field of age.
802 |
803 | ```js
804 | $session.startTransaction()
805 |
806 | db.users.insert({_id: 6, name: "John"})
807 |
808 | db.users.updateOne({_id: 3, {$set: {age:26} }})
809 |
810 | session.commit_transaction()
811 | ```
812 |
813 | Transactions can be applied to operations against multiple documents contained in one or many collection/database. Any changes due to document transaction do not impact performance for workloads not related or do not require them. Until the transaction is committed, uncommitted writes are neither replicated to the secondary nodes nor are they readable outside the transactions.
814 |
815 |
818 |
819 | ## Q. What are the best practices for MongoDB Transactions?
820 |
821 | The multi-document transactions are only supported in the `WiredTiger` storage engine. For a single ACID transaction, if you try performing an excessive number of operations, it can result in high pressure on the WiredTiger cache. The cache is always dictated to maintain state for all subsequent writes since the oldest snapshot was created. This means new writes will accumulate in the cache throughout the duration of the transaction and will be flushed only after transactions currently running on old snapshots are committed or aborted.
822 |
823 | For the best database performance on the transaction, developers should consider:
824 |
825 | 1. Always modify a small number of documents in a transaction. Otherwise, you will need to break the transaction into different parts and process the documents in different batches. At most, process 1000 documents at a time.
826 |
827 | 2. Temporary exceptions such as awaiting to elect primary and transient network hiccups may result in abortion of the transaction. Developers should establish a logic to retry the transaction if the defined errors are presented.
828 |
829 | 3. Configure optimal duration for the execution of the transaction from the default 60 seconds provided by MongoDB. Besides, employ indexing so that it can allow fast data access within the transaction.
830 |
831 | 4. Decompose your transaction into a small set of operation so that it fits the 16MB size constraints. Otherwise, if the operation together with oplog description exceed this limit, the transaction will be aborted.
832 |
833 | 5. All data relating to an entity should be stored in a single, rich document structure. This is to reduce the number of documents that are to be cached when different fields are going to be changed.
834 |
835 |
838 |
839 | ## Q. Explain limitations of MongoDB Transactions?
840 |
841 | MongoDB transactions can exist only for relatively short time periods. By default, a transaction must span no more than one minute of clock time. This limitation results from the underlying MongoDB implementation. MongoDB uses MVCC, but unlike databases such as Oracle, the “older” versions of data are kept only in memory.
842 |
843 | 1. You cannot create or drop a collection inside a transaction.
844 | 2. Transactions cannot make writes to a capped collection
845 | 3. Transactions take plenty of time to execute and somehow they can slow the performance of the database.
846 | 4. Transaction size is limited to 16MB requiring one to split any that tends to exceed this size into smaller transactions.
847 | 5. Subjecting a large number of documents to a transaction may exert excessive pressure on the WiredTiger engine and since it relies on the snapshot capability, there will be a retention of large unflushed operations in memory. This renders some performance cost on the database.
848 |
849 |
852 |
853 | ## Q. Should I normalize my data before storing it in MongoDB?
854 |
855 | Data used by multiple documents can either be embedded (denormalized) or referenced (normalized). Normalization, which is increasing the complexity of the schema by splitting tables into multiple smaller ones to reduce the data redundancy( 1NF, 2NF, 3NF).
856 |
857 | But Mongo follows the exact opposite way of what we do with SQL. In MongoDB, data normalization is not requried. Indeed we need to de-normalize and fit it into a collection of multiple documents.
858 |
859 | **Example:** Let\'s say we have three tables
860 |
861 | * Table - 1 : ColumnA, ColumnB (primary key)
862 | * Table - 2 : ColumnC (Foreign key), ColumnD (primary key)
863 | * Table - 3 : ColumnE (foreign key), ColumnF
864 |
865 | In this case, mongoDB document structure should be as follows.
866 |
867 | ```js
868 | {
869 | ColumnA : ValueA,
870 | ColumnB : ValueB,
871 | Subset1 : [{
872 | ColumnC : ValueC,
873 | ColumnD : ValueD,
874 | Subset2 : [{
875 | ColumnE : ValueE,
876 | ColumnF : ValueF
877 | }]
878 | }]
879 | }
880 | ```
881 |
882 |
885 |
886 | ## Q. What is upsert operation in MongoDB?
887 |
888 | Upsert operation in MongoDB is utilized to save document into collection. If document matches query criteria then it will perform update operation otherwise it will insert a new document into collection.
889 |
890 | Upsert operation is useful while importing data from external source which will update existing documents if matched otherwise it will insert new documents into collection.
891 |
892 | **Example:** Upsert option set for update
893 |
894 | This operation first searches for the document if not present then inserts the new document into the database.
895 |
896 | ```js
897 |
898 | > db.car.update(
899 | ... { name: "Qualis" },
900 | ... {
901 | ... name: "Qualis",
902 | ... speed: 50
903 | ... },
904 | ... { upsert: true }
905 | ... )
906 | WriteResult({
907 | "nMatched" : 0,
908 | "nUpserted" : 1,
909 | "nModified" : 0,
910 | "_id" : ObjectId("548d3a955a5072e76925dc1c")
911 | })
912 | ```
913 |
914 | The car with the name Qualis is checked for existence and if not, a document with car name "Qualis" and speed 50 is inserted into the database. The nUpserted with value "1" indicates a new document is inserted.
915 |
916 |
919 |
920 | ## Q. Is there an "upsert" option in the mongodb insert command?
921 |
922 | The `db.collection.insert()` provides no upsert possibility. Instead, mongo insert inserts a new document into a collection. Upsert is only possible using `db.collection.update()` and `db.collection.save()`.
923 |
924 |
927 |
928 | ## Q. What is oplog?
929 |
930 | The OpLog (Operations Log) is a special capped collection that keeps a rolling record of all operations that modify the data stored in databases.
931 |
932 | MongoDB applies database operations on the primary and then records the operations on the primary\'s oplog. The secondary members then copy and apply these operations in an asynchronous process. All replica set members contain a copy of the oplog, in the **local.oplog.rs** collection, which allows them to maintain the current state of the database.
933 |
934 | Each operation in the oplog is idempotent. That is, oplog operations produce the same results whether applied once or multiple times to the target dataset.
935 |
936 | **Example:** Querying The OpLog
937 |
938 | ```js
939 | MongoDB shell version: 2.0.4
940 | connecting to: mongodb:27017/test
941 | PRIMARY> use local
942 | PRIMARY> db.oplog.rs.find()
943 | ```
944 |
945 |
948 |
949 | ## Q. Does MongoDB pushes the writes to disk immediately or lazily?
950 |
951 | MongoDB pushes the data to disk lazily. It updates the immediately written to the journal but writing the data from journal to disk happens lazily.
952 |
953 |
986 |
987 | ## Q. If you remove a document from database, does MongoDB remove it from disk?
988 |
989 | Yes. If you remove a document from database, MongoDB will remove it from disk too.
990 |
991 |
994 |
995 | ## Q. Explain the structure of ObjectID in MongoDB?
996 |
997 | The **ObjectId(``)** class is the default primary key for a MongoDB document and is usually found in the `_id` field in an inserted document. It returns a new ObjectId value. The 12-byte ObjectId value consists of:
998 |
999 | * a 4-byte timestamp value, representing the ObjectId\'s creation, measured in seconds since the Unix epoch
1000 | * a 5-byte random value
1001 | * a 3-byte incrementing counter, initialized to a random value
1002 |
1003 |
1004 |
1005 |
1006 |
1007 | While the BSON format itself is little-endian, the timestamp and counter values are big-endian, with the most significant bytes appearing first in the byte sequence.
1008 |
1009 | **Create ObjectId:**
1010 |
1011 | To create a new objectID manually within the MongoDB we can declare `objectId()` as a method.
1012 |
1013 | ```js
1014 | > newObjectId = ObjectId();
1015 |
1016 | // Output
1017 | ObjectId("5349b4ddd2781d08c09890f3")
1018 | ```
1019 |
1020 | **MongoDB provides three methods for ObjectId:**
1021 |
1022 | | Method |Description |
1023 | |-----------------------|------------------------|
1024 | |ObjectId.getTimestamp()|Returns the timestamp portion of the object as a Date.|
1025 | |ObjectId.toString() |Returns the JavaScript representation in the form of a string literal "ObjectId(...)".|
1026 | |ObjectId.valueOf() |Returns the representation of the object as a hexadecimal string.|
1027 |
1028 |
1031 |
1032 | ## Q. What is a covered query in MongoDB?
1033 |
1034 | The MongoDB covered query is one which uses an index and does not have to examine any documents. An index will cover a query if it satisfies the following conditions:
1035 |
1036 | * All fields in a query are part of an index.
1037 | * All fields returned in the results are of the same index.
1038 | * No fields in the query are equal to null
1039 |
1040 | Since all the fields present in the query are part of an index, MongoDB matches the query conditions and returns the result using the same index without actually looking inside the documents.
1041 |
1042 | **Example:**
1043 |
1044 | A collection inventory has the following index on the type and item fields:
1045 |
1046 | ```js
1047 | db.inventory.createIndex( { type: 1, item: 1 } )
1048 | ```
1049 |
1050 | This index will cover the following operation which queries on the type and item fields and returns only the item field:
1051 |
1052 | ```js
1053 | db.inventory.find(
1054 | { type: "food", item:/^c/ },
1055 | { item: 1, _id: 0 }
1056 | )
1057 | ```
1058 |
1059 |
1062 |
1063 | ## Q. Why MongoDB is not preferred over a 32-bit system?
1064 |
1065 | When running a 32-bit system build of MongoDB, the total storage size for the server, including data and indexes, is 2 gigabytes. The reason for this is that the MongoDB storage engine uses memory-mapped files for performance.
1066 |
1067 | If you are running a 64-bit build of MongoDB, there is virtually no limit to storage size.
1068 |
1069 |
1072 |
1073 | ## Q. Can one MongoDB operation lock more than one database?
1074 |
1075 | Yes. Operations like `db.copyDatabase()`, `db.repairDatabase()`, etc. can lock more than one databases involved.
1076 |
1077 |
1080 |
1081 | ## Q. What is Sharding in MongoDB?
1082 |
1083 | **Sharding** is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations.
1084 |
1085 | Database systems with large data sets or high throughput applications can challenge the capacity of a single server. For example, high query rates can exhaust the CPU capacity of the server. Working set sizes larger than the system\'s RAM stress the I/O capacity of disk drives. There are two methods for addressing system growth: vertical and horizontal scaling.
1086 |
1087 | **1. Vertical Scaling:**
1088 |
1089 | Vertical Scaling involves increasing the capacity of a single server, such as using a more powerful CPU, adding more RAM, or increasing the amount of storage space.
1090 |
1091 | **2. Horizontal Scaling:**
1092 |
1093 | Horizontal Scaling involves dividing the system dataset and load over multiple servers, adding additional servers to increase capacity as required. While the overall speed or capacity of a single machine may not be high, each machine handles a subset of the overall workload, potentially providing better efficiency than a single high-speed high-capacity server.
1094 |
1095 |
1096 |
1097 |
1098 |
1099 | MongoDB supports horizontal scaling through `sharding`. A MongoDB sharded cluster consists of the following components:
1100 |
1101 | * **Shards**: Each shard contains a subset of the sharded data. Each shard can be deployed as a replica set.
1102 | * **Mongos**: The mongos acts as a query router, providing an interface between client applications and the sharded cluster. Starting in MongoDB 4.4, mongos can support hedged reads to minimize latencies.
1103 | * **Config Servers**: Config servers store metadata and configuration settings for the cluster.
1104 |
1105 |
1108 |
1109 | ## Q. What is Aggregation in MongoDB?
1110 |
1111 | Aggregation in MongoDB is an operation used to process the data that returns the computed results. Aggregation basically groups the data from multiple documents and operates in many ways on those grouped data in order to return one combined result.
1112 |
1113 | Aggregate function groups the records in a collection, and can be used to provide total number(sum), average, minimum, maximum etc out of the group selected. In order to perform the aggregate function in MongoDB, aggregate () is the function to be used.
1114 |
1115 | **Syntax:**
1116 |
1117 | ```js
1118 | db.collection_name.aggregate(aggregate_operation)
1119 | ```
1120 |
1121 | MongoDB provides three ways to perform aggregation:
1122 |
1123 | * the aggregation pipeline,
1124 | * the map-reduce function,
1125 | * and single purpose aggregation methods and commands.
1126 |
1127 | MongoDB\'s aggregation framework is modeled on the concept of data processing pipelines. Documents enter a multi-stage pipeline that transforms the documents into an aggregated result.
1128 |
1129 | **Example:**
1130 |
1131 | ```js
1132 | db.orders.aggregate([
1133 | { $match: { status: "A" } },
1134 | { $group: { _id: "$cust_id", total: { $sum: "$amount" } } }
1135 | ]);
1136 | ```
1137 |
1138 | The `$match` stage filters the documents by the status field and passes to the next stage those documents that have status equal to "A". The `$group` stage groups the documents by the cust_id field to calculate the sum of the amount for each unique cust_id.
1139 |
1140 | **Expressions used by Aggregate function:**
1141 |
1142 | |Expression |Description |
1143 | |------------|----------------------------------------------------------------------|
1144 | |$sum |Summates the defined values from all the documents in a collection|
1145 | |$avg |Calculates the average values from all the documents in a collection|
1146 | |$min |Return the minimum of all values of documents in a collection|
1147 | |$max |Return the maximum of all values of documents in a collection|
1148 | |$addToSet |Inserts values to an array but no duplicates in the resulting document|
1149 | |$push |Inserts values to an array in the resulting document|
1150 | |$first |Returns the first document from the source document|
1151 | |$last |Returns the last document from the source document|
1152 |
1153 |
1156 |
1157 | ## Q. Why are MongoDB data files large in size?
1158 |
1159 | MongoDB preallocates data files to reserve space and avoid file system fragmentation when you setup the server.
1160 |
1161 |
1164 |
1165 | ## Q. How can you isolate your cursors from intervening with the write operations?
1166 |
1167 | As cursor is not isolated during its lifetime, thus intervening write operations on a document may result in cursor that returns a document more than once. The `snapshot()` method can be used on a cursor to isolate the operation for a very specific case. `snapshot()` traverses the index on the `_id` field and guarantees that the query will return each document no more than once.
1168 |
1169 | **Restriction:**
1170 |
1171 | * We cannot use `snapshot()` with sharded collections.
1172 | * We cannot use `snapshot()` with `sort()` or `hint()` cursor methods.
1173 |
1174 |
1177 |
1178 | ## Q. At what interval does MongoDB write updates to the disk?
1179 |
1180 | By default configuration, MongoDB writes updates to the disk every 60 seconds. However, this is configurable with the `commitIntervalMs` and `syncPeriodSecs` options.
1181 |
1182 |
1185 |
1186 | ## Q. What happens if an index does not fit into RAM?
1187 |
1188 | If the indexes does not fit into RAM, MongoDB reads data from disk which is relatively very much slower than reading from RAM.
1189 |
1190 | Indexes do not have to fit entirely into RAM in all cases. If the value of the indexed field increments with every insert, and most queries select recently added documents; then MongoDB only needs to keep the parts of the index that hold the most recent or "right-most" values in RAM. This allows for efficient index use for read and write operations and minimize the amount of RAM required to support the index.
1191 |
1192 | **Example:** To check the size of indexes
1193 |
1194 | ```js
1195 | > db.collection.totalIndexSize()
1196 |
1197 | // Output (in bytes)
1198 | 4294976499
1199 | ```
1200 |
1201 |
1255 |
1256 | ## Q. How does Journaling work in MongoDB?
1257 |
1258 | Mongod primarily hosts the write operations in memory in shared view. It is called shared because it has memory mapping in actual disc. In this process, a write operation occurs in mongod, which then creates changes in private view. The first block is memory and the second block is "my disc". After a specified interval, which is called a "journal commit interval", the private view writes those operations in journal directory (residing in the disc).
1259 |
1260 | Once the journal commit happens, mongod pushes data into shared view. As part of the process, it gets written to actual data directory from the shared view (as this process happens in background). The basic advantage is, we have a reduced cycle from 60 seconds to 200 milliseconds.
1261 |
1262 | In a scenario where an abruption occurs at any point of time or flash disc remains unavailable for last 59 seconds , then when the next time mongod starts, it basically replays all write operation logs and writes into the actual data directory.
1263 |
1264 |
1271 |
1272 | ## Q. Is MongoDB schema-less?
1273 |
1274 | As a NoSQL database, MongoDB is considered schemaless because it does not require a rigid, pre-defined schema like a relational database. The database management system (DBMS) enforces a partial schema as data is written, explicitly listing collections and indexes.
1275 |
1276 | MongoDB is a document based database, which does not use the concept of tables and columns, instead of which it uses the concept of documents and collections. All the referential data with respect to different modules will be stored as one collection. More over the BSON data structure used by MongoDB can easily have varying sets of data and fields with different types.
1277 |
1278 | When we say **schemaless**, we actually mean **dynamically typed schema**, as opposed to statically typed schemas as available in RDBMS(SQL) databases. JSON is a completely schema free data structure, as opposed to XML which allows you to specify XSD if you need.
1279 |
1280 |
1283 |
1284 | ## Q. What is a Storage Engine in MongoDB?
1285 |
1286 | The storage engine is the component of the database that is responsible for managing how data is stored, both in memory and on disk. MongoDB supports multiple storage engines, as different engines perform better for specific workloads.
1287 |
1288 | **Example:** command to find storage engine
1289 |
1290 | ```js
1291 | > db.serverStatus().storageEngine
1292 |
1293 | // Output
1294 | {
1295 | "name" : "wiredTiger",
1296 | "supportsCommittedReads" : true,
1297 | "oldestRequiredTimestampForCrashRecovery" : Timestamp(0, 0),
1298 | "supportsPendingDrops" : true,
1299 | "dropPendingIdents" : NumberLong(0),
1300 | "supportsTwoPhaseIndexBuild" : true,
1301 | "supportsSnapshotReadConcern" : true,
1302 | "readOnly" : false,
1303 | "persistent" : true,
1304 | "backupCursorOpen" : false
1305 | }
1306 | ```
1307 |
1308 |
1309 |
1310 |
1311 |
1312 | MongoDB supports mainly 3 storage engines whose performance differ in accordance to some specific workloads. The storage engines are:
1313 |
1314 | * WiredTiger Storage Engine
1315 | * In-Memory Storage Engine
1316 | * ~~MMAPv1 Storage Engine~~ (This Storage Engine has been deprecated after MongoDB 4.2)
1317 |
1318 | **1. WiredTiger Storage Engine**
1319 |
1320 | `WiredTiger` is the default storage engine starting in MongoDB 3.2. It is well-suited for most workloads and is recommended for new deployments. WiredTiger provides a document-level concurrency model, checkpointing, and compression, among other features. The WiredTiger storage engine has both configurations of a `B-Tree` Based Engine and a `Log Structured Merge Tree` Based Engine.
1321 |
1322 | **2. In-Memory Storage Engine**
1323 |
1324 | In-Memory Storage Engine is available in MongoDB Enterprise. Rather than storing documents on-disk, it retains them in-memory for more predictable data latencies.
1325 |
1326 | **3. MMAPv1 Storage Engine**
1327 |
1328 | MMAPv1 is a B-tree based system which powers many of the functions such as storage interaction and memory management to the operating system. Its name comes from the fact that it uses memory mapped files to access data. It does so by directly loading and modifying file contents, which are in a virtual memory through a `mmap()` `syscall` methodology.
1329 |
1330 |
1337 |
1338 | ## Q. How to condense large volumes of data in Mongodb?
1339 |
1340 | **compact:**
1341 |
1342 | Rewrites and defragments all data and indexes in a collection. On `WiredTiger` databases, this command will release unneeded disk space to the operating system. This command will perform a compaction "in-line".
1343 |
1344 | MongoDB compresses the files by:
1345 |
1346 | * copying the files to a new location
1347 | * looping through the documents and re-ordering / re-solving them
1348 | * replacing the original files with the new files
1349 |
1350 | **Syntax:**
1351 |
1352 | ```js
1353 | { compact: }
1354 | ```
1355 |
1356 |
1359 |
1360 | ## Q. Explain relationships in MongoDB?
1361 |
1362 | Relationships in MongoDB are used to specify how one or more documents are related to each other. In MongoDB, the relationships can be modelled either by Embedded way or by using the Reference approach. These relationships can be of the following forms:
1363 |
1364 | * One to One
1365 | * One to Many
1366 | * Many to Many
1367 |
1368 | **Example:** Let us consider the case of storing addresses for users. So, one user can have multiple addresses making this a `1:N` relationship.
1369 |
1370 | User Collection
1371 |
1372 | ```js
1373 | {
1374 | "_id":ObjectId("52ffc33cd85242f436000001"),
1375 | "name": "Alex K",
1376 | "contact": "987654321",
1377 | "dob": "01-01-1990"
1378 | }
1379 | ```
1380 |
1381 | Address Collection
1382 |
1383 | ```js
1384 | {
1385 | "_id":ObjectId("52ffc4a5d85242602e000000"),
1386 | "building": "22 A, Indiana Apt",
1387 | "pincode": 123456,
1388 | "city": "Los Angeles",
1389 | "state": "California"
1390 | }
1391 | ```
1392 |
1393 | **1. Modeling Embedded Relationships**
1394 |
1395 | In the embedded approach, we will embed the address document inside the user document.
1396 |
1397 | ```js
1398 | > db.users.insert({
1399 | {
1400 | "_id":ObjectId("52ffc33cd85242f436000001"),
1401 | "contact": "987654321",
1402 | "dob": "01-01-1991",
1403 | "name": "Alex K",
1404 | "address": [
1405 | {
1406 | "building": "22 A, Indiana Apt",
1407 | "pincode": 123456,
1408 | "city": "Los Angeles",
1409 | "state": "California"
1410 | },
1411 | {
1412 | "building": "170 A, Acropolis Apt",
1413 | "pincode": 456789,
1414 | "city": "Chicago",
1415 | "state": "Illinois"
1416 | }
1417 | ]
1418 | }
1419 | })
1420 | ```
1421 |
1422 | This approach maintains all the related data in a single document, which makes it easy to retrieve and maintain. The whole document can be retrieved in a single query such as −
1423 |
1424 | ```js
1425 | >db.users.findOne({"name":"Alex K"},{"address":1})
1426 | ```
1427 |
1428 | The drawback is that if the embedded document keeps on growing too much in size, it can impact the read/write performance.
1429 |
1430 | **2. Modeling Referenced Relationships:**
1431 |
1432 | This is the approach of designing normalized relationship. In this approach, both the user and address documents will be maintained separately but the user document will contain a field that will reference the address document\'s id field.
1433 |
1434 | ```js
1435 | {
1436 | "_id":ObjectId("52ffc33cd85242f436000001"),
1437 | "contact": "987654321",
1438 | "dob": "01-01-1991",
1439 | "name": "Alex K",
1440 | "address_ids": [
1441 | ObjectId("52ffc4a5d85242602e000000"),
1442 | ObjectId("52ffc4a5d85242602e000001")
1443 | ]
1444 | }
1445 | ```
1446 |
1447 | With this approach, we will need two queries: first to fetch the `address_ids` fields from user document and second to fetch these addresses from address collection.
1448 |
1449 | ```js
1450 | >var result = db.users.findOne({"name":"Alex K"},{"address_ids":1})
1451 | >var addresses = db.address.find({"_id":{"$in":result["address_ids"]}})
1452 | ```
1453 |
1454 |
1457 |
1458 | ## Q. What is use of capped collection in MongoDB?
1459 |
1460 | **Capped collections** are fixed-size collections that support high-throughput operations that insert and retrieve documents based on insertion order. Capped collections work in a way similar to `circular buffers`: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.
1461 |
1462 | Capped collections restrict updates to the documents if the update results in increased document size. Since capped collections store documents in the order of the disk storage, it ensures that the document size does not increase the size allocated on the disk. Capped collections are best for storing log information, cache data, or any other high volume data.
1463 |
1464 | **Example:**
1465 |
1466 | ```js
1467 | >db.createCollection( "log", { capped: true, size: 100000 } )
1468 |
1469 |
1470 | // specify a maximum number of documents for the collection
1471 | >db.createCollection("log", { capped: true, size: 5242880, max: 5000 } )
1472 |
1473 |
1474 | // check whether a collection is capped or not
1475 | >db.cappedLogCollection.isCapped()
1476 |
1477 |
1478 | // convert existing collection to capped
1479 | >db.runCommand({"convertToCapped": "posts", size: 10000})
1480 |
1481 |
1482 | // Querying Capped Collection
1483 | >db.cappedLogCollection.find().sort({$natural: -1})
1484 | ```
1485 |
1486 |
1489 |
1490 | ## Q. What is splitting in MongoDB?
1491 |
1492 | Splitting is a process that keeps chunks from growing too large. When a chunk grows beyond a specified chunk size, or if the number of documents in the chunk exceeds Maximum Number of Documents Per Chunk to Migrate, MongoDB splits the chunk based on the shard key values the chunk represent.
1493 |
1494 |
1495 |
1496 |
1497 |
1498 | ## Q. Explain what is horizontal scalability in mongodb?
1499 |
1500 | Horizontal Scaling involves dividing the system dataset and load over multiple servers, adding additional servers to increase capacity as required. While the overall speed or capacity of a single machine may not be high, each machine handles a subset of the overall workload, potentially providing better efficiency than a single high-speed high-capacity server. Expanding the capacity of the deployment only requires adding additional servers as needed, which can be a lower overall cost than high-end hardware for a single machine. The trade off is increased complexity in infrastructure and maintenance for the deployment.
1501 |
1502 |
23 |
24 | ## Q. ***How to combine data from multiple collections into one collection?***
25 |
26 | **$lookup:**
27 |
28 | Performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing. To each input document, the `$lookup` stage adds a new array field whose elements are the matching documents from the “joined” collection. The `$lookup` stage passes these reshaped documents to the next stage.
29 |
30 | **Syntax:**
31 |
32 | ```js
33 | {
34 | $lookup:
35 | {
36 | from: ,
37 | localField: ,
38 | foreignField: ,
39 | as:
35 | 
45 | 
57 | 
71 | 
114 | 
531 | 
664 | 
1005 | 
1097 | 
1266 | 
1310 | 
1332 | 
1496 | 
1504 |