├── .gitignore ├── 01-basic-operations ├── README.md ├── node │ ├── createTable.js │ ├── deleteTable.js │ ├── getUser.js │ ├── insertUsers.js │ └── insertUsersConditional.js ├── python │ ├── create_table.py │ ├── delete_table.py │ ├── get_user.py │ ├── insert_users.py │ └── insert_users_conditional.py └── users.json ├── 02-query ├── README.md ├── items.json ├── node │ ├── createTable.js │ ├── deleteTable.js │ ├── getRoles.js │ ├── insertItems.js │ └── queryRoles.js └── python │ ├── create_table.py │ ├── delete_table.py │ ├── get_roles.py │ ├── insert_items.py │ └── query_roles.py ├── 03-secondary-indexes ├── README.md ├── items.json ├── node │ ├── createTable.js │ ├── deleteTable.js │ ├── insertItems.js │ └── queryRoles.js └── python │ ├── create_table.py │ ├── delete_table.py │ ├── insert_items.py │ └── query_roles.py ├── README.md ├── package-lock.json └── package.json /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules/ 2 | .vscode/ 3 | -------------------------------------------------------------------------------- /01-basic-operations/README.md: -------------------------------------------------------------------------------- 1 | # DynamoDB -- Basic Operations 2 | 3 | In this first lesson, you will learn the basics of writing to and reading from DynamoDB. This lesson has five steps: 4 | 5 | - [Creating a DynamoDB table](#creating-a-dynamodb-table); 6 | - [Loading data using the PutItem operation](#loading-data-using-the-putitem-operation); 7 | - [Reading data using the GetItem operation](#reading-data-using-the-getitem-operation); 8 | - [Preventing overwrites with Condition Expressions](#preventing-overwrites-with-condition-expressions); 9 | - [Deleting your DynamoDB table](#deleting-your-dynamodb-table); 10 | 11 | ## Creating a DynamoDB table 12 | 13 | First, we'll need to create a DynamoDB table. When creating a DynamoDB table, you need to specify the following properties at a minimum: 14 | 15 | 1. **Table name:** Give your table a name that will be unique within an AWS region. 16 | 17 | 2. **Primary key:** 18 | 19 | DynamoDB is a schemaless database. Unlike relational databases, this means you don't need to specify the names and types of all attribute that your items will have. Instead, you will manage your schema within your application code. 20 | 21 | However, you do need to define a _primary key_ for your table. Every item in your table must have the primary key for the table, and each item in your table is uniquely identifiable by the primary key. 22 | 23 | There are two types of primary keys: 24 | 25 | - **Simple:** A primary key that consists of a single element (the _partition_ key). 26 | 27 | - **Composite:** A primary key that consists of two elements (a _partition_ key and a _sort_ key). 28 | 29 | To define your table, you must provide two properties: 30 | 31 | - **KeySchema:** This defines the elements of your primary key. It must include a partition (also called a "HASH" key) and may include a sort key (also called a "RANGE" key). 32 | 33 | - **AttributeDefinitions:** For each element in your primary key, you must declare the attribute name and type in the `AttributeDefinitions` property. 34 | 35 | 3. **Throughput settings:** 36 | 37 | With traditional databases, you often spin up servers. You might specify CPU, RAM, and networking settings for your instance. You need to estimate your traffic and make guesses as to how that translates to computing resources. 38 | 39 | With DynamoDB, it's different. You pay for throughput directly rather than computing resources. This is split into Read Capacity Units (RCUs), which refers to a strongly-consistent read of 4KB of data, and Write Capacity Units (WCUs), which refers to a write of 1KB of data. 40 | 41 | There are two throughput modes you can use with DynamoDB: 42 | 43 | - **Provisioned throughput:** You specify the number of RCUs and WCUs you want available on a per-second basis; 44 | 45 | - **On demand:** You are charged on a per-request basis for each read and write you make. You don't need to specify the amount you want ahead of time. 46 | 47 | On a fully-utilized basis, on-demand billing is more expensive that provisioned throughput. However, it's difficult to get full utilization or anything close to it, particularly if your traffic patterns vary over the time of day or day of week. Many people actually save money with on demand, while also reducing the amount of capacity planning and adjustments you need to do. 48 | 49 | ### Task 50 | 51 | For this part of the workshop, you want to create a DynamoDB table with a simple primary key. This table will store users in your application. Name your table "Users" and give it a primary key with the hash key of "Username". 52 | 53 | For the throughput settings, choose to use provisioned throughput with `ReadCapacityUnits` and `WriteCapacityUnits` of `5`. This will fit within the AWS Free Tier and will be more than enough for this demo. 54 | 55 | Try it on your own first, but if you want to see an example of this, look at the following files: 56 | 57 | - [Node.js](./node/createTable.js). 58 | - [Python](./python/create_table.py). 59 | 60 | ## Loading data using the PutItem operation 61 | 62 | Now that you have your DynamoDB table, let's write some data into it. To write data into DynamoDB, you can use the [PutItem](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html) operation. 63 | 64 | As you insert items into DynamoDB, keep two things in mind: 65 | 66 | 1. **Primary key requirement:** 67 | 68 | Remember that every item in your DynamoDB table must include your primary key. When you are creating your item in the `PutItem` operation, make sure you have all elements of the primary key included. 69 | 70 | 2. **DynamoDB object format:** 71 | 72 | Each attribute on a DynamoDB item has a type. This can be simple types, such as strings or numbers, or they can be complex types like arrays, maps, and sets. 73 | 74 | When writing an item to DynamoDB, you need to include the type for each attribute. 75 | 76 | For example, when writing an attribute of "Username" with a type of `string`, your attribute might look as follows: 77 | 78 | ```js 79 | "Username": { "S": "alexdebrie" }, 80 | ``` 81 | 82 | We've used the `"S"` to indicate it's of type `string`. 83 | 84 | If you make a complex object, you need to note the type of the complex attribute as well as the elements within the attribute. 85 | 86 | In the [users.json](./users.json) file in this directory, you can see examples of items. Look at the `Interests` property to see an example of a `list` attribute and the `Address` property to see a `map` attribute. 87 | 88 | ### Task 89 | 90 | There is a file named `users.json` that includes three Users in DynamoDB's object format. Write a script that reads the three Users from the file and inserts them into the table using the `PutItem` operation. Because they are already in the DynamoDB object format, you don't need to add the attribute type information yourself. 91 | 92 | If you get stuck, look at the following files for examples: 93 | 94 | - [Node.js](./node/insertUsers.js). 95 | - [Python](./python/insert_users.py). 96 | 97 | ## Reading data using the GetItem operation 98 | 99 | By this point, you have data in your DynamoDB table. But data isn't put in your database to sit there -- you want to read it back out to use it! Let's do that here. 100 | 101 | To do so, we'll use the [GetItem](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html) operation in DynamoDB. The `GetItem` operation reads an individual item from DynamoDB. 102 | 103 | ### Task 104 | 105 | Write a script that reads one of your three items from your DynamoDB table. To do so, you must provide the primary key of your item. Recall that the primary key uniquely identifies each item in your table. Further, you must specify the type of your primary key attribute(s). 106 | 107 | The usernames of the three items are `alexdebrie`, `the_serena`, and `bigtimebrad`. 108 | 109 | If you're having trouble, look at the following files for an example: 110 | 111 | - [Node.js](./node/getUser.js). 112 | - [Python](./python/get_user.py). 113 | 114 | ## Preventing overwrites with Condition Expressions 115 | 116 | We've done the basics of both writing to and reading from DynamoDB. We're going to cover one last point before moving on. 117 | 118 | Previously, we used the `PutItem` operation to write to DynamoDB. This will write the item to DynamoDB and _completely overwrite_ any existing item that had the same primary key. 119 | 120 | At times, this may not be desirable. You may want to prevent overwriting an item if it already exists. To do so, you can use DynamoDB Condition Expressions. 121 | 122 | DynamoDB Condition Expressions allow you to specify conditions on write-based operations. These conditions must evaluate to True or the write will be aborted. 123 | 124 | The nuances of Condition Expressions are deep, but we won't go too far in this workshop. See this post on [using DynamoDB Condition Expressions](https://www.alexdebrie.com/posts/dynamodb-condition-expressions/) for more information. 125 | 126 | ### Task 127 | 128 | Update your script from the Loading Data step to add a `ConditionExpression` on your `PutItem` request. The Condition Expression should assert that there is not an existing item with the same primary key. 129 | 130 | Hint: you should use the [`attribute_not_exists()`](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html#Expressions.OperatorsAndFunctions.Functions) function. 131 | 132 | Check the following files for an example of using the `ConditionExpression`. 133 | 134 | - [Node.js](./node/insertUsersConditional.js). 135 | - [Python](./python/insert_users_conditional.py). 136 | 137 | ## Deleting your DynamoDB Table 138 | 139 | We've completed the main steps for this lesson, so let's remove your DynamoDB table. You can use the `DeleteTable` operation to delete your table and avoid incurring charges in your AWS account. 140 | 141 | ### Task 142 | 143 | Write a script to delete your `Users` table. See the following files for an example: 144 | 145 | - [Node.js](./node/deleteTable.js). 146 | - [Python](./python/delete_table.py). 147 | 148 | ## Conclusion 149 | 150 | That completes this first lesson! You learned how to use DynamoDB in a key-value store with a simple primary key. You loaded data with the `PutItem` operation and read it back with the `GetItem` operation. Then you saw how to prevent accidental overwrites of your data using Condition Expressions. 151 | 152 | In the [next lesson](../02-query/README.md), you will learn how to use a table with a composite primary key to handle more complex access patterns. 153 | -------------------------------------------------------------------------------- /01-basic-operations/node/createTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.createTable( 5 | { 6 | TableName: "Users", 7 | AttributeDefinitions: [ 8 | { 9 | AttributeName: "Username", 10 | AttributeType: "S", 11 | }, 12 | ], 13 | KeySchema: [{ AttributeName: "Username", KeyType: "HASH" }], 14 | ProvisionedThroughput: { 15 | ReadCapacityUnits: 5, 16 | WriteCapacityUnits: 5, 17 | }, 18 | }, 19 | function (err) { 20 | if (err) console.log(`Error creating table: ${err}`); 21 | else console.log("Table created succesfully! 🚀"); 22 | } 23 | ); 24 | -------------------------------------------------------------------------------- /01-basic-operations/node/deleteTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.deleteTable( 5 | { 6 | TableName: "Users", 7 | }, 8 | function (err) { 9 | if (err) console.log(`Error deleting table: ${err}`); 10 | else console.log("Table deleted successfully! 🙌"); 11 | } 12 | ); 13 | -------------------------------------------------------------------------------- /01-basic-operations/node/getUser.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.getItem( 5 | { 6 | TableName: "Users", 7 | Key: { Username: { S: "alexdebrie" } }, 8 | }, 9 | function (err, data) { 10 | if (err) console.log(`Error fetching user": ${err}`); 11 | else if (!data.Item) console.log("User does not exist"); 12 | else console.log(`Found user: ${JSON.stringify(data.Item, null, 2)}`); 13 | } 14 | ); 15 | -------------------------------------------------------------------------------- /01-basic-operations/node/insertUsers.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | const fs = require("fs"); 5 | const path = require("path"); 6 | 7 | const data = fs.readFileSync(path.join(__dirname, "../users.json")); 8 | const users = JSON.parse(data); 9 | 10 | users.forEach((user) => { 11 | DynamoDB.putItem( 12 | { 13 | TableName: "Users", 14 | Item: user, 15 | }, 16 | function (err) { 17 | if (err) console.log(`Error creating user ${user.Username.S}: ${err}`); 18 | else console.log(`User ${user.Username.S} created successfully!`); 19 | } 20 | ); 21 | }); 22 | -------------------------------------------------------------------------------- /01-basic-operations/node/insertUsersConditional.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | const fs = require("fs"); 5 | const path = require("path"); 6 | 7 | const data = fs.readFileSync(path.join(__dirname, "../users.json")); 8 | const users = JSON.parse(data); 9 | 10 | users.forEach((user) => { 11 | DynamoDB.putItem( 12 | { 13 | TableName: "Users", 14 | Item: user, 15 | ConditionExpression: "attribute_not_exists(Username)", 16 | }, 17 | function (err) { 18 | if (err) console.log(`Error creating user ${user.Username.S}: ${err}`); 19 | else console.log(`User ${user.Username.S} created successfully!`); 20 | } 21 | ); 22 | }); 23 | -------------------------------------------------------------------------------- /01-basic-operations/python/create_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.create_table( 7 | TableName="Users", 8 | AttributeDefinitions=[{"AttributeName": "Username", "AttributeType": "S"}], 9 | KeySchema=[{"AttributeName": "Username", "KeyType": "HASH"}], 10 | ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, 11 | ) 12 | print("Table created successfully! 🚀") 13 | except Exception as err: 14 | print(f"Error creating table: {err}") 15 | -------------------------------------------------------------------------------- /01-basic-operations/python/delete_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.delete_table( 7 | TableName="Users", 8 | ) 9 | print("Table deleted successfully! 🙌") 10 | except Exception as err: 11 | print(f"Error deleting table: {err}") 12 | -------------------------------------------------------------------------------- /01-basic-operations/python/get_user.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import boto3 4 | 5 | client = boto3.client("dynamodb", region_name="us-east-1") 6 | 7 | try: 8 | response = client.get_item(TableName="Users", Key={"Username": {"S": "alexdebrie"}}) 9 | if response["Item"]: 10 | print(f"Found user: {json.dumps(response['Item'], indent=2)}") 11 | else: 12 | print("User does not exist!") 13 | except Exception as err: 14 | print(f"Error fetching user: {err}") 15 | -------------------------------------------------------------------------------- /01-basic-operations/python/insert_users.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | import boto3 5 | 6 | client = boto3.client("dynamodb", region_name="us-east-1") 7 | 8 | lesson_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 9 | users_path = os.path.join(lesson_directory, "users.json") 10 | 11 | with open(users_path, "r") as f: 12 | users = json.load(f) 13 | 14 | for user in users: 15 | try: 16 | client.put_item(TableName="Users", Item=user) 17 | print(f"User {user['Username']['S']} created successfully!") 18 | except Exception as err: 19 | print(f"Error creating user {user['Username']['S']}: {err}") 20 | -------------------------------------------------------------------------------- /01-basic-operations/python/insert_users_conditional.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | import boto3 5 | 6 | client = boto3.client("dynamodb", region_name="us-east-1") 7 | 8 | lesson_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 9 | users_path = os.path.join(lesson_directory, "users.json") 10 | 11 | with open(users_path, "r") as f: 12 | users = json.load(f) 13 | 14 | for user in users: 15 | try: 16 | client.put_item( 17 | TableName="Users", 18 | Item=user, 19 | ConditionExpression="attribute_not_exists(Username)", 20 | ) 21 | print(f"User {user['Username']['S']} created successfully!") 22 | except Exception as err: 23 | print(f"Error creating user {user['Username']['S']}: {err}") 24 | -------------------------------------------------------------------------------- /01-basic-operations/users.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "Username": { 4 | "S": "alexdebrie" 5 | }, 6 | "Name": { 7 | "S": "Alex DeBrie" 8 | }, 9 | "Interests": { 10 | "L": [ 11 | { 12 | "S": "AWS" 13 | }, 14 | { 15 | "S": "Basketball" 16 | }, 17 | { 18 | "S": "Board games" 19 | } 20 | ] 21 | }, 22 | "Address": { 23 | "M": { 24 | "Street": { 25 | "S": "1122 1st Avenue" 26 | }, 27 | "City": { 28 | "S": "Omaha" 29 | }, 30 | "State": { 31 | "S": "Nebraska" 32 | } 33 | } 34 | } 35 | }, 36 | { 37 | "Username": { 38 | "S": "bigtimebrad" 39 | }, 40 | "Name": { 41 | "S": "Brad Pitt" 42 | }, 43 | "Interests": { 44 | "L": [ 45 | { 46 | "S": "Movies" 47 | }, 48 | { 49 | "S": "Kids" 50 | } 51 | ] 52 | }, 53 | "Address": { 54 | "M": { 55 | "Street": { 56 | "S": "555 Broadway" 57 | }, 58 | "City": { 59 | "S": "Beverly Hills" 60 | }, 61 | "State": { 62 | "S": "California" 63 | } 64 | } 65 | } 66 | }, 67 | { 68 | "Username": { 69 | "S": "the_serena" 70 | }, 71 | "Name": { 72 | "S": "Serena Williams" 73 | }, 74 | "Interests": { 75 | "L": [ 76 | { 77 | "S": "Tennis" 78 | }, 79 | { 80 | "S": "French" 81 | } 82 | ] 83 | }, 84 | "Address": { 85 | "M": { 86 | "Street": { 87 | "S": "987 Avenue A" 88 | }, 89 | "City": { 90 | "S": "Palm Beach" 91 | }, 92 | "State": { 93 | "S": "Florida" 94 | } 95 | } 96 | } 97 | } 98 | ] 99 | -------------------------------------------------------------------------------- /02-query/README.md: -------------------------------------------------------------------------------- 1 | # DynamoDB -- Composite Primary key + the Query operation 2 | 3 | In the previous lesson, you learned the basics of DynamoDB, including terminology and the basic read and write operations. 4 | 5 | In this second lesson, we'll learn some more advanced access patterns. We'll see how to use a table with a composite primary key. Then, we'll see a few different ways to work with multiple items in a single request. In particular, we'll see the power of the Query operation, which is foundational for advanced DynamoDB work. 6 | 7 | This lesson has five steps: 8 | 9 | - [Creating a DynamoDB table with a composite primary key](#creating-a-dynamodb-table-with-a-composite-primary-key); 10 | - [Loading data into your table](#loading-data-into-your-table) 11 | - [Reading multiple items using the BatchGetItem operation](#reading-multiple-items-using-the-batchgetitem-operation); 12 | - [Reading multiple items with the Query operation](#reading-multiple-items-with-the-query-operation); 13 | - [Deleting your DynamoDB table](#deleting-your-dynamodb-table); 14 | 15 | ## Creating a DynamoDB table with a composite primary key 16 | 17 | Like in the last lesson, we'll start with creating a DynamoDB table. However, this lesson uses a composite primary key, not a simple primary key. 18 | 19 | Much of the table creation should be the same, with two differences: 20 | 21 | 1. Your `KeySchema` property will include both a `HASH` key and a `RANGE` key; 22 | 23 | 2. The `AttributeDefinitions` property should include both attributes from the key schema. 24 | 25 | ### Task 26 | 27 | In this lesson, we will store data about movie roles. Each item will represent a role played by an actor in a movie. 28 | 29 | To begin, create a DynamoDB table with a composite primary key. Name your table `MovieRoles`, and give it a composite primary key with a partition (or hash) key of `Actor` and a sort (or range) key of `Movie`. 30 | 31 | Like last time, choose to use provisioned throughput with `ReadCapacityUnits` and `WriteCapacityUnits` of `5`. This will fit within the AWS Free Tier and will be more than enough for this demo. 32 | 33 | Try it on your own first, but if you want to see an example of this, look at the following files: 34 | 35 | - [Node.js](./node/createTable.js). 36 | - [Python](./python/create_table.py). 37 | 38 | ## Loading data into your table 39 | 40 | Now that you have your DynamoDB table, let's write some data into it. 41 | 42 | In the last lesson, you loaded data by calling the `PutItem` operation multiple times. If you want a faster way to load multiple items, you can use the [BatchWriteItem](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html) operation. This allows you to combine up to 25 `PutItem` requests into a single request to DynamoDB. 43 | 44 | When using the `BatchWriteItem` operation, your writes can succeed and fail independently. After you receive a response, check the `UnprocessedItems` object in the response body. If there are items in this object, they were not processed successfully and need to be retried. 45 | 46 | ### Task 47 | 48 | There is a file named `items.json` that includes four MovieRole items in DynamoDB's object format. Write a script that reads the items and inserts them into DynamoDB using the `BatchWriteItem` operation. Be sure to check for unprocessed items! 49 | 50 | If you get stuck, look to the following files for an example: 51 | 52 | - [Node.js](./node/insertItems.js). 53 | - [Python](./python/insert_items.py). 54 | 55 | ## Reading multiple items using the BatchGetItem operation 56 | 57 | In the previous lesson, you read individual items from your table using the `GetItem` operation and including the `Username` attribute for the key. You can still use the `GetItem` operation on a table that has a composite primary key. However, you must provide _both_ elements of your primary key to read an individual item. 58 | 59 | Additionally, you may have situations where you want to retrieve multiple items at the same time. If you know the full primary key for each item you want to retrieve, you can use the `BatchGetItem` operation to retrieve multiple items in a single request. With the `BatchGetItem` operation, you can retrieve up to 100 items in a single request. 60 | 61 | ### Task 62 | 63 | Write a script that uses `BatchGetItem` to read your items from your DynamoDB table. This is similar to the `GetItem` request you made in the last lesson, but it requires specifying both elements of the primary key for each item you want to retrieve. 64 | 65 | The four items in your table have the following primary key values: 66 | 67 | 1. Actor: Tom Hanks; Movie: Cast Away 68 | 2. Actor: Tom Hanks; Movie: Toy Story 69 | 3. Actor: Tim Allen; Movie: Toy Story 70 | 4. Actor: Natalie Portman; Movie: Black Swan 71 | 72 | If you're having trouble, look at the following files for an example: 73 | 74 | - [Node.js](./node/getRoles.js). 75 | - [Python](./python/get_roles.py). 76 | 77 | ## Reading multiple items with the Query operation 78 | 79 | So far, we've read specific, individual items from a table. But reading individual items is limiting in your application. Often, you'll want to read multiple, _related_ items in a single request. For example, you might want to satisfy the following access patterns: 80 | 81 | - _Give me all the Users that belong to Organization ABC_ 82 | 83 | - _Retrieve all the Pull Requests in this GitHub repository_ 84 | 85 | - _Show me all the readings for this IoT device_ 86 | 87 | In these cases, you may not know the full primary key. To handle this, you can use the [Query](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html) operation to efficiently retrieve multiple items. 88 | 89 | The Query operation can be used on a table with a composite primary key, and it can return as many items as match the request (up to 1MB of data). If your Query would match more than 1MB of data, you can make a paginated request to continue your Query where it left off. 90 | 91 | To use the Query operation, you need to include a _Key Condition Expression_ in your request. A Key Condition Expression describes the items you want to match in your Query. 92 | 93 | When writing a Key Condition Expression, you **must** include an exact match on the partition key. You _may_ include conditions on the sort key as well. 94 | 95 | Imagine you had a DynamoDB table that stored temperature readings for an IoT device. You have a composite primary key where the partition key is `DeviceId` and the sort key is `Timestamp`. 96 | 97 | If you wanted to fetch all readings for Device `1234` that occurred _after_ January 1, 2021, you could use the following parameters: 98 | 99 | ``` 100 | KeyConditionExpression="#deviceId = :deviceId AND #timestamp > :timestamp", 101 | ExpressionAttributeNames={ 102 | "#deviceId": "DeviceId", 103 | "#timestamp": "Timestamp" 104 | }, 105 | ExpressionAttributeValues={ 106 | ":deviceId": { "N": "1234" }, 107 | ":timestamp": { "S": "2021-01-01" } 108 | } 109 | ``` 110 | 111 | There's a lot going on here, so let's walk through it. 112 | 113 | Let's start with `ExpressionAttributeNames` and `ExpressionAttributeValues`. Whenever you're writing an expression in DynamoDB (including Key Condition Expressions, Update Expressions, Condition Expressions, and more), you can use these parameters. They act as variables that will be substituted into your expressions. 114 | 115 | `ExpressionAttributeNames` _may be_ used whenever you are referring to the name of an attribute in your expression. Here, we are using `DeviceId` and `Timestamp`, which are the partition key and sort key names, respectively. You don't have to use `ExpressionAttributeNames` in your expressions unless the attribute name is a reserved word. There are a [ton of reserved words in DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ReservedWords.html), including `Timestamp`, which we're using here. I generally recommend using `ExpressionAttributeNames` just to avoid consulting the documentation on reserved words. In your expressions, attribute names that are specified in `ExpressionAttributeNames` must start with a `#`. 116 | 117 | `ExpressionAttributeValues` _must be_ used whenever you are referring to the value of an attribute. This is because each attribute in DynamoDB is typed, and you must refer to the table when comparing or setting the value. Given that, it's cleaner to pull the values out into the `ExpressionAttributeValues` property. In your expressions, attribute values specified in `ExpressionAttributeValues` must start with a `:`. 118 | 119 | With that background, look at the `KeyCondiionExpression`. Using substitution from the `ExpressionAttributeNames` and `ExpressionAttributeValues`, you can think of the key condition expression as containing two statements: 120 | 121 | 1. The `DeviceId` property must be `1234`; and 122 | 2. The `Timestamp` property must be greater than `2021-01-01`. 123 | 124 | This matches our requirements on key condition expressions -- exact match on the partition key, and optional conditions on the sort key. 125 | 126 | ### Task 127 | 128 | Let's see a Query in action. There are two movies -- `Cast Away` and `Toy Story` -- where Tom Hanks played a role. Write a Query to find all of Tom Hanks' movies. 129 | 130 | Hint: you don't need to use a sort key condition here -- just the partition key match. If you want to get fancy, try adding a sort key condition to fetch all of Tom Hanks' movies whose titles are before `Forrest Gump` in the alphabet. 131 | 132 | If you get stuck, check below for examples of using the Query operation: 133 | 134 | - [Node.js](./node/queryRoles.js). 135 | - [Python](./python/query_roles.py). 136 | 137 | ## Deleting your DynamoDB Table 138 | 139 | We've completed the main steps for this lesson, so let's remove your DynamoDB table. You can use the `DeleteTable` operation to delete your table and avoid incurring charges in your AWS account. 140 | 141 | ### Task 142 | 143 | Write a script to delete your `MovieRoles` table. See below for examples. 144 | 145 | - [Node.js](./node/deleteTable.js). 146 | - [Python](./python/delete_table.py). 147 | 148 | ## Conclusion 149 | 150 | That completes this second lesson! You saw how to create a DynamoDB table with a composite primary key. You then quickly loaded data with the `BatchWriteItem` operation and read multiple items back with the `BatchGetItem` operation. Finally, you saw the power of a composite primary key when you used the `Query` operation to find multiple items with the same primary key. 151 | 152 | In the [next lesson](../03-secondary-indexes/README.md), you will learn how to enable multiple access patterns on the same item using secondary indexes. 153 | -------------------------------------------------------------------------------- /02-query/items.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "Actor": { 4 | "S": "Tom Hanks" 5 | }, 6 | "Movie": { 7 | "S": "Cast Away" 8 | }, 9 | "Role": { 10 | "S": "Chuck Noland" 11 | }, 12 | "Year": { 13 | "N": "2000" 14 | }, 15 | "Genre": { 16 | "S": "Drama" 17 | } 18 | }, 19 | { 20 | "Actor": { 21 | "S": "Tom Hanks" 22 | }, 23 | "Movie": { 24 | "S": "Toy Story" 25 | }, 26 | "Role": { 27 | "S": "Woody" 28 | }, 29 | "Year": { 30 | "N": "1995" 31 | }, 32 | "Genre": { 33 | "S": "Children's" 34 | } 35 | }, 36 | { 37 | "Actor": { 38 | "S": "Tim Allen" 39 | }, 40 | "Movie": { 41 | "S": "Toy Story" 42 | }, 43 | "Role": { 44 | "S": "Buzz Lightyear" 45 | }, 46 | "Year": { 47 | "N": "1995" 48 | }, 49 | "Genre": { 50 | "S": "Children's" 51 | } 52 | }, 53 | { 54 | "Actor": { 55 | "S": "Natalie Portman" 56 | }, 57 | "Movie": { 58 | "S": "Black Swan" 59 | }, 60 | "Role": { 61 | "S": "Nina Sayers" 62 | }, 63 | "Year": { 64 | "N": "2010" 65 | }, 66 | "Genre": { 67 | "S": "Drama" 68 | } 69 | } 70 | ] 71 | -------------------------------------------------------------------------------- /02-query/node/createTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.createTable( 5 | { 6 | TableName: "MovieRoles", 7 | AttributeDefinitions: [ 8 | { 9 | AttributeName: "Actor", 10 | AttributeType: "S", 11 | }, 12 | { 13 | AttributeName: "Movie", 14 | AttributeType: "S", 15 | }, 16 | ], 17 | KeySchema: [ 18 | { AttributeName: "Actor", KeyType: "HASH" }, 19 | { AttributeName: "Movie", KeyType: "RANGE" }, 20 | ], 21 | ProvisionedThroughput: { 22 | ReadCapacityUnits: 5, 23 | WriteCapacityUnits: 5, 24 | }, 25 | }, 26 | function (err) { 27 | if (err) console.log(`Error creating table: ${err}`); 28 | else console.log("Table created succesfully! 🚀"); 29 | } 30 | ); 31 | -------------------------------------------------------------------------------- /02-query/node/deleteTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.deleteTable( 5 | { 6 | TableName: "MovieRoles", 7 | }, 8 | function (err) { 9 | if (err) console.log(`Error deleting table: ${err}`); 10 | else console.log("Table deleted successfully! 🙌"); 11 | } 12 | ); 13 | -------------------------------------------------------------------------------- /02-query/node/getRoles.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.batchGetItem( 5 | { 6 | RequestItems: { 7 | MovieRoles: { 8 | Keys: [ 9 | { Actor: { S: "Tom Hanks" }, Movie: { S: "Cast Away" } }, 10 | { Actor: { S: "Tom Hanks" }, Movie: { S: "Toy Story" } }, 11 | { Actor: { S: "Tim Allen" }, Movie: { S: "Toy Story" } }, 12 | { Actor: { S: "Natalie Portman" }, Movie: { S: "Black Swan" } }, 13 | ], 14 | }, 15 | }, 16 | }, 17 | function (err, data) { 18 | if (err) console.log(`Error fetching roles": ${err}`); 19 | else if (!data.Responses) console.log("Roles do not exist"); 20 | else console.log(`Found roles: ${JSON.stringify(data.Responses, null, 2)}`); 21 | } 22 | ); 23 | -------------------------------------------------------------------------------- /02-query/node/insertItems.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | const fs = require("fs"); 5 | const path = require("path"); 6 | 7 | const data = fs.readFileSync(path.join(__dirname, "../items.json")); 8 | const movieRoles = JSON.parse(data); 9 | 10 | const requests = movieRoles.map((role) => { 11 | return { 12 | PutRequest: { 13 | Item: role, 14 | }, 15 | }; 16 | }); 17 | 18 | DynamoDB.batchWriteItem( 19 | { 20 | RequestItems: { 21 | MovieRoles: requests, 22 | }, 23 | }, 24 | function (err, data) { 25 | if (err) console.log(`Error creating items: ${err}`); 26 | else if (data.UnprocessedItems.MovieRoles) 27 | console.log( 28 | `Unprocessed items. Run again. ${JSON.stringify(data.UnprocessedItems)}` 29 | ); 30 | else console.log(`Movie role items created successfully!`); 31 | } 32 | ); 33 | -------------------------------------------------------------------------------- /02-query/node/queryRoles.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.query( 5 | { 6 | TableName: "MovieRoles", 7 | KeyConditionExpression: "Actor = :actor", 8 | ExpressionAttributeValues: { 9 | ":actor": { S: "Tom Hanks" }, 10 | }, 11 | }, 12 | function (err, data) { 13 | if (err) console.log(`Error performing query": ${err}`); 14 | else 15 | console.log( 16 | `Found ${data.Items.length} roles!\n${JSON.stringify(data.Items)}` 17 | ); 18 | } 19 | ); 20 | -------------------------------------------------------------------------------- /02-query/python/create_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.create_table( 7 | TableName="MovieRoles", 8 | AttributeDefinitions=[ 9 | {"AttributeName": "Actor", "AttributeType": "S"}, 10 | {"AttributeName": "Movie", "AttributeType": "S"}, 11 | ], 12 | KeySchema=[ 13 | {"AttributeName": "Actor", "KeyType": "HASH"}, 14 | {"AttributeName": "Movie", "KeyType": "RANGE"}, 15 | ], 16 | ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, 17 | ) 18 | print("Table created successfully! 🚀") 19 | except Exception as err: 20 | print(f"Error creating table: {err}") 21 | -------------------------------------------------------------------------------- /02-query/python/delete_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.delete_table( 7 | TableName="MovieRoles", 8 | ) 9 | print("Table deleted successfully! 🙌") 10 | except Exception as err: 11 | print(f"Error deleting table: {err}") 12 | -------------------------------------------------------------------------------- /02-query/python/get_roles.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import boto3 4 | 5 | client = boto3.client("dynamodb", region_name="us-east-1") 6 | 7 | try: 8 | response = client.batch_get_item( 9 | RequestItems={ 10 | "MovieRoles": { 11 | "Keys": [ 12 | {"Actor": {"S": "Tom Hanks"}, "Movie": {"S": "Cast Away"}}, 13 | {"Actor": {"S": "Tom Hanks"}, "Movie": {"S": "Toy Story"}}, 14 | {"Actor": {"S": "Tim Allen"}, "Movie": {"S": "Toy Story"}}, 15 | { 16 | "Actor": {"S": "Natalie Portman"}, 17 | "Movie": {"S": "Black Swan"}, 18 | }, 19 | ] 20 | } 21 | } 22 | ) 23 | if not response["Responses"]: 24 | print("Roles do not exist") 25 | else: 26 | print(f"Found roles: {json.dumps(response['Responses'], indent=2)}") 27 | except Exception as err: 28 | print(f"Error fetching roles: {err}") 29 | -------------------------------------------------------------------------------- /02-query/python/insert_items.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | import boto3 5 | 6 | client = boto3.client("dynamodb", region_name="us-east-1") 7 | 8 | lesson_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 9 | items_path = os.path.join(lesson_directory, "items.json") 10 | 11 | with open(items_path, "r") as f: 12 | roles = json.load(f) 13 | 14 | requests = [{"PutRequest": {"Item": role}} for role in roles] 15 | 16 | try: 17 | response = client.batch_write_item(RequestItems={"MovieRoles": requests}) 18 | if response["UnprocessedItems"].get("MovieRoles"): 19 | print( 20 | f"Unprocessed items. Run again. {json.dumps(response['UnprocessedItems']['MovieRoles'])}" 21 | ) 22 | else: 23 | print("Movie role items created successfully!") 24 | except Exception as err: 25 | raise err 26 | print(f"Error creating items: {err}") 27 | -------------------------------------------------------------------------------- /02-query/python/query_roles.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import boto3 4 | 5 | client = boto3.client("dynamodb", region_name="us-east-1") 6 | 7 | try: 8 | response = client.query( 9 | TableName="MovieRoles", 10 | KeyConditionExpression="#actor = :actor", 11 | ExpressionAttributeNames={"#actor": "Actor"}, 12 | ExpressionAttributeValues={":actor": {"S": "Tom Hanks"}}, 13 | ) 14 | print( 15 | f"Found {len(response['Items'])} roles!\n{json.dumps(response['Items'], indent=2)}" 16 | ) 17 | except Exception as err: 18 | print(f"Error fetching roles: {err}") 19 | -------------------------------------------------------------------------------- /03-secondary-indexes/README.md: -------------------------------------------------------------------------------- 1 | # DynamoDB -- Handling additional access patterns with secondary indexes 2 | 3 | In the previous lesson, you learned how to use DynamoDB as more than a key-value store by using the composite primary key and Query operation. We saw that the primary key structure is important as it drives access patterns for your items. 4 | 5 | However, what if you need to allow multiple, different access pattern for a certain type of items? How can you enable these patterns with only a single primary key? 6 | 7 | In this final lesson, we'll learn how to enable multiple access patterns on the same item using secondary indexes. 8 | 9 | This lesson has four steps: 10 | 11 | - [Creating a DynamoDB table with a secondary index](#creating-a-dynamodb-table-with-a-secondary-index); 12 | - [Loading data into your table](#loading-data-into-your-table) 13 | - [Using the Query operation on your secondary index](#using-the-query-operation-on-your-secondary-index); 14 | - [Deleting your DynamoDB table](#deleting-your-dynamodb-table); 15 | 16 | ## Creating a DynamoDB table with a secondary index 17 | 18 | In this lesson, we will start by creating a DynamoDB table. The table will be similar as the one in the last lesson with one difference -- we'll be adding a global secondary index. 19 | 20 | > Note: there are two types of secondary indexes -- global and local. In almost all occasions, you'll want to use a global secondary index. For the rest of this lesson, I'll use "global secondary index" and "secondary index" interchangeably. For more on the types of indexes, check out [Local or Global: Choosing a secondary index type in DynamoDB](https://www.dynamodbguide.com/local-or-global-choosing-a-secondary-index-type-in-dynamo-db/). 21 | 22 | A secondary index is something you create on your DynamoDB table that gives you additional access patterns on the items in your table. When you add a secondary index to your table, you will declare the primary key schema for the secondary index. When an item is written into your table, DynamoDB will check if the item has the attributes for your secondary index's primary key schema. If it does, the item will be copied into the secondary index with the primary key for the secondary index. You can then issue read requests against your secondary index to access items with secondary access patterns. 23 | 24 | In essence, a secondary index gives you an additional, read-only view on your data. 25 | 26 | There are a few main differences to note between your base table and your secondary indexes: 27 | 28 | 1. Unlike your base table, attributes for your secondary index key schema are not required on each item. If an item is written to your table but is missing one or both of the elements to your secondary index, it won't be written to your secondary index. This is often helpful in a pattern called a "sparse index". 29 | 30 | 2. Primary key values for your secondary index do not have to be unique. You can have multiple items with the same key schema values in your secondary index. 31 | 32 | 3. You cannot do writes to your secondary index -- only reads. All write operations need to go through your base table. 33 | 34 | 4. Data is asychronously replicated to your secondary indexes, so you could see slightly stale data in your secondary index. 35 | 36 | Your DynamoDB table may have up to 20 global secondary indexes and up to 5 local secondary indexes. Global and local secondary indexes may be added when you create the table, and global secondary indexes may be added to your table after creation. 37 | 38 | To add a secondary index, you need to specify: 39 | 40 | - The index name; 41 | - The key schema for your secondary index; 42 | - The provisioned throughput (if your table is not using on-demand billing); 43 | - The index projection (do you want the entire item copied over or a subset?) 44 | 45 | Additionally, if you use attributes in your secondary index that are not already used in the primary key or another secondary index, you will need to specify them in `AttributeDefinitions` when creating your table. 46 | 47 | For specifics on each of these properties, [check out the CreateTable docs](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html#DDB-CreateTable-request-GlobalSecondaryIndexes). 48 | 49 | ### Task 50 | 51 | In this lesson, we will continue with our movie roles example from the last lesson. Each item with represent a role played by an actor in a movie. 52 | 53 | Recall that our primary key for the last lesson used a partition key of `Actor` and a sort key of `Movie`. This allowed us to fetch all movie roles for a particular Actor. 54 | 55 | But what if we have additional access patterns? For example, we may want to get all the movie roles for a particular genre, or even the movies for a genre for a specific year or range of years. 56 | 57 | We can handle these additional access patterns with a secondary index. 58 | 59 | Write and execute a script that will create a `MovieRoles` table. You can build on the script you wrote in the last lesson. This script should include a global secondary index called `GenreYearIndex` whose key schema uses `Genre` for the partition key and `Year` for the sort key. 60 | 61 | Try it on your own first, but if you want to see an example of this, look at the following files: 62 | 63 | - [Node.js](./node/createTable.js). 64 | - [Python](./python/create_table.py). 65 | 66 | ## Loading data into your table 67 | 68 | We're going to re-use the data from the last lesson, so use the same script to insert the `items.json` items into your table. 69 | 70 | ### Task 71 | 72 | There is a file named `items.json` that includes four MovieRole items in DynamoDB's object format. Write a script that reads the items and inserts them into DynamoDB using the `BatchWriteItem` operation. Be sure to check for unprocessed items! 73 | 74 | If you get stuck, look to the following files for examples: 75 | 76 | - [Node.js](./node/insertItems.js). 77 | - [Python](./python/insert_items.py). 78 | 79 | ## Using the Query operation on your secondary index 80 | 81 | Just like in the last lesson, we have an access pattern -- "Fetch all roles in a given genre" -- that is a "fetch many" access pattern. To handle this, we'll use the Query operation again. In this case, we'll be running the Query against our secondary index. 82 | 83 | ### Task 84 | 85 | Use the Query operation on your secondary index to query for all roles for a given genre. The structure around `KeyConditionExpression` should be similar to the last one. The biggest difference here is that you'll need to pass an `IndexName` property into the Query operation to indicate you want to use the secondary index. 86 | 87 | Use the genre of `Drama` to execute your queries. It should return two roles -- Tom Hanks in Cast Away and Natalie Portman in Black Swan. 88 | 89 | If you get stuck, check below for an example of using Query with a secondary index: 90 | 91 | - [Node.js](./node/queryRoles.js). 92 | - [Python](./python/query_roles.py). 93 | 94 | ## Deleting your DynamoDB Table 95 | 96 | We've completed the main steps for this lesson, so let's remove your DynamoDB table. You can use the `DeleteTable` operation to delete your table and avoid incurring charges in your AWS account. 97 | 98 | ### Task 99 | 100 | Write a script to delete your `MovieRoles` table. See below for an example. 101 | 102 | - [Node.js](./node/deleteTable.js). 103 | - [Python](./python/delete_table.py). 104 | 105 | ## Conclusion 106 | 107 | This completes the third lesson and the workshop. In this lesson, we learned about supporting additional access patterns in our data via secondary indexes. We learned some specifics about secondary indexes, including how to add them to our table. Finally, we used the Query operation against our secondary index. 108 | 109 | This completes the DynamoDB workshop. At this point, you should be familiar with some of the core concepts around DynamoDB, including primary keys, expressions, and secondary indexes. You should also be familiar with some of the core operations against DynamoDB, such as GetItem, PutItem, and Query. 110 | 111 | There's a lot more to learn about effective data modeling with DynamoDB, so don't stop learning here! :) 112 | -------------------------------------------------------------------------------- /03-secondary-indexes/items.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "Actor": { 4 | "S": "Tom Hanks" 5 | }, 6 | "Movie": { 7 | "S": "Cast Away" 8 | }, 9 | "Role": { 10 | "S": "Chuck Noland" 11 | }, 12 | "Year": { 13 | "N": "2000" 14 | }, 15 | "Genre": { 16 | "S": "Drama" 17 | } 18 | }, 19 | { 20 | "Actor": { 21 | "S": "Tom Hanks" 22 | }, 23 | "Movie": { 24 | "S": "Toy Story" 25 | }, 26 | "Role": { 27 | "S": "Woody" 28 | }, 29 | "Year": { 30 | "N": "1995" 31 | }, 32 | "Genre": { 33 | "S": "Children's" 34 | } 35 | }, 36 | { 37 | "Actor": { 38 | "S": "Tim Allen" 39 | }, 40 | "Movie": { 41 | "S": "Toy Story" 42 | }, 43 | "Role": { 44 | "S": "Buzz Lightyear" 45 | }, 46 | "Year": { 47 | "N": "1995" 48 | }, 49 | "Genre": { 50 | "S": "Children's" 51 | } 52 | }, 53 | { 54 | "Actor": { 55 | "S": "Natalie Portman" 56 | }, 57 | "Movie": { 58 | "S": "Black Swan" 59 | }, 60 | "Role": { 61 | "S": "Nina Sayers" 62 | }, 63 | "Year": { 64 | "N": "2010" 65 | }, 66 | "Genre": { 67 | "S": "Drama" 68 | } 69 | } 70 | ] 71 | -------------------------------------------------------------------------------- /03-secondary-indexes/node/createTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.createTable( 5 | { 6 | TableName: "MovieRoles", 7 | AttributeDefinitions: [ 8 | { 9 | AttributeName: "Actor", 10 | AttributeType: "S", 11 | }, 12 | { 13 | AttributeName: "Movie", 14 | AttributeType: "S", 15 | }, 16 | { 17 | AttributeName: "Genre", 18 | AttributeType: "S", 19 | }, 20 | { 21 | AttributeName: "Year", 22 | AttributeType: "N", 23 | }, 24 | ], 25 | KeySchema: [ 26 | { AttributeName: "Actor", KeyType: "HASH" }, 27 | { AttributeName: "Movie", KeyType: "RANGE" }, 28 | ], 29 | GlobalSecondaryIndexes: [ 30 | { 31 | IndexName: "GenreYearIndex", 32 | KeySchema: [ 33 | { AttributeName: "Genre", KeyType: "HASH" }, 34 | { AttributeName: "Year", KeyType: "RANGE" }, 35 | ], 36 | ProvisionedThroughput: { 37 | ReadCapacityUnits: 5, 38 | WriteCapacityUnits: 5, 39 | }, 40 | Projection: { 41 | ProjectionType: "ALL", 42 | }, 43 | }, 44 | ], 45 | ProvisionedThroughput: { 46 | ReadCapacityUnits: 5, 47 | WriteCapacityUnits: 5, 48 | }, 49 | }, 50 | function (err) { 51 | if (err) console.log(`Error creating table: ${err}`); 52 | else console.log("Table created succesfully! 🚀"); 53 | } 54 | ); 55 | -------------------------------------------------------------------------------- /03-secondary-indexes/node/deleteTable.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.deleteTable( 5 | { 6 | TableName: "MovieRoles", 7 | }, 8 | function (err) { 9 | if (err) console.log(`Error deleting table: ${err}`); 10 | else console.log("Table deleted successfully! 🙌"); 11 | } 12 | ); 13 | -------------------------------------------------------------------------------- /03-secondary-indexes/node/insertItems.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | const fs = require("fs"); 5 | const path = require("path"); 6 | 7 | const data = fs.readFileSync(path.join(__dirname, "../items.json")); 8 | const movieRoles = JSON.parse(data); 9 | 10 | const requests = movieRoles.map((role) => { 11 | return { 12 | PutRequest: { 13 | Item: role, 14 | }, 15 | }; 16 | }); 17 | 18 | DynamoDB.batchWriteItem( 19 | { 20 | RequestItems: { 21 | MovieRoles: requests, 22 | }, 23 | }, 24 | function (err, data) { 25 | if (err) console.log(`Error creating items: ${err}`); 26 | else if (data.UnprocessedItems.MovieRoles) 27 | console.log( 28 | `Unprocessed items. Run again. ${JSON.stringify(data.UnprocessedItems)}` 29 | ); 30 | else console.log(`Movie role items created successfully!`); 31 | } 32 | ); 33 | -------------------------------------------------------------------------------- /03-secondary-indexes/node/queryRoles.js: -------------------------------------------------------------------------------- 1 | const AWS = require("aws-sdk"); 2 | const DynamoDB = new AWS.DynamoDB({ region: "us-east-1" }); 3 | 4 | DynamoDB.query( 5 | { 6 | TableName: "MovieRoles", 7 | IndexName: "GenreYearIndex", 8 | KeyConditionExpression: "#genre= :genre", 9 | ExpressionAttributeNames: { 10 | "#genre": "Genre", 11 | }, 12 | ExpressionAttributeValues: { 13 | ":genre": { S: "Drama" }, 14 | }, 15 | }, 16 | function (err, data) { 17 | if (err) console.log(`Error performing query": ${err}`); 18 | else 19 | console.log( 20 | `Found ${data.Items.length} roles!\n${JSON.stringify(data.Items)}` 21 | ); 22 | } 23 | ); 24 | -------------------------------------------------------------------------------- /03-secondary-indexes/python/create_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.create_table( 7 | TableName="MovieRoles", 8 | AttributeDefinitions=[ 9 | {"AttributeName": "Actor", "AttributeType": "S"}, 10 | {"AttributeName": "Movie", "AttributeType": "S"}, 11 | {"AttributeName": "Genre", "AttributeType": "S"}, 12 | {"AttributeName": "Year", "AttributeType": "N"}, 13 | ], 14 | KeySchema=[ 15 | {"AttributeName": "Actor", "KeyType": "HASH"}, 16 | {"AttributeName": "Movie", "KeyType": "RANGE"}, 17 | ], 18 | GlobalSecondaryIndexes=[ 19 | { 20 | "IndexName": "GenreYearIndex", 21 | "KeySchema": [ 22 | {"AttributeName": "Genre", "KeyType": "HASH"}, 23 | {"AttributeName": "Year", "KeyType": "RANGE"}, 24 | ], 25 | "ProvisionedThroughput": { 26 | "ReadCapacityUnits": 5, 27 | "WriteCapacityUnits": 5, 28 | }, 29 | "Projection": {"ProjectionType": "ALL"}, 30 | } 31 | ], 32 | ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, 33 | ) 34 | print("Table created successfully! 🚀") 35 | except Exception as err: 36 | print(f"Error creating table: {err}") 37 | -------------------------------------------------------------------------------- /03-secondary-indexes/python/delete_table.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | 3 | client = boto3.client("dynamodb", region_name="us-east-1") 4 | 5 | try: 6 | client.delete_table( 7 | TableName="MovieRoles", 8 | ) 9 | print("Table deleted successfully! 🙌") 10 | except Exception as err: 11 | print(f"Error deleting table: {err}") 12 | -------------------------------------------------------------------------------- /03-secondary-indexes/python/insert_items.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | import boto3 5 | 6 | client = boto3.client("dynamodb", region_name="us-east-1") 7 | 8 | lesson_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 9 | items_path = os.path.join(lesson_directory, "items.json") 10 | 11 | with open(items_path, "r") as f: 12 | roles = json.load(f) 13 | 14 | requests = [{"PutRequest": {"Item": role}} for role in roles] 15 | 16 | try: 17 | response = client.batch_write_item(RequestItems={"MovieRoles": requests}) 18 | if response["UnprocessedItems"].get("MovieRoles"): 19 | print( 20 | f"Unprocessed items. Run again. {json.dumps(response['UnprocessedItems']['MovieRoles'])}" 21 | ) 22 | else: 23 | print("Movie role items created successfully!") 24 | except Exception as err: 25 | raise err 26 | print(f"Error creating items: {err}") 27 | -------------------------------------------------------------------------------- /03-secondary-indexes/python/query_roles.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import boto3 4 | 5 | client = boto3.client("dynamodb", region_name="us-east-1") 6 | 7 | try: 8 | response = client.query( 9 | TableName="MovieRoles", 10 | IndexName="GenreYearIndex", 11 | KeyConditionExpression="#genre= :genre", 12 | ExpressionAttributeNames={"#genre": "Genre"}, 13 | ExpressionAttributeValues={":genre": {"S": "Drama"}}, 14 | ) 15 | print( 16 | f"Found {len(response['Items'])} roles!\n{json.dumps(response['Items'], indent=2)}" 17 | ) 18 | except Exception as err: 19 | print(f"Error fetching roles: {err}") 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DynamoDB Workshop 2 | 3 | This repository contains materials for a workshop to learn the basics about DynamoDB. It is focused on core concepts and API operations, but it doesn't dive deep into advanced data modeling concepts such as single-table design. If you want to know more about data modeling with DynamoDB, check out [The DynamoDB Book](https://www.dynamodbbook.com/) or some of the [DynamoDB content on my blog](https://www.alexdebrie.com/). 4 | 5 | There are slides that go along with this workshop, but you can _probably_ get along without them. You might just have to Google a few concepts if you don't understand them. 6 | 7 | This workshop is separated into three lessons with are to be completed in order: 8 | 9 | 1. **Basic Operations** 10 | 11 | In this lesson, you will learn some of the core vocabulary of DynamoDB, including primary keys, attributes, and provisioned throughput. Then, you will create your first DynamoDB table with a simple primary key. Finally, you will interact with the DynamoDB API by writing to and reading from your table. You'll see how to use condition expressions to avoid overwriting existing data. 12 | 13 | [Start Lesson 1 here](./01-basic-operations/README.md) 14 | 15 | 2. **Composite primary keys and the Query operation** 16 | 17 | In the second lesson, you'll learn how to use DynamoDB as more than a key-value store. You'll create a DynamoDB table with a composite primary key and use the Query operation to retrieve multiple items in a single request. This will lay the foundation for more advanced use cases in your applications. 18 | 19 | [Start Lesson 2 here](./02-query/README.md) 20 | 21 | 3. **Handling additional access patterns with secondary indexes** 22 | 23 | In the third and final lesson, you'll see how to handle multiple access pattern on the same items. You'll use secondary indexes to reshape your data and handle even more complex access patterns. 24 | 25 | [Start Lesson 3 here](./03-secondary-indexes/README.md) 26 | -------------------------------------------------------------------------------- /package-lock.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "dynamodb-workshop", 3 | "version": "1.0.0", 4 | "lockfileVersion": 1, 5 | "requires": true, 6 | "dependencies": { 7 | "aws-sdk": { 8 | "version": "2.836.0", 9 | "resolved": "https://registry.npmjs.org/aws-sdk/-/aws-sdk-2.836.0.tgz", 10 | "integrity": "sha512-lVOT5/yr9ONfbn6UXYYIdFlVIFmSoX0CnjAQzXkfcYg+k7CZklbqqVIdgdLoiQYwQGLceoWghSevPGe7fjFg9Q==", 11 | "requires": { 12 | "buffer": "4.9.2", 13 | "events": "1.1.1", 14 | "ieee754": "1.1.13", 15 | "jmespath": "0.15.0", 16 | "querystring": "0.2.0", 17 | "sax": "1.2.1", 18 | "url": "0.10.3", 19 | "uuid": "3.3.2", 20 | "xml2js": "0.4.19" 21 | } 22 | }, 23 | "base64-js": { 24 | "version": "1.5.1", 25 | "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", 26 | "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==" 27 | }, 28 | "buffer": { 29 | "version": "4.9.2", 30 | "resolved": "https://registry.npmjs.org/buffer/-/buffer-4.9.2.tgz", 31 | "integrity": "sha512-xq+q3SRMOxGivLhBNaUdC64hDTQwejJ+H0T/NB1XMtTVEwNTrfFF3gAxiyW0Bu/xWEGhjVKgUcMhCrUy2+uCWg==", 32 | "requires": { 33 | "base64-js": "^1.0.2", 34 | "ieee754": "^1.1.4", 35 | "isarray": "^1.0.0" 36 | } 37 | }, 38 | "events": { 39 | "version": "1.1.1", 40 | "resolved": "https://registry.npmjs.org/events/-/events-1.1.1.tgz", 41 | "integrity": "sha1-nr23Y1rQmccNzEwqH1AEKI6L2SQ=" 42 | }, 43 | "ieee754": { 44 | "version": "1.1.13", 45 | "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.13.tgz", 46 | "integrity": "sha512-4vf7I2LYV/HaWerSo3XmlMkp5eZ83i+/CDluXi/IGTs/O1sejBNhTtnxzmRZfvOUqj7lZjqHkeTvpgSFDlWZTg==" 47 | }, 48 | "isarray": { 49 | "version": "1.0.0", 50 | "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", 51 | "integrity": "sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE=" 52 | }, 53 | "jmespath": { 54 | "version": "0.15.0", 55 | "resolved": "https://registry.npmjs.org/jmespath/-/jmespath-0.15.0.tgz", 56 | "integrity": "sha1-o/Iiqarp+Wb10nx5ZRDigJF2Qhc=" 57 | }, 58 | "punycode": { 59 | "version": "1.3.2", 60 | "resolved": "https://registry.npmjs.org/punycode/-/punycode-1.3.2.tgz", 61 | "integrity": "sha1-llOgNvt8HuQjQvIyXM7v6jkmxI0=" 62 | }, 63 | "querystring": { 64 | "version": "0.2.0", 65 | "resolved": "https://registry.npmjs.org/querystring/-/querystring-0.2.0.tgz", 66 | "integrity": "sha1-sgmEkgO7Jd+CDadW50cAWHhSFiA=" 67 | }, 68 | "sax": { 69 | "version": "1.2.1", 70 | "resolved": "https://registry.npmjs.org/sax/-/sax-1.2.1.tgz", 71 | "integrity": "sha1-e45lYZCyKOgaZq6nSEgNgozS03o=" 72 | }, 73 | "url": { 74 | "version": "0.10.3", 75 | "resolved": "https://registry.npmjs.org/url/-/url-0.10.3.tgz", 76 | "integrity": "sha1-Ah5NnHcF8hu/N9A861h2dAJ3TGQ=", 77 | "requires": { 78 | "punycode": "1.3.2", 79 | "querystring": "0.2.0" 80 | } 81 | }, 82 | "uuid": { 83 | "version": "3.3.2", 84 | "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.3.2.tgz", 85 | "integrity": "sha512-yXJmeNaw3DnnKAOKJE51sL/ZaYfWJRl1pK9dr19YFCu0ObS231AB1/LbqTKRAQ5kw8A90rA6fr4riOUpTZvQZA==" 86 | }, 87 | "xml2js": { 88 | "version": "0.4.19", 89 | "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.4.19.tgz", 90 | "integrity": "sha512-esZnJZJOiJR9wWKMyuvSE1y6Dq5LCuJanqhxslH2bxM6duahNZ+HMpCLhBQGZkbX6xRf8x1Y2eJlgt2q3qo49Q==", 91 | "requires": { 92 | "sax": ">=0.6.0", 93 | "xmlbuilder": "~9.0.1" 94 | } 95 | }, 96 | "xmlbuilder": { 97 | "version": "9.0.7", 98 | "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-9.0.7.tgz", 99 | "integrity": "sha1-Ey7mPS7FVlxVfiD0wi35rKaGsQ0=" 100 | } 101 | } 102 | } 103 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "dynamodb-workshop", 3 | "version": "1.0.0", 4 | "description": "", 5 | "main": "index.js", 6 | "scripts": { 7 | "test": "echo \"Error: no test specified\" && exit 1" 8 | }, 9 | "keywords": [], 10 | "author": "", 11 | "license": "ISC", 12 | "dependencies": { 13 | "aws-sdk": "^2.836.0" 14 | } 15 | } 16 | --------------------------------------------------------------------------------