├── .gitignore ├── Part1-Installation ├── ubuntu.MD └── windows.MD ├── Part2-Basics ├── Tutorial-1.MD └── commands.sql ├── Part3-Advanced data selection ├── Tutorial-2.MD └── commands.sql ├── Part4-Relationships and JOINS ├── Tutorial-3.MD ├── commands.sql └── img.png ├── Part5-Subqueries and Ranks ├── Tutorial-4.MD ├── commands.sql └── preparation.sql ├── Part6-SQLAlchemy - Basics ├── Tutorial-5.MD ├── main.py └── tutorial-5.ipynb ├── README.MD └── img.png /.gitignore: -------------------------------------------------------------------------------- 1 | .idea/* -------------------------------------------------------------------------------- /Part1-Installation/ubuntu.MD: -------------------------------------------------------------------------------- 1 | 1. Update the system 2 | `sudo apt update` 3 | 4 | 5 | 2. Install the postgresql 6 | `sudo apt install postgresql` 7 | 8 | 9 | 3. Start the postgresql 10 | `sudo systemctl start postgresql.service` 11 | 12 | 13 | 4. Log into the postgres user 14 | `su - postgres` 15 | 16 | 17 | 5. First create a role with password and superuser privileges 18 | `createuser --interactive --pwprompt --createdb --superuser testuser` 19 | 20 | 21 | 6. Then enter the database 22 | `psql -U testuser --db testuser` 23 | 24 | 25 | -------------------------------------------------------------------------------- /Part1-Installation/windows.MD: -------------------------------------------------------------------------------- 1 | 1. Go to 2 | the [link and follow the instructions](https://www.postgresqltutorial.com/postgresql-getting-started/install-postgresql/) 3 | to install PostgreSQL. 4 | 5 | 2. Run the SQL Shell and enter the default user. 6 | 7 | 3. Create a new role with password and superuser privileges. You may see in other tutorials that 'user' is created, but 8 | in more recent versions [it is better to user 'role' concept](https://stackoverflow.com/questions/27709456/what-is-the-difference-between-a-user-and-a-role) 9 | . 10 | 11 | `CREATE ROLE testuser WITH LOGIN ENCRYPTED PASSWORD 'testpassword' SUPERUSER;` 12 | 13 | 4. Create a new database. 14 | 15 | `CREATE DATABASE testuser OWNER testuser;` 16 | 17 | 5. Quit the shell and enter the new database with the new user. 18 | 19 | -------------------------------------------------------------------------------- /Part2-Basics/Tutorial-1.MD: -------------------------------------------------------------------------------- 1 | ### For full info check this tutorial: [Tutorial](https://www.postgresqltutorial.com/) 2 | 3 | This is not a full tutorial, it's just a brief introduction to PostgreSQL and a script for video-tutorial. 4 | 5 | # Part 1. Basics. 6 | 7 | ## How data is stored in PostgreSQL? 8 | 9 | - To store information in a database, you need to create a **table**. 10 | - Tables always contain rows and columns. Tables also called _'relations'_ 11 | - Rows are called records and contain different data. Also called _'tuples'_. 12 | - Columns separate data in one row by different types. Also called _'fields/attributes'_. 13 | - Rows are horizontal. 14 | - Columns are vertical. 15 | 16 | Example of a table: 17 | 18 | ![img.png](../img.png) 19 | 20 | #### In this example you have 3 columns: 21 | 22 | 1. telegram_id - integer (bigint / 64bit). _This is the unique identifier of the user in Telegram_. 23 | 2. full_name - varchar / string. _Stores full name of the user_. 24 | 3. created_at - timestamp (timestamp). _Stores time when the user was added to database_. 25 | 26 | ### SQL Syntax 27 | 28 | - Every action in PostgreSQL is performed by some SQL statement. 29 | - Every statement must be separated by semicolon: `;` 30 | 31 | **Most common SQL** statements are written in the following format: 32 | 33 | ``` 34 | ; 35 | 36 | ; 37 | 38 | ; 39 | ... 40 | ``` 41 | 42 | > **Note:** 43 | > 44 | > - SQL **commands are case-insensitive**. You can write `SELECT`/`select`/`Select` and all will work. 45 | > - **Indentation is unimportant**. You can put newlines and tabs wherever you want. 46 | > - SQL is very effective for SWIFT storing and retrieving data. 47 | 48 | ### SQL subtypes: 49 | 50 | You can use SQL to manipulate database/table structure and settings, and most often **to manipulate data**. 51 | 52 | There are different subtypes of SQL: 53 | 54 | - DDL (Data Definition Language) - used to create/alter/drop tables. 55 | - DML (Data Manipulation Language) - used to insert/update/delete data. 56 | - DCL (Data Control Language) - used to grant/revoke privileges for users. 57 | - TCL (Transaction Control Language) - used to start/commit/rollback transactions. 58 | 59 | > **Note:** 60 | > - Transactions are used to prevent data loss. You can run multiple SQL statements in one transaction. If one of them 61 | fails, all of them will be rolled back. This is useful for example when you want to insert data into one table, but 62 | you don't want to insert data into second table if the first one fails. 63 | > - Priviledges are used to grant/revoke access to different commands for users. You can restrict one user to 64 | use `INSERT`, 65 | `UPDATE`, `DELETE` in a table, but allow to `SELECT` data. 66 | 67 | ### Actions 68 | 69 | - `SELECT` - selects data from a table. 70 | - `INSERT` - Creates new records in a table. 71 | - `UPDATE` - updates data in a table. 72 | - `DELETE` - deletes data from a table. 73 | - `CREATE TABLE` - creates a table. 74 | - `DROP TABLE` - drops a table. 75 | - `ALTER TABLE` - alters a table. 76 | - ... 77 | 78 | Firstly, to perform some action you need to create a table. 79 | 80 | #### To create a table you will need to: 81 | 82 | 1. Think what data you want to keep track of. _Example: Data about registered users._ 83 | 2. Think about a name for the table. Better use simple lowercase names. _Example: users_ 84 | 3. List all variety of data and each datatype (E.g. Python datatype) you want to store. _Example: telegram_id - integer, 85 | full_name - text, created_at - date and time._ 86 | 4. Determine what [types of data](https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-data-types/) are 87 | supported by your database system. _Example: PostgreSQL supports `BIGINT`, `VARCHAR`, `TIMESTAMP`, respectively_. 88 | 5. Determine which [constraints](https://www.tutorialspoint.com/postgresql/postgresql_constraints.htm) you want to apply 89 | to your table. _Example: You can use `PRIMARY KEY` to make sure that each row has a unique and not empty identifier. 90 | PostgreSQL will throw an error if you try to insert a duplicate identifier. In this example it is `telegram_id`_ 91 | 6. If you want to create a `PRIMARY KEY` that will be automatically filled `(e.g. 1, 2, 3 ...)`, you can use `SERIAL` 92 | datatype with `PRIMARY KEY`. 93 | _Example: `telegram_id SERIAL PRIMARY KEY`_ 94 | 7. Determine which columns are nullable, and which ones are not. _Example: You can use `NOT NULL` to make sure that 95 | each row has a `telegram_id` and `full_name`._ 96 | 97 | > Syntax: `CREATE TABLE table_name (column_name data_type, column_name data_type, ...);` 98 | 99 | > **Note:** 100 | > 101 | > You can set the default value of a column by adding a `DEFAULT` clause after the data type. 102 | > 103 | > You can set additional constraints for a column. 104 | 105 | #### Data types 106 | 107 | - `INTEGER` - integer 32-bit. Same as Python `int` type. 108 | - `BIGINT` - integer 64-bit. Same as Python `int` type. 109 | - `FLOAT` - Same as Python `float` type. 110 | - `BOOLEAN` - Same as Python `bool` type. 111 | - `TEXT` - Same as Python `str` type of unknown length. 112 | - `VARCHAR` - Same as Python `str` type of fixed length. 113 | - `DATE` - Same as Python `datetime.date` type. 114 | - `TIME` - Same as Python `datetime.time` type. 115 | - `TIMESTAMP` - Same as Python `datetime.datetime` type. 116 | - `SERIAL` - An autoincrementing integer (e.g. 1, 2, 3 ...). It will be calculated automatically by PostgreSQL. 117 | - `NULL` - allows to store NULL values. Same as Python `None` type. 118 | - other types. 119 | 120 | #### To create a table you need to: 121 | 122 | 1. Write `CREATE TABLE` command. 123 | 2. Then you state the name of the table. 124 | 3. Then you open the parenthesis and list all columns: 125 | 1. Column name. 126 | 2. Column type. 127 | 3. Constraints. 128 | 4. Then you close the parenthesis. 129 | 130 | #### Example 131 | 132 | Let's create a table called `users` for storing data about registered telegram users. We will use `BIGINT` 133 | for `telegram_id` instead of `INTEGER` because it has to be 64bit. 134 | 135 | ```postgresql 136 | CREATE TABLE users 137 | ( 138 | telegram_id BIGINT PRIMARY KEY, 139 | full_name VARCHAR, 140 | created_at TIMESTAMP DEFAULT NOW() 141 | ); 142 | ``` 143 | 144 | ##### Response 145 | 146 | ``` 147 | CREATE TABLE 148 | ``` 149 | 150 | ## Altering Tables 151 | 152 | If you want to edit table structure you can use `ALTER TABLE` command. 153 | You can add, remove or rename columns, or change their data types. 154 | 155 | > Syntax: 156 | > 157 | > - `ALTER TABLE table_name ADD column_name data_type;` 158 | > - `ALTER TABLE table_name DROP column_name;` 159 | > - `ALTER TABLE table_name RENAME column_name TO new_column_name;` 160 | > - `ALTER TABLE table_name ALTER column_name SET DATA TYPE data_type;` 161 | > - `ALTER TABLE table_name ALTER column_name SET NOT NULL;` 162 | > - ... 163 | 164 | ## Dropping Tables 165 | 166 | If you want you can delete the entire table with all data. 167 | 168 | > Syntax: `DROP TABLE table_name;` 169 | 170 | #### Example 171 | 172 | ```postgresql 173 | DROP TABLE users; 174 | ``` 175 | 176 | ## Inserting Data 177 | 178 | > Syntax: `INSERT INTO table_name (column_name, column_name, ...) VALUES (value, value, ...);` 179 | 180 | #### To insert data into a table you need to: 181 | 182 | 1. Write `INSERT INTO` command. 183 | 2. Then you state the name of the table. 184 | 3. Then you list all columns names inside the parenthesis. 185 | 4. Write `VALUES`. 186 | 5. Then you list all values you want to insert inside the parenthesis. 187 | 188 | > **Note:** 189 | > 190 | > - You must enter all values **in the same order** as you listed columns. 191 | > 192 | > - **You can skip filling columns** that will be filled with default (or null) values. 193 | 194 | > You must use specified datatypes for each value: 195 | > 196 | >- For example, if you want to insert a `INTEGER` value, you must write 197 | > `1` instead of `1.0` 198 | > 199 | > - If you want to insert a `VARCHAR/TEXT` value, you must put a string inside single quotes. 200 | > 201 | >- If you want to insert a `TIMESTAMP` value, you must use the following format: 202 | > `'YYYY-MM-DD HH:MM:SS'`, where `YYYY` is year, `MM` is month, `DD` is day, `HH` is hour, `MM` is minute, `SS` is 203 | second. 204 | 205 | #### Example 206 | 207 | Let's insert one user into the table with name 'John Doe'. 208 | 209 | ```postgresql 210 | INSERT INTO users (telegram_id, full_name, created_at) 211 | VALUES (123456789, 'John Doe', '2020-01-01 00:00:00'); 212 | ``` 213 | 214 | ##### Response 215 | 216 | ``` 217 | INSERT 0 1 218 | ``` 219 | 220 | ## Selecting Data 221 | 222 | > Syntax: `SELECT column_name, column_name, ... FROM table_name WHERE condition;` 223 | 224 | #### To select data from a table you need to: 225 | 226 | 1. Write `SELECT` command. 227 | 2. Then you write the name of the column (or several columns separated by commas) you want to select. 228 | 3. Then you write `FROM` command. 229 | 4. Then you write the name of the table. 230 | 231 | > **Filtering data** 232 | > 233 | > If you want to filter data you can use `WHERE` command and write a condition. 234 | > 235 | > Condition is a piece of code that checks if the data is true or false. If the condition is true, the data is selected. 236 | > 237 | > Example: `WHERE telegram_id = 123456789`, where `telegram_id` is the name of the column. 238 | 239 | 240 | > **Note:** 241 | > 242 | > You can select all columns from a table by writing `*`, 243 | > 244 | but [this is not recommended.](https://www.red-gate.com/hub/product-learning/sql-prompt/finding-code-smells-using-sql-prompt-asterisk-select-list) 245 | 246 | #### Example 1: 247 | 248 | Let's select all users from the table. 249 | 250 | ```postgresql 251 | SELECT telegram_id 252 | FROM users; 253 | ``` 254 | 255 | ##### Response: 256 | 257 | ``` 258 | telegram_id 259 | ------------- 260 | 123456789 261 | ``` 262 | 263 | #### Example 2: 264 | 265 | Let's select all users from the table where their telegram_id is 123456789. 266 | 267 | ```postgresql 268 | SELECT * 269 | FROM users 270 | WHERE telegram_id = 123456789; 271 | ``` 272 | 273 | ##### Response: 274 | 275 | ``` 276 | telegram_id | full_name | created_at 277 | -------------+-----------+--------------------- 278 | 123456789 | John Doe | 2020-01-01 00:00:00 279 | ``` 280 | 281 | ## Updating Data 282 | 283 | > Syntax: `UPDATE table_name SET column1 = value1, column2 = value2, ... WHERE condition;` 284 | 285 | #### To update data in a table you need to: 286 | 287 | 1. Write `UPDATE` command. 288 | 2. Then you write the name of the table. 289 | 3. Then you write `SET` command. 290 | 4. Then you iterate through the names of the columns and the values you want to set. _ 291 | Example: `telegram_id = 987654321`_ 292 | 5. Then you write condition if you want to filter data. _Example: `WHERE full_name = 'John Doe'`_ 293 | 294 | #### Example 295 | 296 | Let's update the full name of the user with telegram_id `123456789` to `John Moe`. 297 | 298 | ```postgresql 299 | UPDATE users 300 | SET full_name = 'John Moe' 301 | WHERE telegram_id = 123456789; 302 | ``` 303 | 304 | ##### Response 305 | 306 | ``` 307 | UPDATE 1 308 | ``` 309 | 310 | ## Deleting Data 311 | 312 | > Syntax: `DELETE FROM table_name WHERE condition;` 313 | 314 | #### To delete data from a table you need to: 315 | 316 | 1. Write `DELETE FROM` command. 317 | 2. Then you write the name of the table. 318 | 3. Then you write condition if you want to filter data. _Example: `WHERE full_name = 'John Doe'`_ 319 | 320 | #### Example 321 | 322 | Let's delete the user with telegram_id `123456789`. 323 | 324 | ```postgresql 325 | DELETE 326 | FROM users 327 | WHERE telegram_id = 123456789; 328 | ``` 329 | 330 | ##### Response 331 | 332 | ``` 333 | DELETE 1 334 | ``` 335 | -------------------------------------------------------------------------------- /Part2-Basics/commands.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE users 2 | ( 3 | telegram_id BIGINT PRIMARY KEY, 4 | full_name VARCHAR, 5 | created_at TIMESTAMP DEFAULT NOW() 6 | ); 7 | 8 | INSERT INTO users (telegram_id, full_name, created_at) 9 | VALUES (123456789, 'John Doe', '2020-01-01 00:00:00'); 10 | 11 | SELECT telegram_id 12 | FROM users; 13 | 14 | SELECT * 15 | FROM users 16 | WHERE telegram_id = 123456789; 17 | 18 | UPDATE users 19 | SET full_name = 'John Moe' 20 | WHERE telegram_id = 123456789; 21 | 22 | DELETE 23 | FROM users 24 | WHERE telegram_id = 123456789; -------------------------------------------------------------------------------- /Part3-Advanced data selection/Tutorial-2.MD: -------------------------------------------------------------------------------- 1 | ### For full info check this tutorial: [Tutorial](https://www.postgresqltutorial.com/) 2 | 3 | This is not a full tutorial, it's just a brief introduction to PostgreSQL and a script for video-tutorial. 4 | 5 | # Part 2. Advanced data selection. 6 | 7 | ## Preparing the data. 8 | 9 | ```postgresql 10 | INSERT INTO users (telegram_id, full_name, created_at) 11 | VALUES (1, 'John Doe', '2022-06-19 04:24:16.775551'), 12 | (2, 'Jane Doe', '2022-06-19 04:24:16.775551'), 13 | (3, 'John Smith', '2022-06-19 04:24:16.775551'), 14 | (4, 'Jane Smith', '2022-06-19 04:24:16.775551'); 15 | ``` 16 | 17 | ## Ordering Data 18 | 19 | You can sort data in PostgreSQL using the `ORDER BY` clause with `SELECT` statement. 20 | 21 | > Syntax: `SELECT column_name, ... FROM table_name ORDER BY column_name;` 22 | 23 | > **Note:** 24 | > 25 | > - You can sort by multiple columns by separating them with commas. It will sort by the first column first, then the 26 | second column (if the first one has duplicates), and so on. 27 | > - You can also sort in descending or ascending order by adding `DESC` or `ASC` after the column name. 28 | 29 | #### Example 30 | 31 | Let's select all records from the `users` table and sort them by their `telegram_id` column in descending order. 32 | 33 | ```postgresql 34 | SELECT * 35 | FROM users 36 | ORDER BY telegram_id DESC; 37 | ``` 38 | 39 | ##### Response 40 | 41 | ``` 42 | telegram_id | full_name | created_at 43 | -------------+------------+---------------------------- 44 | 4 | Jane Smith | 2022-06-19 04:24:16.775551 45 | 3 | John Smith | 2022-06-19 04:24:16.775551 46 | 2 | Jane Doe | 2022-06-19 04:24:16.775551 47 | 1 | John Doe | 2022-06-19 04:24:16.775551 48 | ``` 49 | 50 | ## Aggregating Data 51 | 52 | You can use different mathematical functions to aggregate data in PostgreSQL: 53 | 54 | - `COUNT`: Counts the number of rows in a table. 55 | - `SUM`: Sums the values of a column. 56 | - `AVG`: Calculates the average value of a column. 57 | - `MIN`: Finds the minimum value of a column. 58 | - `MAX`: Finds the maximum value of a column. 59 | - ... 60 | 61 | > Syntax: `SELECT function(column_name) FROM table_name;` 62 | 63 | ## Grouping Data 64 | 65 | You can apply aggregation functions to columns and divide the result into groups by using the `GROUP BY` clause 66 | with `SELECT` statement. 67 | 68 | > Syntax: `SELECT column_to_divide_by, aggregate_func(column_to_aggreage) FROM table_name GROUP BY column_to_divide_by;` 69 | 70 | > **Note:** 71 | > 72 | > It is not necessary to use aggregation functions with `GROUP BY`, it will group and remove the 73 | > duplicates from the result. But you will need to list all columns in the `GROUP BY` clause. 74 | 75 | For this example we will add another column to the table `users` called `role` 76 | and set it to `admin` for the first user, with the default value `user` for the rest: 77 | 78 | ```postgresql 79 | ALTER TABLE users 80 | ADD COLUMN role varchar(20) default 'user'; 81 | 82 | UPDATE users 83 | SET role = 'admin' 84 | WHERE telegram_id = 1; 85 | ``` 86 | 87 | #### Example 1 88 | 89 | Let's count the users by their role: 90 | 91 | ```postgresql 92 | SELECT role, COUNT(telegram_id) 93 | FROM users 94 | GROUP BY role; 95 | ``` 96 | 97 | ##### Response 98 | 99 | ``` 100 | role | count 101 | -------+------- 102 | admin | 1 103 | user | 3 104 | ``` 105 | 106 | #### Example 2 107 | 108 | You can also sort by aggregated columns: 109 | 110 | ```postgresql 111 | SELECT role, COUNT(telegram_id) 112 | FROM users 113 | GROUP BY role 114 | ORDER BY COUNT(telegram_id) DESC; 115 | ``` 116 | 117 | ##### Response 118 | 119 | ``` 120 | role | count 121 | -------+------- 122 | user | 3 123 | admin | 1 124 | ``` 125 | 126 | ## Advanced filtering 127 | 128 | - You can filter data by multiple boolean conditions if you need to. 129 | Just join them with `AND` or `OR` operators. 130 | - You can also use `NOT` operator to negate a condition. 131 | - You can use `IN` operator to filter by a list of values. 132 | - You can use `LIKE`/`ILIKE` operator to filter by a pattern (`LIKE` is case-sensitive). Use `%` to match any 133 | characters, `_` to match a single character. 134 | - You can use `BETWEEN` operator to filter by a range of values. 135 | - You can use `IS NULL` and `IS NOT NULL` operators to filter by null values. 136 | 137 | > Syntax: `SELECT * FROM table_name WHERE (condition1 AND condition2) OR (condition3 AND condition4);` 138 | 139 | #### Example 1. 140 | 141 | Filtering by a list of values 142 | 143 | ```postgresql 144 | SELECT telegram_id, full_name 145 | FROM users 146 | WHERE telegram_id IN (1, 3); 147 | ``` 148 | 149 | ##### Response 150 | 151 | ``` 152 | telegram_id | full_name 153 | -------------+------------ 154 | 1 | John Doe 155 | 3 | John Smith 156 | ``` 157 | 158 | #### Example 2. 159 | 160 | Filtering by a pattern. Let's filter by the name `John` but any case: 161 | 162 | ```postgresql 163 | SELECT telegram_id, full_name 164 | FROM users 165 | WHERE full_name ILIKE 'john%'; 166 | ``` 167 | 168 | ##### Response 169 | 170 | ``` 171 | telegram_id | full_name 172 | -------------+------------ 173 | 3 | John Smith 174 | 1 | John Doe 175 | (2 rows) 176 | ``` 177 | 178 | #### Example 3. 179 | 180 | Let's use previous example but add all 'Jane' users; 181 | 182 | ```postgresql 183 | SELECT telegram_id, full_name 184 | FROM users 185 | WHERE full_name ILIKE 'john%' 186 | OR full_name ILIKE 'jane%'; 187 | ``` 188 | 189 | ##### Response 190 | 191 | ``` 192 | telegram_id | full_name 193 | -------------+------------ 194 | 3 | John Smith 195 | 4 | Jane Smith 196 | 1 | John Doe 197 | 2 | Jane Doe 198 | (4 rows) 199 | ``` 200 | 201 | #### Example 4. 202 | 203 | Let's use previous example but remove all users that have 'doe' in their name. 204 | To do this, we need to put both `AND` **conditions** in parentheses: 205 | 206 | ```postgresql 207 | SELECT telegram_id, full_name 208 | FROM users 209 | WHERE (full_name ILIKE 'john%' 210 | OR 211 | full_name ILIKE 'jane%') 212 | AND NOT full_name ILIKE '%doe%'; 213 | ``` 214 | 215 | ##### Response 216 | 217 | ``` 218 | telegram_id | full_name 219 | -------------+------------ 220 | 3 | John Smith 221 | 4 | Jane Smith 222 | (2 rows) 223 | ``` 224 | 225 | #### Example 5. 226 | 227 | IF you don't put parentheses around the `AND` conditions, PostgreSQL will interpret them as a single condition with the 228 | previous condition. 229 | 230 | ```postgresql 231 | SELECT telegram_id, full_name 232 | FROM users 233 | WHERE full_name ILIKE 'john%' 234 | OR full_name ILIKE 'jane%' 235 | AND NOT full_name ILIKE '%doe%'; 236 | ``` 237 | 238 | ##### Response 239 | 240 | Now 'doe' is excluded only with condition `ILIKE 'jane%'` condition. 241 | 242 | ``` 243 | telegram_id | full_name 244 | -------------+------------ 245 | 3 | John Smith 246 | 4 | Jane Smith 247 | 1 | John Doe 248 | (3 rows) 249 | ``` 250 | 251 | ## Distinct values 252 | 253 | You can use the `DISTINCT` clause select different values from a column. 254 | 255 | > Syntax: `SELECT DISTINCT column_name FROM table_name;` 256 | 257 | #### Example 1 258 | 259 | Let's first see what roles we have: 260 | 261 | ```postgresql 262 | SELECT role 263 | FROM users; 264 | ``` 265 | 266 | ##### Response 267 | 268 | You can see that there are duplicate values in the `role` column, because it shows roles for all users. 269 | 270 | ``` 271 | role 272 | ------- 273 | user 274 | user 275 | admin 276 | user 277 | (4 rows) 278 | ``` 279 | 280 | #### Example 2 281 | 282 | Let's remove the duplicates: 283 | 284 | ```postgresql 285 | SELECT DISTINCT role 286 | FROM users; 287 | ``` 288 | 289 | ##### Response 290 | 291 | ``` 292 | role 293 | ------- 294 | admin 295 | user 296 | (2 rows) 297 | ``` 298 | 299 | ## Aliasing outputs 300 | 301 | You can use the `AS` clause to set a specific name for selected data. 302 | This is useful if you want to get an output with a different name than the column name or give a name to an aggregated 303 | value. 304 | 305 | > Syntax: `SELECT some_func(column_name1) AS alias_name, column_name2 FROM table_name;` 306 | 307 | #### Example 1 308 | 309 | Let's get the number of users by their role: 310 | 311 | ```postgresql 312 | SELECT role, COUNT(telegram_id) AS number_of_users 313 | FROM users 314 | GROUP BY role; 315 | ``` 316 | 317 | ##### Response 318 | 319 | ``` 320 | role | number_of_users 321 | -------+----------------- 322 | admin | 1 323 | user | 3 324 | 325 | ``` 326 | 327 | Well done! You learned how to use Advanced Selection and Filtering. -------------------------------------------------------------------------------- /Part3-Advanced data selection/commands.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO users (telegram_id, full_name, created_at) 2 | VALUES (1, 'John Doe', '2022-06-19 04:24:16.775551'), 3 | (2, 'Jane Doe', '2022-06-19 04:24:16.775551'), 4 | (3, 'John Smith', '2022-06-19 04:24:16.775551'), 5 | (4, 'Jane Smith', '2022-06-19 04:24:16.775551'); 6 | 7 | SELECT * 8 | FROM users 9 | ORDER BY telegram_id DESC; 10 | 11 | ALTER TABLE users 12 | ADD COLUMN role varchar(20) default 'user'; 13 | 14 | UPDATE users 15 | SET role = 'admin' 16 | WHERE telegram_id = 1; 17 | 18 | SELECT role, COUNT(telegram_id) 19 | FROM users 20 | GROUP BY role; 21 | 22 | SELECT role, COUNT(telegram_id) 23 | FROM users 24 | GROUP BY role 25 | ORDER BY COUNT(telegram_id) DESC; 26 | 27 | SELECT telegram_id, full_name 28 | FROM users 29 | WHERE telegram_id IN (1, 3); 30 | 31 | SELECT telegram_id, full_name 32 | FROM users 33 | WHERE full_name ILIKE 'john%'; 34 | 35 | 36 | SELECT telegram_id, full_name 37 | FROM users 38 | WHERE full_name ILIKE 'john%' 39 | OR full_name ILIKE 'jane%'; 40 | 41 | 42 | SELECT telegram_id, full_name 43 | FROM users 44 | WHERE (full_name ILIKE 'john%' 45 | OR 46 | full_name ILIKE 'jane%') 47 | AND NOT full_name ILIKE '%doe%'; 48 | 49 | SELECT telegram_id, full_name 50 | FROM users 51 | WHERE full_name ILIKE 'john%' 52 | OR full_name ILIKE 'jane%' 53 | AND NOT full_name ILIKE '%doe%'; 54 | 55 | SELECT role 56 | FROM users; 57 | 58 | SELECT DISTINCT role 59 | FROM users; 60 | 61 | SELECT role, COUNT(telegram_id) AS number_of_users 62 | FROM users 63 | GROUP BY role; 64 | -------------------------------------------------------------------------------- /Part4-Relationships and JOINS/Tutorial-3.MD: -------------------------------------------------------------------------------- 1 | ### For full info check this tutorial: [Tutorial](https://www.postgresql.org/docs/current/tutorial-fk.html) 2 | 3 | This is not a full tutorial, it's just a brief introduction to PostgreSQL and a script for video-tutorial. 4 | 5 | # Part 3. Relationships and JOINS 6 | 7 | ## Relationships 8 | 9 | At some point you will need to store related entities in the database. 10 | These may include: 11 | 12 | - Users that may somehow relate to some other users (Referrals, ...); 13 | - Orders that owned by (or related to) some users; 14 | - Different related items that orders contain; 15 | 16 | These are better represented by the diagram below: 17 | ![img.png](img.png) 18 | 19 | > **Note:** 20 | > 21 | > Draw a diagraw **every time** when you need to create a new database. 22 | > 23 | > You can use specific tools to create diagrams (e.g. DBDesigner). 24 | 25 | Here we created 4 interconnected tables: 26 | 27 | **users:** 28 | 29 | - `telegram_id` 30 | - `full_name` 31 | - `username` 32 | - `language_code` 33 | - `created_at` 34 | - `referrer_id` 35 | 36 | **orders:** 37 | 38 | - `order_id` 39 | - `user_id` 40 | - `created_at` 41 | 42 | **products:** 43 | 44 | - `product_id` 45 | - `title` 46 | - `description` 47 | - `created_at` 48 | 49 | **order_products:** 50 | 51 | - `order_id` 52 | - `product_id` 53 | - `quantity` 54 | 55 | Why do we need to create so many tables? 56 | Because we need to follow specific rules for the relationship between the tables. 57 | 58 | There are 4 types of relationships: 59 | 60 | - One-to-one 61 | - One-to-many 62 | - Many-to-many 63 | - Self-referencing 64 | 65 | ### One-to-one 66 | 67 | One record in one table is related to one record in another table. 68 | 69 | > Note: **No duplicates**. 70 | 71 | #### Example (not in our diagram): 72 | Here we have 3 users and each user has a unique password. 73 | 74 | ``` 75 | |------passwords-----| 76 | | password | user_id | 77 | |----------|---------| 78 | | 123456 | 1 | 79 | | 123321 | 2 | 80 | | 483831 | 3 | 81 | |----------|---------| 82 | ``` 83 | 84 | ### One-to-many 85 | 86 | One user can have many orders. But one order can belong **only to one user**. This is why we have 87 | a `user_id` column in the `orders` table. We are making each unique order and assigning it to a specific user. There 88 | cannot be two orders with the same `order_id`. 89 | 90 | > Note: **Duplicates allowed in the `user_id` column**. 91 | 92 | #### Example: 93 | Here we have a user with the id `1` and he has 3 different orders. 94 | 95 | ``` 96 | |--------orders------| 97 | | order_id | user_id | 98 | |----------|---------| 99 | | 1 | 1 | 100 | | 2 | 1 | 101 | | 3 | 1 | 102 | |----------|---------| 103 | ``` 104 | 105 | ### Many-to-many 106 | 107 | Many orders can contain many products and vice versa: any product can be repeated in many orders. This is why 108 | we have a `order_id` and `product_id` columns in separate `order_products` table. We created a separate third table to 109 | allow creating intercrossing links between two tables `orders` and `products`. 110 | 111 | > Note: **Duplicates allowed in both columns**. 112 | 113 | #### Example: 114 | Here we have 3 orders with 1 product in 1st, 3 products in 2nd order and 2 products in 3rd order. 115 | 116 | ``` 117 | |-----order_products----| 118 | | order_id | product_id | 119 | |----------|------------| 120 | | 1 | 1 | 121 | | 2 | 1 | 122 | | 2 | 2 | 123 | | 2 | 3 | 124 | | 3 | 1 | 125 | | 3 | 2 | 126 | |----------|------------| 127 | ``` 128 | 129 | ## Foreign keys and Joining tables 130 | 131 | Now we need to create these tables with the correct relationships. We need to create a `FOREIGN KEY` column for each 132 | table **that references the primary key of the other** table. 133 | 134 | > **Note:** 135 | > 136 | > To remind you, the primary key is the column that automatically sets the 137 | `NOT NULL` and `UNIQUE` constraints and is used to identify each record. 138 | 139 | ``` 140 | [CONSTRAINT fk_name] 141 | FOREIGN KEY(fk_columns) 142 | REFERENCES parent_table(parent_key_columns) 143 | [ON DELETE delete_action] 144 | [ON UPDATE update_action] 145 | ``` 146 | 147 | ### Table users: 148 | 149 | First, if you have created `users` table from the previous lessons you will need to delete it. 150 | 151 | ```postgresql 152 | DROP TABLE IF EXISTS users; 153 | ``` 154 | 155 | Now, we will create a new table with the following columns: 156 | 157 | ```postgresql 158 | CREATE TABLE users 159 | ( 160 | telegram_id BIGINT PRIMARY KEY, 161 | full_name VARCHAR(255) NOT NULL, 162 | username VARCHAR(255), 163 | language_code VARCHAR(255) NOT NULL, 164 | created_at TIMESTAMP DEFAULT NOW(), 165 | referrer_id BIGINT, 166 | FOREIGN KEY (referrer_id) 167 | REFERENCES users (telegram_id) 168 | ON DELETE SET NULL 169 | ); 170 | ``` 171 | 172 | You can see that we have a `referrer_id` column and created a `FOREIGN KEY` for it that references the `telegram_id` 173 | column of the `users` table. It is Self-referencing relationship. 174 | 175 | > **Note:** 176 | > 177 | > A setting `REFERENCES users (telegram_id)` makes the postgresql to check if that specified `telegram_id` exists in 178 | > the `users` table. A command with arbitrary `telegram_id` will be rejected. 179 | > 180 | > Also, we have a `ON DELETE SET NULL` setting that automatically sets the `referrer_id` column to `NULL` if the user 181 | > with the specified `telegram_id` is deleted by other commands. 182 | 183 | > There are other options for the `ON DELETE` setting, but we will not use them all in this tutorial. 184 | > - `ON DELETE CASCADE` - Will delete the user record if the user with the specified referred record is deleted. 185 | > - `ON DELETE RESTRICT` - Will reject the command to delete **the referrer record**. 186 | > - `ON DELETE NO ACTION` - Will do nothing. 187 | > - `ON DELETE SET DEFAULT` - Will set the `referrer_id` column to the default value if defined. 188 | 189 | #### Example: 190 | 191 | Let's create a user 192 | 193 | ```postgresql 194 | INSERT INTO users 195 | (telegram_id, full_name, username, language_code, created_at) 196 | VALUES (1, 'John Doe', 'johnny', 'en', '2020-01-01'); 197 | ``` 198 | 199 | Now we'll add a referral to the user. We will specify the `referrer_id` as `1`: 200 | 201 | ```postgresql 202 | INSERT INTO users 203 | (telegram_id, full_name, username, language_code, created_at, referrer_id) 204 | VALUES (2, 'Jane Doe', 'jane', 'en', '2020-01-02', 1); 205 | ``` 206 | 207 | Now we can join tables to get the second user and his referrer. 208 | 209 | > You can learn about different types of joins in other tutorials. We won't cover them here. 210 | 211 | > Syntax: `SELECT table1.column1, ... FROM table1 JOIN table2 ON table2.primary_key = table1.foreign_key` 212 | 213 | So we still need to specify the columns that are connecting the tables: 214 | 215 | ```ON table2.primary_key = table1.foreign_key``` 216 | 217 | Let's see each user's name and his referrer's name. Since we are connecting 'users' table with itself we also need to 218 | specify the alias for the second table. We will call it 'ref'. 219 | 220 | Let's specify what we mean then saying that we want to connect tables. We want to get the result of 221 | the `SELECT users.full_name` statement, but also add to each row the respective `ref.full_name` column for the user that 222 | has the same `telegram_id` as the `users.referrer_id` column. 223 | 224 | ```postgresql 225 | SELECT users.full_name AS "user", ref.full_name AS referrer 226 | FROM users 227 | LEFT OUTER JOIN users AS ref ON ref.telegram_id = users.referrer_id; 228 | ``` 229 | 230 | ##### Response: 231 | 232 | ``` 233 | user | referrer 234 | ----------+---------- 235 | Jane Doe | John Doe 236 | 237 | (1 row) 238 | ``` 239 | 240 | ##### Explanation: 241 | This type of join is called `INNER JOIN`, it is standard when you write `JOIN`, but there are many other types of joins. 242 | 243 | ### Types of joins 244 | 245 | By inclusion of data: 246 | 247 | - `INNER JOIN` - Standard. Will only return rows that have a match in both tables by `ON` clause. 248 | - `OUTER JOIN` - Will return all rows from one of the tables, even if they don't have a match with the 249 | second table. See difference below. 250 | - `FULL JOIN` - Will return rows from both tables, even if they don't have a match. 251 | 252 | By statement side: 253 | 254 | - `LEFT (OUTER) JOIN` - Will return all rows written in `FROM` statement (_left-side_). If there is no match in 255 | the right table, the columns will be filled with `NULL`. 256 | - `RIGHT (OUTER) JOIN` - Will return all rows written in `JOIN` statement (_right-side_). If there is no match 257 | in the left table, the columns will be filled with `NULL`. 258 | 259 | #### Syntax examples: 260 | 261 | ```postgresql 262 | SELECT table1.column1, ..., table2.column1, ... 263 | FROM table1 264 | LEFT JOIN table2 265 | ON table2.primary_key = table1.foreign_key; 266 | 267 | SELECT table1.column1, ..., table2.column1, ... 268 | FROM table1 269 | RIGHT OUTER JOIN table2 270 | ON table2.primary_key = table1.foreign_key; 271 | 272 | SELECT table1.column1, ..., table2.column1, ... 273 | FROM table1 274 | FULL OUTER JOIN table2 275 | ON table2.primary_key = table1.foreign_key; 276 | 277 | SELECT table1.column1, ..., table2.column1, ... 278 | FROM table1 279 | JOIN table2 280 | ON table2.primary_key = table1.foreign_key; 281 | ``` 282 | 283 | #### Difference between `INNER JOIN` and `OUTER JOIN` 284 | 285 | Let's repeat the same query but with `OUTER JOIN` instead of `INNER JOIN`. 286 | 287 | ```postgresql 288 | SELECT users.full_name AS "user", ref.full_name AS referrer 289 | FROM users 290 | LEFT OUTER JOIN users AS ref ON ref.telegram_id = users.referrer_id; 291 | ``` 292 | 293 | ##### Response: 294 | 295 | ``` 296 | user | referrer 297 | ----------+---------- 298 | John Doe | 299 | Jane Doe | John Doe 300 | 301 | (2 rows) 302 | ``` 303 | 304 | ##### Explanation: 305 | You can see that the referrer atribute is missing for the user John Doe, because he doesn't have a matching referrer. 306 | 307 | #### Difference between `LEFT (OUTER) JOIN` and `RIGHT (OUTER) JOIN` 308 | 309 | Let's repeat the same query but with `LEFT (OUTER) JOIN` instead of `RIGHT (OUTER) JOIN`. 310 | 311 | ```postgresql 312 | SELECT users.full_name AS "user", ref.full_name AS referrer 313 | FROM users 314 | RIGHT JOIN users AS ref ON ref.telegram_id = users.referrer_id; 315 | ``` 316 | 317 | ##### Response: 318 | 319 | ``` 320 | user | referrer 321 | ----------+---------- 322 | Jane Doe | John Doe 323 | | Jane Doe 324 | 325 | (2 rows) 326 | ``` 327 | 328 | ##### Explanation: 329 | You can see that now the user (referral) atribute is missing for the user Jane Doe, because she doesn't have a 330 | matching **referral**. So we've got all records from the second table, but records for missing attributes in the first 331 | are filled with `NULL`. 332 | 333 | ### Table orders: 334 | 335 | We need to create a `FOREIGN KEY` for the `user_id` column in the `orders` table. 336 | 337 | However, now we specify `ON DELETE CASCADE` for the `user_id` column. 338 | This means that if we delete a user, all his orders will be deleted as well. 339 | 340 | ```postgresql 341 | CREATE TABLE orders 342 | ( 343 | order_id SERIAL PRIMARY KEY, 344 | user_id BIGINT NOT NULL, 345 | created_at TIMESTAMP DEFAULT NOW(), 346 | FOREIGN KEY (user_id) 347 | REFERENCES users (telegram_id) 348 | ON DELETE CASCADE 349 | ); 350 | ``` 351 | 352 | #### Example: 353 | 354 | Let's create an order for the user with the id `1`. We also add `RETURNING` setting to get the new order id: 355 | 356 | ```postgresql 357 | INSERT INTO orders (user_id) 358 | VALUES (1) 359 | RETURNING order_id; 360 | ``` 361 | 362 | ##### Response: 363 | 364 | ``` 365 | order_id 366 | ---------- 367 | 1 368 | ``` 369 | 370 | Now let's list all the orders and their owners. 371 | 372 | ```postgresql 373 | SELECT orders.order_id, users.full_name 374 | FROM orders 375 | JOIN users ON users.telegram_id = orders.user_id; 376 | ``` 377 | 378 | ##### Response: 379 | 380 | ``` 381 | order_id | full_name 382 | ----------+----------- 383 | 1 | John Doe 384 | 385 | (1 row) 386 | ``` 387 | 388 | Good! Now it's time to add some products to the order. 389 | 390 | ### Table products: 391 | 392 | First we will create a table for products. It will only contain the info about available products. 393 | 394 | ```postgresql 395 | CREATE TABLE products 396 | ( 397 | product_id SERIAL PRIMARY KEY, 398 | title VARCHAR(255) NOT NULL, 399 | description TEXT, 400 | created_at TIMESTAMP DEFAULT NOW() 401 | ); 402 | ``` 403 | 404 | Now we will need to create a Many-to-Many relationship between the `orders` and `products` tables. 405 | We will do this by creating a `order_products` table. 406 | 407 | ```postgresql 408 | CREATE TABLE order_products 409 | ( 410 | order_id INTEGER NOT NULL, 411 | product_id INTEGER NOT NULL, 412 | quantity INTEGER NOT NULL, 413 | FOREIGN KEY (order_id) 414 | REFERENCES orders (order_id) 415 | ON DELETE CASCADE, 416 | FOREIGN KEY (product_id) 417 | REFERENCES products (product_id) 418 | ON DELETE RESTRICT 419 | ); 420 | ``` 421 | 422 | You can see that we created two `FOREIGN KEY`'s. Each one references the primary key of the other table. 423 | 424 | > We also added `ON DELETE CASCADE` setting to delete the record if the order is deleted. 425 | 426 | > We also added `ON DELETE RESTRICT` setting to **reject the command to delete the product** if database contains orders 427 | > with this product. 428 | 429 | #### Example: 430 | 431 | Let's create a few products. 432 | 433 | ```postgresql 434 | INSERT INTO products (title, description) 435 | VALUES ('Product 1', 'Description 1'), 436 | ('Product 2', 'Description 2'), 437 | ('Product 3', 'Description 3'); 438 | ``` 439 | 440 | Let's add these products to the 1st order. 441 | 442 | ```postgresql 443 | INSERT INTO order_products (order_id, product_id, quantity) 444 | VALUES (1, 1, 1), 445 | (1, 2, 2), 446 | (1, 3, 3); 447 | ``` 448 | 449 | Now we need to join the tables to get the products and their owners for each order. 450 | 451 | ```postgresql 452 | SELECT orders.order_id, products.title as product_name, users.full_name 453 | FROM order_products 454 | JOIN products ON products.product_id = order_products.product_id 455 | JOIN orders ON orders.order_id = order_products.order_id 456 | JOIN users ON users.telegram_id = orders.user_id; 457 | ``` 458 | 459 | ##### Response: 460 | 461 | ``` 462 | order_id | product_name | full_name 463 | ----------+--------------+----------- 464 | 1 | Product 1 | John Doe 465 | 1 | Product 2 | John Doe 466 | 1 | Product 3 | John Doe 467 | 468 | (3 rows) 469 | ``` 470 | 471 | Now, having the orderd_id you can get all the products for that specific order by adding `WHERE` clause: 472 | 473 | ``` 474 | ... WHERE orders.order_id = 1; 475 | ``` 476 | 477 | ### Now let's test deletions. 478 | 479 | 1. Delete the product with the id `1`. 480 | 481 | ```postgresql 482 | DELETE 483 | FROM products 484 | WHERE product_id = 1; 485 | ``` 486 | 487 | Result: 488 | 489 | ``` 490 | ERROR: update or delete on table "products" violates foreign key constraint "order_products_product_id_fkey" on table "order_products" 491 | DETAIL: Key (product_id)=(1) is still referenced from table "order_products". 492 | ``` 493 | 494 | As you see, constraint `ON DELETE RESTRICT` prevents us from deleting the product. 495 | 496 | 2. Delete the user with the id `1`. 497 | 498 | ```postgresql 499 | DELETE 500 | FROM users 501 | WHERE telegram_id = 1; 502 | ``` 503 | 504 | Result: 505 | 506 | ``` 507 | DELETE 1 508 | ``` 509 | 510 | We successfully deleted the user, his/her `orders` and `order_products` records by just one command. 511 | 512 | ```postgresql 513 | select * 514 | from order_products; 515 | ``` 516 | 517 | ##### Response: 518 | 519 | ``` 520 | order_id | product_id | quantity 521 | ----------+------------+---------- 522 | 523 | (0 rows) 524 | ``` 525 | 526 | Well done! Now you know how to create related tables and how to extract data from them! -------------------------------------------------------------------------------- /Part4-Relationships and JOINS/commands.sql: -------------------------------------------------------------------------------- 1 | DROP TABLE IF EXISTS users; 2 | 3 | 4 | CREATE TABLE users 5 | ( 6 | telegram_id BIGINT PRIMARY KEY, 7 | full_name VARCHAR(255) NOT NULL, 8 | username VARCHAR(255), 9 | language_code VARCHAR(255) NOT NULL, 10 | created_at TIMESTAMP DEFAULT NOW(), 11 | referrer_id BIGINT, 12 | FOREIGN KEY (referrer_id) 13 | REFERENCES users (telegram_id) 14 | ON DELETE SET NULL 15 | ); 16 | 17 | 18 | INSERT INTO users 19 | (telegram_id, full_name, username, language_code, created_at) 20 | VALUES (1, 'John Doe', 'johnny', 'en', '2020-01-01'); 21 | 22 | 23 | INSERT INTO users 24 | (telegram_id, full_name, username, language_code, created_at, referrer_id) 25 | VALUES (2, 'Jane Doe', 'jane', 'en', '2020-01-02', 1); 26 | 27 | 28 | SELECT users.full_name AS "user", ref.full_name AS referrer 29 | FROM users 30 | JOIN users AS ref ON ref.telegram_id = users.referrer_id; 31 | 32 | 33 | SELECT users.full_name AS "user", ref.full_name AS referrer 34 | FROM users 35 | LEFT OUTER JOIN users AS ref ON ref.telegram_id = users.referrer_id; 36 | 37 | 38 | SELECT users.full_name AS "user", ref.full_name AS referrer 39 | FROM users 40 | RIGHT JOIN users AS ref ON ref.telegram_id = users.referrer_id; 41 | 42 | 43 | CREATE TABLE orders 44 | ( 45 | order_id SERIAL PRIMARY KEY, 46 | user_id BIGINT NOT NULL, 47 | created_at TIMESTAMP DEFAULT NOW(), 48 | FOREIGN KEY (user_id) 49 | REFERENCES users (telegram_id) 50 | ON DELETE CASCADE 51 | ); 52 | 53 | 54 | INSERT INTO orders (user_id) 55 | VALUES (1) 56 | RETURNING order_id; 57 | 58 | 59 | SELECT orders.order_id, users.full_name 60 | FROM orders 61 | JOIN users ON users.telegram_id = orders.user_id; 62 | 63 | 64 | CREATE TABLE products 65 | ( 66 | product_id SERIAL PRIMARY KEY, 67 | title VARCHAR(255) NOT NULL, 68 | description TEXT, 69 | created_at TIMESTAMP DEFAULT NOW() 70 | ); 71 | 72 | 73 | CREATE TABLE order_products 74 | ( 75 | order_id INTEGER NOT NULL, 76 | product_id INTEGER NOT NULL, 77 | quantity INTEGER NOT NULL, 78 | FOREIGN KEY (order_id) 79 | REFERENCES orders (order_id) 80 | ON DELETE CASCADE, 81 | FOREIGN KEY (product_id) 82 | REFERENCES products (product_id) 83 | ON DELETE RESTRICT 84 | ); 85 | 86 | 87 | INSERT INTO products (title, description) 88 | VALUES ('Product 1', 'Description 1'), 89 | ('Product 2', 'Description 2'), 90 | ('Product 3', 'Description 3'); 91 | 92 | 93 | INSERT INTO order_products (order_id, product_id, quantity) 94 | VALUES (1, 1, 1), 95 | (1, 2, 2), 96 | (1, 3, 3); 97 | 98 | 99 | SELECT orders.order_id, products.title as product_name, users.full_name 100 | FROM order_products 101 | JOIN products ON products.product_id = order_products.product_id 102 | JOIN orders ON orders.order_id = order_products.order_id 103 | JOIN users ON users.telegram_id = orders.user_id; 104 | 105 | 106 | DELETE 107 | FROM products 108 | WHERE product_id = 1; 109 | 110 | 111 | DELETE 112 | FROM users 113 | WHERE telegram_id = 1; 114 | 115 | 116 | select * 117 | from order_products; -------------------------------------------------------------------------------- /Part4-Relationships and JOINS/img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Latand/SQL-Tutorial/da30d7b54f21f8673076d88a656b30d765a5e158/Part4-Relationships and JOINS/img.png -------------------------------------------------------------------------------- /Part5-Subqueries and Ranks/Tutorial-4.MD: -------------------------------------------------------------------------------- 1 | # Part-4. Subqueries and Ranks 2 | 3 | Here we are going to quickly learn about subqueries. 4 | 5 | Sometimes you need to send more than one request to get the results you need. 6 | 7 | This is because you may need to use the result of the first request to get the results of the second request. 8 | Instead of sending two requests and processing them by your programming language, you can use a subquery, which will be 9 | faster. 10 | 11 | > **Note:** 12 | > 13 | >Also, if you do not know how to build a complicated query, you can too divide it into smaller parts and run as a 14 | > query + subquery. 15 | > 16 | > This may be useful if you want to avoid `JOIN`'s. 17 | 18 | For example: 19 | 20 | 1. Get some average result from a table (average ordered items) and filter values that are greater than that average. 21 | 2. Get rank of a specific user over all users by some column. (e.g. to show 'You are the #1 by number of orders!'). 22 | 3. Get some ids from one table and filter another query by them (to exclude/include only them). 23 | 24 | This may be sound very complicated, but we will show this with an example. 25 | 26 | ## Preparations 27 | 28 | Let's clean our database first, and recreate tables from scratch. We will also create some dummy data. 29 | You can just copy and paste the code below into your terminal. 30 | 31 | ```sql 32 | DROP TABLE IF EXISTS users, order_products, orders, products CASCADE; 33 | 34 | CREATE TABLE users 35 | ( 36 | telegram_id BIGINT PRIMARY KEY, 37 | full_name VARCHAR(255) NOT NULL, 38 | username VARCHAR(255), 39 | language_code VARCHAR(255) NOT NULL, 40 | created_at TIMESTAMP DEFAULT NOW(), 41 | referrer_id BIGINT, 42 | FOREIGN KEY (referrer_id) 43 | REFERENCES users (telegram_id) 44 | ON DELETE SET NULL 45 | ); 46 | 47 | CREATE TABLE orders 48 | ( 49 | order_id SERIAL PRIMARY KEY, 50 | user_id BIGINT NOT NULL, 51 | created_at TIMESTAMP DEFAULT NOW(), 52 | FOREIGN KEY (user_id) 53 | REFERENCES users (telegram_id) 54 | ON DELETE CASCADE 55 | ); 56 | 57 | CREATE TABLE products 58 | ( 59 | product_id SERIAL PRIMARY KEY, 60 | title VARCHAR(255) NOT NULL, 61 | description TEXT, 62 | created_at TIMESTAMP DEFAULT NOW() 63 | ); 64 | 65 | 66 | CREATE TABLE order_products 67 | ( 68 | order_id INTEGER NOT NULL, 69 | product_id INTEGER NOT NULL, 70 | quantity INTEGER NOT NULL, 71 | FOREIGN KEY (order_id) 72 | REFERENCES orders (order_id) 73 | ON DELETE CASCADE, 74 | FOREIGN KEY (product_id) 75 | REFERENCES products (product_id) 76 | ON DELETE RESTRICT 77 | ); 78 | INSERT INTO users 79 | (telegram_id, full_name, username, language_code, created_at) 80 | VALUES (1, 'Test User', 'test_user', 'en', NOW()), 81 | (2, 'Test User 2', 'test_user_2', 'en', NOW()), 82 | (3, 'Test User 3', 'test_user_3', 'en', NOW()), 83 | (4, 'Test User 4', 'test_user_4', 'en', NOW()), 84 | (5, 'Test User 5', 'test_user_5', 'en', NOW()), 85 | (6, 'Test User 6', 'test_user_6', 'en', NOW()), 86 | (7, 'Test User 7', 'test_user_7', 'en', NOW()), 87 | (8, 'Test User 8', 'test_user_8', 'en', NOW()); 88 | 89 | 90 | INSERT INTO orders (user_id, created_at) 91 | VALUES (1, '2022-02-01 00:00:00'), 92 | (2, '2022-03-01 00:00:00'), 93 | (3, '2022-04-01 00:00:00'), 94 | (4, '2022-05-01 00:00:00'), 95 | (6, '2022-02-15 00:00:00'), 96 | (8, '2022-04-16 00:00:00'); 97 | 98 | 99 | 100 | INSERT INTO products (title, description) 101 | VALUES ('Product 4', 'Description 4'), 102 | ('Product 5', 'Description 5'), 103 | ('Product 6', 'Description 6'), 104 | ('Product 7', 'Description 7'), 105 | ('Product 8', 'Description 8'), 106 | ('Product 9', 'Description 9'), 107 | ('Product 10', 'Description 10'), 108 | ('Product 11', 'Description 11'), 109 | ('Product 12', 'Description 12'), 110 | ('Product 13', 'Description 13'), 111 | ('Product 14', 'Description 14'), 112 | ('Product 15', 'Description 15'), 113 | ('Product 16', 'Description 16'), 114 | ('Product 17', 'Description 17'), 115 | ('Product 18', 'Description 18'), 116 | ('Product 19', 'Description 19'), 117 | ('Product 20', 'Description 20'), 118 | ('Product 21', 'Description 21'), 119 | ('Product 22', 'Description 22'), 120 | ('Product 23', 'Description 23'), 121 | ('Product 24', 'Description 24'), 122 | ('Product 25', 'Description 25'), 123 | ('Product 26', 'Description 26'), 124 | ('Product 27', 'Description 27'), 125 | ('Product 28', 'Description 28'), 126 | ('Product 29', 'Description 29'), 127 | ('Product 30', 'Description 30'), 128 | ('Product 31', 'Description 31'), 129 | ('Product 32', 'Description 32'), 130 | ('Product 33', 'Description 33'), 131 | ('Product 34', 'Description 34'), 132 | ('Product 35', 'Description 35'), 133 | ('Product 36', 'Description 36'), 134 | ('Product 37', 'Description 37'), 135 | ('Product 38', 'Description 38'), 136 | ('Product 39', 'Description 39'), 137 | ('Product 40', 'Description 40'), 138 | ('Product 41', 'Description 41'), 139 | ('Product 42', 'Description 42'), 140 | ('Product 43', 'Description 43'), 141 | ('Product 44', 'Description 44'), 142 | ('Product 45', 'Description 45'), 143 | ('Product 46', 'Description 46'), 144 | ('Product 47', 'Description 47'), 145 | ('Product 48', 'Description 48'), 146 | ('Product 49', 'Description 49'), 147 | ('Product 50', 'Description 50'), 148 | ('Product 51', 'Description 51'), 149 | ('Product 52', 'Description 52'), 150 | ('Product 53', 'Description 53'); 151 | 152 | 153 | 154 | INSERT INTO order_products (order_id, product_id, quantity) 155 | VALUES (1, 6, 2), 156 | (1, 8, 1), 157 | (1, 7, 1), 158 | (2, 8, 6), 159 | (2, 8, 1), 160 | (2, 9, 1), 161 | (3, 10, 1), 162 | (3, 11, 1), 163 | (3, 12, 1), 164 | (3, 13, 1), 165 | (3, 14, 1), 166 | (3, 15, 1), 167 | (3, 16, 1), 168 | (3, 17, 1), 169 | (4, 18, 1), 170 | (5, 19, 1), 171 | (6, 20, 1), 172 | (6, 21, 1), 173 | (6, 22, 1), 174 | (6, 23, 1), 175 | (6, 24, 1), 176 | (6, 25, 1), 177 | (6, 26, 1), 178 | (6, 27, 1), 179 | (6, 28, 1), 180 | (6, 29, 1), 181 | (6, 30, 1), 182 | (6, 31, 1), 183 | (6, 32, 1), 184 | (6, 33, 1), 185 | (6, 34, 1); 186 | ``` 187 | 188 | ## Examples 189 | 190 | ### Find products that haven't been sold yet. 191 | 192 | So you have a number interconnected of tables, and you want to determine which products are not so popular. 193 | Let's divide the task into 2 parts: 194 | 195 | 1. Find all id's of products that have been sold. 196 | 2. Select all products that are not in this list (i.e. exclude them from the result). 197 | 198 | ```sql 199 | SELECT DISTINCT(product_id) 200 | FROM order_products 201 | ORDER BY product_id; 202 | ``` 203 | 204 | #### Response 205 | 206 | ``` 207 | product_id 208 | ------------ 209 | 6 210 | 7 211 | 8 212 | 9 213 | 10 214 | 11 215 | 12 216 | 13 217 | 14 218 | 15 219 | 16 220 | 17 221 | 18 222 | 19 223 | 20 224 | 21 225 | 22 226 | 23 227 | 24 228 | 25 229 | 26 230 | 27 231 | 28 232 | 29 233 | 30 234 | 31 235 | 32 236 | 33 237 | 34 238 | (29 rows) 239 | ``` 240 | 241 | ```sql 242 | SELECT product_id, title, description 243 | FROM products 244 | WHERE product_id NOT IN ( 245 | 6, 246 | 7, 247 | 8, 248 | 9, 249 | 10, 250 | 11, 251 | 12, 252 | 13, 253 | 14, 254 | 15, 255 | 16, 256 | 17, 257 | 18, 258 | 19, 259 | 20, 260 | 21, 261 | 22, 262 | 23, 263 | 24, 264 | 25, 265 | 26, 266 | 27, 267 | 28, 268 | 29, 269 | 30, 270 | 31, 271 | 32, 272 | 33, 273 | 34); 274 | ``` 275 | 276 | ``` 277 | product_id | title | description 278 | ------------+------------+---------------- 279 | 1 | Product 4 | Description 4 280 | 2 | Product 5 | Description 5 281 | 3 | Product 6 | Description 6 282 | 4 | Product 7 | Description 7 283 | 5 | Product 8 | Description 8 284 | 35 | Product 38 | Description 38 285 | 36 | Product 39 | Description 39 286 | 37 | Product 40 | Description 40 287 | 38 | Product 41 | Description 41 288 | 39 | Product 42 | Description 42 289 | 40 | Product 43 | Description 43 290 | 41 | Product 44 | Description 44 291 | 42 | Product 45 | Description 45 292 | 43 | Product 46 | Description 46 293 | 44 | Product 47 | Description 47 294 | 45 | Product 48 | Description 48 295 | 46 | Product 49 | Description 49 296 | 47 | Product 50 | Description 50 297 | 48 | Product 51 | Description 51 298 | 49 | Product 52 | Description 52 299 | 50 | Product 53 | Description 53 300 | (21 rows) 301 | ``` 302 | 303 | Not very convenient, isn't it? Let's make a more efficient query. 304 | 305 | ```sql 306 | SELECT product_id, title, description 307 | FROM products 308 | WHERE product_id NOT IN (SELECT DISTINCT(product_id) 309 | FROM order_products 310 | ORDER BY product_id); 311 | ``` 312 | 313 | #### Explanation 314 | 315 | So we've just put the first query into the brackets, and put after `NOT IN` statement of the second query. 316 | 317 | ### Find users that haven't placed an order yet 318 | 319 | Let's now see who hasn't placed any orders yet. Maybe you'll want to send a message to these users. 320 | 321 | ```sql 322 | SELECT telegram_id, full_name, username 323 | FROM users 324 | WHERE telegram_id NOT IN (SELECT DISTINCT(user_id) 325 | FROM orders 326 | ORDER BY user_id); 327 | ``` 328 | 329 | #### Response 330 | 331 | ``` 332 | telegram_id | full_name | username 333 | -------------+-------------+------------- 334 | 5 | Test User 5 | test_user_5 335 | 7 | Test User 7 | test_user_7 336 | (2 рядки) 337 | ``` 338 | 339 | ### Top users' leaderboard by their order count 340 | 341 | Suppose you want to say something like "You are rank #1 in our community by placing orders". 342 | 343 | To do something like that you need to use `RANK() OVER (...)` function. 344 | 345 | Syntax: `RANK() OVER ([PARTITION BY partition_exp, .... ] ORDER BY sort_exp [ASC | DESC], ....)` 346 | 347 | > **Note:** 348 | > 349 | > `RANK()` can return repeated values for repeating input values. For example, if you have two rows with the same value 350 | > for `sort_exp`, then `RANK()` will return `1` for both rows. 351 | > If you want to avoid this, you can use `ROW_NUMBER()` instead. 352 | 353 | Let's first place a new order by the user with id `1`. 354 | 355 | ```sql 356 | INSERT INTO orders (user_id) 357 | VALUES (1); 358 | ``` 359 | 360 | Now let's find the rank **of all users**. Don't forget the `GROUP BY` clause when using aggregating functions. 361 | 362 | We will also show users that have not placed orders. To do this we will need to inclulde `NULL` values from the users 363 | table. In that case we need to use `LEFT (OUTER) JOIN` instead of simple `(INNER) JOIN`. 364 | 365 | ```sql 366 | SELECT RANK() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 367 | FROM users 368 | LEFT JOIN orders ON users.telegram_id = orders.user_id 369 | GROUP BY telegram_id, full_name, username; 370 | ``` 371 | 372 | ##### Response 373 | 374 | ``` 375 | rank | telegram_id | full_name | username | count 376 | ------+-------------+-------------+-------------+------- 377 | 1 | 1 | Test User | test_user | 2 378 | 2 | 6 | Test User 6 | test_user_6 | 1 379 | 2 | 3 | Test User 3 | test_user_3 | 1 380 | 2 | 8 | Test User 8 | test_user_8 | 1 381 | 2 | 2 | Test User 2 | test_user_2 | 1 382 | 2 | 4 | Test User 4 | test_user_4 | 1 383 | 7 | 7 | Test User 7 | test_user_7 | 0 384 | 7 | 5 | Test User 5 | test_user_5 | 0 385 | 386 | (8 rows) 387 | ``` 388 | 389 | ##### Explanation 390 | 391 | You can see that we have repeated ranks for users with the same order count. 392 | 393 | Also, the last 2 users ranks have `7` because there are 7 users before them and they both have equal order count. 394 | 395 | This is not a goodlooking leaderboard, we need to show ranks that increment by 1. Let's replace `RANK()` 396 | with `DENSE_RANK()`. 397 | 398 | ```sql 399 | SELECT DENSE_RANK() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 400 | FROM users 401 | LEFT JOIN orders ON users.telegram_id = orders.user_id 402 | GROUP BY telegram_id, full_name, username; 403 | ``` 404 | 405 | ##### Response 406 | 407 | ``` 408 | rank | telegram_id | full_name | username | count 409 | ------+-------------+-------------+-------------+------- 410 | 1 | 1 | Test User | test_user | 2 411 | 2 | 6 | Test User 6 | test_user_6 | 1 412 | 2 | 3 | Test User 3 | test_user_3 | 1 413 | 2 | 8 | Test User 8 | test_user_8 | 1 414 | 2 | 2 | Test User 2 | test_user_2 | 1 415 | 2 | 4 | Test User 4 | test_user_4 | 1 416 | 3 | 7 | Test User 7 | test_user_7 | 0 417 | 3 | 5 | Test User 5 | test_user_5 | 0 418 | 419 | (8 рядків) 420 | ``` 421 | 422 | Still not perfect, we still have repeated ranks, however, now they increment by 1. Let's use `ROW_NUMBER()` function to 423 | show the real leaderboard. 424 | 425 | ```postgresql 426 | SELECT ROW_NUMBER() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 427 | FROM users 428 | LEFT JOIN orders ON users.telegram_id = orders.user_id 429 | GROUP BY telegram_id, full_name, username; 430 | ``` 431 | 432 | ##### Response 433 | 434 | ``` 435 | rank | telegram_id | full_name | username | count 436 | ------+-------------+-------------+-------------+------- 437 | 1 | 1 | Test User | test_user | 2 438 | 2 | 6 | Test User 6 | test_user_6 | 1 439 | 3 | 3 | Test User 3 | test_user_3 | 1 440 | 4 | 8 | Test User 8 | test_user_8 | 1 441 | 5 | 2 | Test User 2 | test_user_2 | 1 442 | 6 | 4 | Test User 4 | test_user_4 | 1 443 | 7 | 7 | Test User 7 | test_user_7 | 0 444 | 8 | 5 | Test User 5 | test_user_5 | 0 445 | 446 | (8 rows) 447 | ``` 448 | 449 | ### User's real rank by their order count 450 | 451 | Suppose, you do not show a leaderboard for all users, but you only want to show the real rank of the user. 452 | It might not very fair to put `Test User 6` above `Test User 4` since they have the same order count. 453 | 454 | Thus, we will omit showing full leaderboard and show only the real rank. Now we put the query from the 1st 455 | example as a subquery and filter by user_id. 456 | 457 | ```sql 458 | SELECT r.rank, full_name, username, telegram_id, r.count 459 | FROM (SELECT RANK() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) as count 460 | FROM users 461 | LEFT JOIN orders ON users.telegram_id = orders.user_id 462 | GROUP BY telegram_id, full_name, username) as r 463 | WHERE telegram_id = 4; 464 | ``` 465 | 466 | #### Response 467 | 468 | ``` 469 | rank | full_name | username | telegram_id | count 470 | ------+-------------+-------------+-------------+------- 471 | 2 | Test User 4 | test_user_4 | 4 | 1 472 | 473 | (1 row) 474 | ``` 475 | 476 | ##### Explanation 477 | 478 | Now we put the subquery into a `FROM` statement, and we also gave a name (alias) for the resulting table `r`. 479 | 480 | Since we did that, we could get the attributes of the table `r` by using the alias (as Python attribute `r.attribute`), 481 | or just by column name. 482 | 483 | > **Note:** To get results from the aggregated functions from the subquery you must use the alias. 484 | 485 | Well done! You have learned how to use Subqueries and a little about `RANK()` functions. -------------------------------------------------------------------------------- /Part5-Subqueries and Ranks/commands.sql: -------------------------------------------------------------------------------- 1 | SELECT DISTINCT(product_id) 2 | FROM order_products 3 | ORDER BY product_id; 4 | 5 | 6 | SELECT product_id, title, description 7 | FROM products 8 | WHERE product_id NOT IN ( 9 | 6, 10 | 7, 11 | 8, 12 | 9, 13 | 10, 14 | 11, 15 | 12, 16 | 13, 17 | 14, 18 | 15, 19 | 16, 20 | 17, 21 | 18, 22 | 19, 23 | 20, 24 | 21, 25 | 22, 26 | 23, 27 | 24, 28 | 25, 29 | 26, 30 | 27, 31 | 28, 32 | 29, 33 | 30, 34 | 31, 35 | 32, 36 | 33, 37 | 34); 38 | 39 | 40 | SELECT product_id, title, description 41 | FROM products 42 | WHERE product_id NOT IN (SELECT DISTINCT(product_id) 43 | FROM order_products 44 | ORDER BY product_id); 45 | 46 | 47 | SELECT telegram_id, full_name, username 48 | FROM users 49 | WHERE telegram_id NOT IN (SELECT DISTINCT(user_id) 50 | FROM orders 51 | ORDER BY user_id); 52 | 53 | 54 | INSERT INTO orders (user_id) 55 | VALUES (1); 56 | 57 | SELECT RANK() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 58 | FROM users 59 | LEFT JOIN orders ON users.telegram_id = orders.user_id 60 | GROUP BY telegram_id, full_name, username; 61 | 62 | 63 | SELECT DENSE_RANK() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 64 | FROM users 65 | LEFT JOIN orders ON users.telegram_id = orders.user_id 66 | GROUP BY telegram_id, full_name, username; 67 | 68 | 69 | SELECT ROW_NUMBER() OVER (ORDER BY count(order_id) DESC) AS rank, telegram_id, full_name, username, count(order_id) 70 | FROM users 71 | LEFT JOIN orders ON users.telegram_id = orders.user_id 72 | GROUP BY telegram_id, full_name, username; 73 | 74 | 75 | SELECT r.rank, full_name, username, telegram_id, r.count 76 | FROM (SELECT RANK() OVER (ORDER BY count(order_id) DESC) AS rank, 77 | telegram_id, 78 | full_name, 79 | username, 80 | count(order_id) as count 81 | FROM users 82 | LEFT JOIN orders ON users.telegram_id = orders.user_id 83 | GROUP BY telegram_id, full_name, username) as r 84 | WHERE telegram_id = 4; 85 | 86 | -------------------------------------------------------------------------------- /Part5-Subqueries and Ranks/preparation.sql: -------------------------------------------------------------------------------- 1 | DROP TABLE IF EXISTS users, order_products, orders, products CASCADE; 2 | 3 | CREATE TABLE users 4 | ( 5 | telegram_id BIGINT PRIMARY KEY, 6 | full_name VARCHAR(255) NOT NULL, 7 | username VARCHAR(255), 8 | language_code VARCHAR(255) NOT NULL, 9 | created_at TIMESTAMP DEFAULT NOW(), 10 | referrer_id BIGINT, 11 | FOREIGN KEY (referrer_id) 12 | REFERENCES users (telegram_id) 13 | ON DELETE SET NULL 14 | ); 15 | 16 | CREATE TABLE orders 17 | ( 18 | order_id SERIAL PRIMARY KEY, 19 | user_id BIGINT NOT NULL, 20 | created_at TIMESTAMP DEFAULT NOW(), 21 | FOREIGN KEY (user_id) 22 | REFERENCES users (telegram_id) 23 | ON DELETE CASCADE 24 | ); 25 | 26 | CREATE TABLE products 27 | ( 28 | product_id SERIAL PRIMARY KEY, 29 | title VARCHAR(255) NOT NULL, 30 | description TEXT, 31 | created_at TIMESTAMP DEFAULT NOW() 32 | ); 33 | 34 | 35 | CREATE TABLE order_products 36 | ( 37 | order_id INTEGER NOT NULL, 38 | product_id INTEGER NOT NULL, 39 | quantity INTEGER NOT NULL, 40 | FOREIGN KEY (order_id) 41 | REFERENCES orders (order_id) 42 | ON DELETE CASCADE, 43 | FOREIGN KEY (product_id) 44 | REFERENCES products (product_id) 45 | ON DELETE RESTRICT 46 | ); 47 | INSERT INTO users 48 | (telegram_id, full_name, username, language_code, created_at) 49 | VALUES (1, 'Test User', 'test_user', 'en', NOW()), 50 | (2, 'Test User 2', 'test_user_2', 'en', NOW()), 51 | (3, 'Test User 3', 'test_user_3', 'en', NOW()), 52 | (4, 'Test User 4', 'test_user_4', 'en', NOW()), 53 | (5, 'Test User 5', 'test_user_5', 'en', NOW()), 54 | (6, 'Test User 6', 'test_user_6', 'en', NOW()), 55 | (7, 'Test User 7', 'test_user_7', 'en', NOW()), 56 | (8, 'Test User 8', 'test_user_8', 'en', NOW()); 57 | 58 | 59 | INSERT INTO orders (user_id, created_at) 60 | VALUES (1, '2022-02-01 00:00:00'), 61 | (2, '2022-03-01 00:00:00'), 62 | (3, '2022-04-01 00:00:00'), 63 | (4, '2022-05-01 00:00:00'), 64 | (6, '2022-02-15 00:00:00'), 65 | (8, '2022-04-16 00:00:00'); 66 | 67 | 68 | 69 | INSERT INTO products (title, description) 70 | VALUES ('Product 4', 'Description 4'), 71 | ('Product 5', 'Description 5'), 72 | ('Product 6', 'Description 6'), 73 | ('Product 7', 'Description 7'), 74 | ('Product 8', 'Description 8'), 75 | ('Product 9', 'Description 9'), 76 | ('Product 10', 'Description 10'), 77 | ('Product 11', 'Description 11'), 78 | ('Product 12', 'Description 12'), 79 | ('Product 13', 'Description 13'), 80 | ('Product 14', 'Description 14'), 81 | ('Product 15', 'Description 15'), 82 | ('Product 16', 'Description 16'), 83 | ('Product 17', 'Description 17'), 84 | ('Product 18', 'Description 18'), 85 | ('Product 19', 'Description 19'), 86 | ('Product 20', 'Description 20'), 87 | ('Product 21', 'Description 21'), 88 | ('Product 22', 'Description 22'), 89 | ('Product 23', 'Description 23'), 90 | ('Product 24', 'Description 24'), 91 | ('Product 25', 'Description 25'), 92 | ('Product 26', 'Description 26'), 93 | ('Product 27', 'Description 27'), 94 | ('Product 28', 'Description 28'), 95 | ('Product 29', 'Description 29'), 96 | ('Product 30', 'Description 30'), 97 | ('Product 31', 'Description 31'), 98 | ('Product 32', 'Description 32'), 99 | ('Product 33', 'Description 33'), 100 | ('Product 34', 'Description 34'), 101 | ('Product 35', 'Description 35'), 102 | ('Product 36', 'Description 36'), 103 | ('Product 37', 'Description 37'), 104 | ('Product 38', 'Description 38'), 105 | ('Product 39', 'Description 39'), 106 | ('Product 40', 'Description 40'), 107 | ('Product 41', 'Description 41'), 108 | ('Product 42', 'Description 42'), 109 | ('Product 43', 'Description 43'), 110 | ('Product 44', 'Description 44'), 111 | ('Product 45', 'Description 45'), 112 | ('Product 46', 'Description 46'), 113 | ('Product 47', 'Description 47'), 114 | ('Product 48', 'Description 48'), 115 | ('Product 49', 'Description 49'), 116 | ('Product 50', 'Description 50'), 117 | ('Product 51', 'Description 51'), 118 | ('Product 52', 'Description 52'), 119 | ('Product 53', 'Description 53'); 120 | 121 | 122 | 123 | INSERT INTO order_products (order_id, product_id, quantity) 124 | VALUES (1, 6, 2), 125 | (1, 8, 1), 126 | (1, 7, 1), 127 | (2, 8, 6), 128 | (2, 8, 1), 129 | (2, 9, 1), 130 | (3, 10, 1), 131 | (3, 11, 1), 132 | (3, 12, 1), 133 | (3, 13, 1), 134 | (3, 14, 1), 135 | (3, 15, 1), 136 | (3, 16, 1), 137 | (3, 17, 1), 138 | (4, 18, 1), 139 | (5, 19, 1), 140 | (6, 20, 1), 141 | (6, 21, 1), 142 | (6, 22, 1), 143 | (6, 23, 1), 144 | (6, 24, 1), 145 | (6, 25, 1), 146 | (6, 26, 1), 147 | (6, 27, 1), 148 | (6, 28, 1), 149 | (6, 29, 1), 150 | (6, 30, 1), 151 | (6, 31, 1), 152 | (6, 32, 1), 153 | (6, 33, 1), 154 | (6, 34, 1); 155 | -------------------------------------------------------------------------------- /Part6-SQLAlchemy - Basics/Tutorial-5.MD: -------------------------------------------------------------------------------- 1 | # Basics 2 | 3 | Now it's time to write some Python code! 4 | 5 | ## Introduction and Installation 6 | 7 | To use SQL efficiently in your bots you need 2 things: 8 | 9 | - some library that will run SQL queries (psycopg2, asyncpg, sqlite3, etc.) 10 | - SQLAlchemy library that will help you to build python-like queries. Since you may be used to Python 11 | syntax you would like to build models and requests to your database with some Python code as well. 12 | 13 | > **Note:** 14 | > 15 | > We will be using syntax for SQLAlchemy v2.0 for this tutorial. 16 | 17 | _This is not a full tutorial, so you might like to check out 18 | the [SQLAlchemy tutorial](https://docs.sqlalchemy.org/en/20/tutorial/index.html) later._ 19 | 20 | First off, let's build a virtual environment and install all the dependencies. 21 | 22 | We will be using the latest stable version, Python 3.10. 23 | 24 | ### Ubuntu installation: 25 | 26 | ```bash 27 | # Install virtualenv 28 | python3 -m pip install virtualenv 29 | 30 | # Create a virtual environment folder 31 | python3 -m virtualenv venv 32 | 33 | # Activate the virtual environment 34 | source venv/bin/activate 35 | 36 | # No we can install all the dependencies into the virtual environment 37 | # Let's install psycopg2==binary-2.9.3 (synchronous driver), asyncpg==0.25.0 (asynchronous driver), also SQLAlchemy~=2.0 38 | pip3 install psycopg2-binary==2.9.3 asyncpg==0.25.0 SQLAlchemy~=2.0 39 | ``` 40 | 41 | Our dependencies are ready to use! 42 | 43 | ## Connecting to a database 44 | 45 | In order to connect to our database we need to 46 | use [SQLAlchemy Engine](https://docs.sqlalchemy.org/en/20/tutorial/engine.html). 47 | It is created by using the `create_engine` function. 48 | 49 | ```python 50 | from sqlalchemy import create_engine 51 | 52 | engine = create_engine('URL', echo=True, future=True) 53 | ``` 54 | 55 | > Echo is a boolean parameter that will print all the queries that are being executed. 56 | > Future is a boolean parameter that will enable the usage of the SQLAlchemy 2.0 syntax. 57 | 58 | Here we are using the string `URL` to connect to our database. It is 59 | called [SQLAlchemy connection string](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls) and looks 60 | something like this: `driver+postgresql://user:password@host:port/dbname`. 61 | 62 | However, you can build it using a URL-builder: 63 | 64 | ```python 65 | from sqlalchemy.engine.url import URL 66 | 67 | URL = URL.create( 68 | drivername="postgresql+psycopg2", # driver name = postgresql + the library we are using 69 | username='testuser', 70 | password='testpassword', 71 | host='localhost', 72 | database='testuser', 73 | port=5432 74 | ) 75 | ``` 76 | 77 | Now you can pass this URL to the `create_engine` function. 78 | 79 | ``` 80 | engine = create_engine(URL, echo=True, future=True) 81 | ``` 82 | 83 | Be aware that we are not yet connected to the database, it will be done when we use the `engine` variable. 84 | 85 | ## Transactions 86 | 87 | As you may have learnt, SQL has Transaction Control Language (TCL) to manage transactions. 88 | 89 | Transaction - is a set of commands that are executed in a single unit of work. 90 | 91 | - A transaction is started by calling the `begin()` method. 92 | - A transaction is committed by calling the `commit()` method. 93 | - A transaction is rolled back by calling the `rollback()` method. 94 | 95 | You can insert some rows and then rollback the transaction, so the rows will not be inserted, otherwise you may commit 96 | the transaction and all changed will be saved. 97 | 98 | In the transactions you can also alter tables structure and rollback that as well. 99 | 100 | ### Syntax for transactions: 101 | 102 | ```SQL 103 | BEGIN; -- start a transaction 104 | INSERT INTO table1 105 | VALUES (1, 'test'); -- insert some dummy data 106 | INSERT INTO table2 107 | VALUES (2, 'test'); -- insert some more dummy data 108 | ALTER TABLE table1 109 | RENAME TO table3; -- rename table1 to table3 110 | ROLLBACK; -- cancel the changes 111 | 112 | BEGIN; 113 | INSERT INTO table1 114 | VALUES (1, 'test'); -- insert some dummy data 115 | COMMIT; -- commit (save) the changes 116 | ``` 117 | 118 | ## Sessions 119 | 120 | Sessions are used to manage the connection to the database. 121 | 122 | Session is used to actually connect to the database using engine. 123 | 124 | We can use a session builder method to quickly create sessions: 125 | 126 | ```python 127 | from sqlalchemy.orm import sessionmaker 128 | 129 | session_pool = sessionmaker(bind=engine) 130 | ``` 131 | 132 | Now every time we need to create a session we can use the `session_pool` variable. 133 | 134 | ```python 135 | session = session_pool() # create a session 136 | session.execute(...) # execute some SQL query 137 | session.commit() # or session.rollback() to commit/rollback the changes 138 | session.close() # close the session (IMPORTANT: you must close the session after you are done with it) 139 | 140 | # OR 141 | 142 | with session_pool() as session: 143 | # do something with session 144 | session.execute(...) 145 | session.commit() # or session.rollback() 146 | ``` 147 | 148 | ## Textual SQL 149 | 150 | You can use the `text()` function to create a SQLAlchemy `Text` object and run raw SQL queries. 151 | This is not how we will use SQLAlchemy in our bots, but I'll show it for smoother transition from SQL lessons. 152 | 153 | We will use tables from the previous tutorials: 154 | 155 | ```python 156 | from sqlalchemy import text # import the `text()` function 157 | 158 | query = text("SELECT * FROM users LIMIT 2") # create a SQLAlchemy `Text` object. 159 | with session_pool() as session: # create a session 160 | result = session.execute(query) # execute the query 161 | for row in result: # result is sqlalchemy.engine.cursor.CursorResult object. You can iterate over it. 162 | print(row) # Every element of the result is a tuple. 163 | ``` 164 | 165 | #### Response: 166 | 167 | ``` 168 | INFO sqlalchemy.engine.Engine BEGIN (implicit) 169 | INFO sqlalchemy.engine.Engine SELECT * FROM users LIMIT 2 170 | INFO sqlalchemy.engine.Engine [generated in 0.00006s] {} 171 | (1, 'Test User', 'test_user', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None) 172 | (2, 'Test User 2', 'test_user_2', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None) 173 | INFO sqlalchemy.engine.Engine ROLLBACK 174 | ``` 175 | 176 | ##### Explanation: 177 | 178 | You can see that firstly the transaction is started, then the query is executed, 179 | then we print the result and finally the transaction is rollbacked automatically. 180 | This is because we didn't commit any changes. 181 | 182 | ## Extracting data from rows 183 | 184 | You can also print every row of the result as a dictionary, or get columns as attributes: 185 | 186 | #### Example 1 187 | 188 | ```python 189 | for row in result: 190 | print(dict(row)) # print every row as a dictionary 191 | print() # print a new line 192 | print(f'ID: {row.telegram_id}, Full name: {row.full_name}, Username: {row.username}') # print as attributes 193 | print() # print a new line 194 | ``` 195 | 196 | ##### Response: 197 | 198 | ``` 199 | {'telegram_id': 1, 'full_name': 'Test User', 'username': 'test_user', 'language_code': 'en', 'created_at': datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), 'referrer_id': None} 200 | 201 | ID: 1, Full name: Test User, Username: test_user 202 | 203 | {'telegram_id': 2, 'full_name': 'Test User 2', 'username': 'test_user_2', 'language_code': 'en', 'created_at': datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), 'referrer_id': None} 204 | 205 | ID: 2, Full name: Test User 2, Username: test_user_2 206 | ``` 207 | 208 | See, now you know that row-objects may be transformed to different types of objects. 209 | 210 | ## Working with Result 211 | 212 | There are different ways you can retrieve the result of a query. Not all results you need are in a form of a table. 213 | 214 | Sometimes you want to retrieve a single row, or a single column at once. 215 | 216 | You can do this by calling different methods of the `Result` object: 217 | 218 | - `all()` - returns all rows of the result as a list of tuples. 219 | - `first()` - returns the first row of the result. 220 | - `scalar()` - returns the first column of the first row of the result. 221 | 222 | > **Note:** 223 | > You can apply these methods only ONCE. The result will be consumed after the first call, and you'll get 224 | > an error: `sqlalchemy.exc.ResourceClosedError: This result object is closed.` 225 | 226 | #### Example 2 227 | 228 | Let's get all rows of the result as a list of tuples: 229 | 230 | ```python 231 | query = text("SELECT * FROM users") 232 | with session_pool() as session: 233 | result = session.execute(query) 234 | all_rows = result.all() 235 | print(f'{all_rows=}') 236 | ``` 237 | 238 | ##### Response: 239 | 240 | ``` 241 | all_rows=[ 242 | (1, 'Test User', 'test_user', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 243 | (2, 'Test User 2', 'test_user_2', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 244 | (3, 'Test User 3', 'test_user_3', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 245 | (4, 'Test User 4', 'test_user_4', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 246 | (5, 'Test User 5', 'test_user_5', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 247 | (6, 'Test User 6', 'test_user_6', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 248 | (7, 'Test User 7', 'test_user_7', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None), 249 | (8, 'Test User 8', 'test_user_8', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None) 250 | ] 251 | ``` 252 | 253 | #### Example 3 254 | 255 | Let's get the first row of the result: 256 | 257 | ```python 258 | query = text("SELECT * FROM users") 259 | with session_pool() as session: 260 | result = session.execute(query) 261 | first_row = result.first() 262 | print(f'{first_row=}') 263 | ``` 264 | 265 | ##### Response: 266 | 267 | ``` 268 | first_row=(1, 'Test User', 'test_user', 'en', datetime.datetime(2022, 7, 2, 9, 2, 40, 711703), None) 269 | ``` 270 | 271 | #### Example 4 272 | 273 | Let's get the first column of the first row of the result: 274 | > **Note:** 275 | > 276 | > If you want to pass the parameters to the query, you can name it as a keyword argument (`:id`) and pass it as a 277 | > keyword argument to the `params` method. 278 | > 279 | > **Never insert a parameter into a string by formating methods, your query will be vulnerable to SQL injection.** 280 | 281 | 282 | ```python 283 | query = text("SELECT full_name FROM users WHERE telegram_id = :id").params(id=1) 284 | with session_pool() as session: 285 | result = session.execute(query) 286 | full_name = result.scalar() 287 | print(f'{full_name=}') 288 | ``` 289 | 290 | ##### Response: 291 | 292 | ``` 293 | full_name='Test User' 294 | ``` 295 | 296 | Great! Now you know how to retrieve data from a result. -------------------------------------------------------------------------------- /Part6-SQLAlchemy - Basics/main.py: -------------------------------------------------------------------------------- 1 | from sqlalchemy import create_engine 2 | from sqlalchemy import text 3 | from sqlalchemy.engine.url import URL 4 | from sqlalchemy.orm import sessionmaker 5 | 6 | URL = URL.create( 7 | drivername="postgresql+psycopg2", # driver name = postgresql + the library we are using 8 | username='testuser', 9 | password='testpassword', 10 | host='localhost', 11 | database='testuser', 12 | port=5439 13 | ) 14 | engine = create_engine(URL, echo=True, future=True) 15 | 16 | session_pool = sessionmaker(bind=engine) 17 | 18 | 19 | def example_1(): 20 | query = text("SELECT * FROM users LIMIT 2") 21 | with session_pool() as session: 22 | result = session.execute(query) 23 | 24 | for row in result: 25 | print(dict(row)) # print every row as a dictionary 26 | print() # print a new line 27 | print(f'ID: {row.telegram_id}, Full name: {row.full_name}, Username: {row.username}') # print as attributes 28 | print() # print a new line 29 | 30 | 31 | def example_2(): 32 | query = text("SELECT * FROM users") 33 | with session_pool() as session: 34 | result = session.execute(query) 35 | all_rows = result.all() 36 | print(f'{all_rows=}') 37 | 38 | 39 | def example_3(): 40 | query = text("SELECT * FROM users") 41 | with session_pool() as session: 42 | result = session.execute(query) 43 | first_row = result.first() 44 | print(f'{first_row=}') 45 | 46 | 47 | def example_4(): 48 | query = text("SELECT full_name FROM users WHERE telegram_id = :id").params(id=1) 49 | with session_pool() as session: 50 | result = session.execute(query) 51 | full_name = result.scalar() 52 | print(f'{full_name=}') 53 | 54 | 55 | example_3() 56 | -------------------------------------------------------------------------------- /Part6-SQLAlchemy - Basics/tutorial-5.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Basics\n", 7 | "\n", 8 | "Now it's time to write some Python code!\n", 9 | "\n", 10 | "## Introduction and Installation\n", 11 | "\n", 12 | "To use SQL efficiently in your bots you need 2 things:\n", 13 | "\n", 14 | "- some library that will run SQL queries (psycopg2, asyncpg, sqlite3, etc.)\n", 15 | "- SQLAlchemy library that will help you to build python-like queries. Since you may be used to Python\n", 16 | " syntax you would like to build models and requests to your database with some Python code as well.\n", 17 | "\n", 18 | "> **Note:**\n", 19 | ">\n", 20 | "> We will be using syntax for SQLAlchemy v2.0 for this tutorial.\n", 21 | "\n", 22 | "_This is not a full tutorial, so you might like to check out\n", 23 | "the [SQLAlchemy tutorial](https://docs.sqlalchemy.org/en/20/tutorial/index.html) later._\n", 24 | "\n", 25 | "First off, let's build a virtual environment and install all the dependencies.\n", 26 | "\n", 27 | "We will be using the latest stable version, Python 3.10.\n", 28 | "\n", 29 | "### Ubuntu installation:\n" 30 | ], 31 | "metadata": { 32 | "collapsed": false 33 | } 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 1, 38 | "outputs": [ 39 | { 40 | "name": "stdout", 41 | "output_type": "stream", 42 | "text": [ 43 | "Collecting psycopg2-binary==2.9.3\n", 44 | " Using cached psycopg2_binary-2.9.3-cp310-cp310-win_amd64.whl (1.2 MB)\n", 45 | "Collecting asyncpg==0.25.0\n", 46 | " Using cached asyncpg-0.25.0-cp310-cp310-win_amd64.whl (525 kB)\n", 47 | "Collecting SQLAlchemy~=2.0\n", 48 | " Using cached SQLAlchemy-2.0.12-cp310-cp310-win_amd64.whl (2.0 MB)\n", 49 | "Collecting greenlet!=0.4.17\n", 50 | " Using cached greenlet-2.0.2-cp310-cp310-win_amd64.whl (192 kB)\n", 51 | "Collecting typing-extensions>=4.2.0\n", 52 | " Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)\n", 53 | "Installing collected packages: typing-extensions, psycopg2-binary, greenlet, asyncpg, SQLAlchemy\n", 54 | "Successfully installed SQLAlchemy-2.0.12 asyncpg-0.25.0 greenlet-2.0.2 psycopg2-binary-2.9.3 typing-extensions-4.5.0\n" 55 | ] 56 | }, 57 | { 58 | "name": "stderr", 59 | "output_type": "stream", 60 | "text": [ 61 | "\n", 62 | "[notice] A new release of pip is available: 23.0.1 -> 23.1.2\n", 63 | "[notice] To update, run: python.exe -m pip install --upgrade pip\n" 64 | ] 65 | } 66 | ], 67 | "source": [ 68 | "!pip3 install psycopg2-binary==2.9.3 asyncpg==0.25.0 SQLAlchemy~=2.0" 69 | ], 70 | "metadata": { 71 | "collapsed": false, 72 | "ExecuteTime": { 73 | "end_time": "2023-05-08T13:44:05.042352300Z", 74 | "start_time": "2023-05-08T13:44:00.165512300Z" 75 | } 76 | } 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "source": [ 81 | "\n", 82 | "Our dependencies are ready to use!\n", 83 | "\n", 84 | "## Connecting to a database\n", 85 | "\n", 86 | "In order to connect to our database we need to\n", 87 | "use [SQLAlchemy Engine](https://docs.sqlalchemy.org/en/20/tutorial/engine.html).\n", 88 | "It is created by using the `create_engine` function.\n", 89 | "\n", 90 | "```python\n", 91 | "from sqlalchemy import create_engine\n", 92 | "\n", 93 | "engine = create_engine('URL', echo=True, future=True)\n", 94 | "```\n", 95 | "\n", 96 | "> Echo is a boolean parameter that will print all the queries that are being executed.\n", 97 | "> Future is a boolean parameter that will enable the usage of the SQLAlchemy 2.0 syntax.\n", 98 | "\n", 99 | "Here we are using the string `URL` to connect to our database. It is\n", 100 | "called [SQLAlchemy connection string](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls) and looks\n", 101 | "something like this: `driver+postgresql://user:password@host:port/dbname`.\n", 102 | "\n", 103 | "However, you can build it using a URL-builder:\n" 104 | ], 105 | "metadata": { 106 | "collapsed": false 107 | } 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": 4, 112 | "outputs": [], 113 | "source": [ 114 | "from sqlalchemy import create_engine\n", 115 | "from sqlalchemy.engine.url import URL\n", 116 | "\n", 117 | "URL = URL.create(\n", 118 | " drivername=\"postgresql+psycopg2\", # driver name = postgresql + the library we are using\n", 119 | " username='testuser',\n", 120 | " password='testpassword',\n", 121 | " host='localhost', # insert your host here\n", 122 | " database='testuser',\n", 123 | " port=5432\n", 124 | ")\n", 125 | "\n", 126 | "# Now you can pass this URL to the `create_engine` function.\n", 127 | "\n", 128 | "engine = create_engine(URL, echo=True, future=True)" 129 | ], 130 | "metadata": { 131 | "collapsed": false, 132 | "ExecuteTime": { 133 | "end_time": "2023-05-08T13:49:56.353823300Z", 134 | "start_time": "2023-05-08T13:49:56.148323Z" 135 | } 136 | } 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "source": [ 141 | "Be aware that we are not yet connected to the database, it will be done when we use the `engine` variable.\n", 142 | "\n", 143 | "## Transactions\n", 144 | "\n", 145 | "As you may have learnt, SQL has Transaction Control Language (TCL) to manage transactions.\n", 146 | "\n", 147 | "Transaction - is a set of commands that are executed in a single unit of work.\n", 148 | "\n", 149 | "- A transaction is started by calling the `begin()` method.\n", 150 | "- A transaction is committed by calling the `commit()` method.\n", 151 | "- A transaction is rolled back by calling the `rollback()` method.\n", 152 | "\n", 153 | "You can insert some rows and then rollback the transaction, so the rows will not be inserted, otherwise you may commit\n", 154 | "the transaction and all changed will be saved.\n", 155 | "\n", 156 | "In the transactions you can also alter tables structure and rollback that as well.\n", 157 | "\n", 158 | "### Syntax for transactions:\n", 159 | "\n", 160 | "```SQL\n", 161 | "BEGIN; -- start a transaction\n", 162 | "INSERT INTO table1\n", 163 | "VALUES (1, 'test'); -- insert some dummy data\n", 164 | "INSERT INTO table2\n", 165 | "VALUES (2, 'test'); -- insert some more dummy data\n", 166 | "ALTER TABLE table1\n", 167 | " RENAME TO table3; -- rename table1 to table3\n", 168 | "ROLLBACK; -- cancel the changes\n", 169 | "\n", 170 | "BEGIN;\n", 171 | "INSERT INTO table1\n", 172 | "VALUES (1, 'test'); -- insert some dummy data\n", 173 | "COMMIT; -- commit (save) the changes\n", 174 | "```\n", 175 | "\n", 176 | "## Sessions\n", 177 | "\n", 178 | "Sessions are used to manage the connection to the database.\n", 179 | "\n", 180 | "Session is used to actually connect to the database using engine." 181 | ], 182 | "metadata": { 183 | "collapsed": false 184 | } 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": 5, 189 | "outputs": [], 190 | "source": [ 191 | "from sqlalchemy.orm import sessionmaker\n", 192 | "\n", 193 | "session_pool = sessionmaker(bind=engine)\n" 194 | ], 195 | "metadata": { 196 | "collapsed": false, 197 | "ExecuteTime": { 198 | "end_time": "2023-05-08T13:50:01.248054100Z", 199 | "start_time": "2023-05-08T13:50:01.060053300Z" 200 | } 201 | } 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "source": [ 206 | "Now every time we need to create a session we can use the `session_pool` variable.\n", 207 | "\n", 208 | "```python\n", 209 | "session = session_pool() # create a session\n", 210 | "session.execute(...) # execute some SQL query\n", 211 | "session.commit() # or session.rollback() to commit/rollback the changes\n", 212 | "session.close() # close the session (IMPORTANT: you must close the session after you are done with it)\n", 213 | "\n", 214 | "# OR\n", 215 | "\n", 216 | "with session_pool() as session:\n", 217 | " # do something with session\n", 218 | " session.execute(...)\n", 219 | " session.commit() # or session.rollback()\n", 220 | "```\n", 221 | "\n", 222 | "## Textual SQL\n", 223 | "\n", 224 | "You can use the `text()` function to create a SQLAlchemy `Text` object and run raw SQL queries.\n", 225 | "This is not how we will use SQLAlchemy in our bots, but I'll show it for smoother transition from SQL lessons.\n", 226 | "\n", 227 | "We will use tables from the previous tutorials:" 228 | ], 229 | "metadata": { 230 | "collapsed": false 231 | } 232 | }, 233 | { 234 | "cell_type": "code", 235 | "execution_count": 10, 236 | "outputs": [ 237 | { 238 | "name": "stdout", 239 | "output_type": "stream", 240 | "text": [ 241 | "2023-05-08 16:57:25,959 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 242 | "2023-05-08 16:57:25,959 INFO sqlalchemy.engine.Engine SELECT * FROM users LIMIT 2\n", 243 | "2023-05-08 16:57:25,960 INFO sqlalchemy.engine.Engine [cached since 152.3s ago] {}\n", 244 | "(1, 'John Doe', 'johnny', 'en', datetime.datetime(2020, 1, 1, 0, 0), None)\n", 245 | "(2, 'Jane Doe', 'jane', 'en', datetime.datetime(2020, 1, 2, 0, 0), 1)\n", 246 | "2023-05-08 16:57:25,963 INFO sqlalchemy.engine.Engine ROLLBACK\n" 247 | ] 248 | } 249 | ], 250 | "source": [ 251 | "from sqlalchemy import text # import the `text()` function\n", 252 | "\n", 253 | "query = text(\"SELECT * FROM users LIMIT 2\") # create a SQLAlchemy `Text` object.\n", 254 | "with session_pool() as session: # create a session\n", 255 | " result = session.execute(query) # execute the query\n", 256 | " for row in result: # result is sqlalchemy.engine.cursor.CursorResult object. You can iterate over it.\n", 257 | " print(row) # Every element of the result is a tuple.\n" 258 | ], 259 | "metadata": { 260 | "collapsed": false, 261 | "ExecuteTime": { 262 | "end_time": "2023-05-08T13:57:26.162268100Z", 263 | "start_time": "2023-05-08T13:57:25.960769400Z" 264 | } 265 | } 266 | }, 267 | { 268 | "cell_type": "markdown", 269 | "source": [ 270 | "# Database preparation\n", 271 | "In case you don't have a postgresql setup yet, use this Docker command (provided you have docker installed):" 272 | ], 273 | "metadata": { 274 | "collapsed": false 275 | } 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "outputs": [], 281 | "source": [ 282 | "!docker run --name postgresql -e POSTGRES_PASSWORD=testpassword -e POSTGRES_USER=testuser -e POSTGRES_DB=testuser -p 5432:5432 -d postgres:13.4-alpine" 283 | ], 284 | "metadata": { 285 | "collapsed": false 286 | } 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": 9, 291 | "outputs": [ 292 | { 293 | "name": "stdout", 294 | "output_type": "stream", 295 | "text": [ 296 | "2023-05-08 16:57:16,935 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 297 | "2023-05-08 16:57:16,935 INFO sqlalchemy.engine.Engine \n", 298 | "\n", 299 | "CREATE TABLE users\n", 300 | "(\n", 301 | " telegram_id BIGINT PRIMARY KEY,\n", 302 | " full_name VARCHAR(255) NOT NULL,\n", 303 | " username VARCHAR(255),\n", 304 | " language_code VARCHAR(255) NOT NULL,\n", 305 | " created_at TIMESTAMP DEFAULT NOW(),\n", 306 | " referrer_id BIGINT,\n", 307 | " FOREIGN KEY (referrer_id)\n", 308 | " REFERENCES users (telegram_id)\n", 309 | " ON DELETE SET NULL\n", 310 | ");\n", 311 | "\n", 312 | "\n", 313 | "INSERT INTO users\n", 314 | " (telegram_id, full_name, username, language_code, created_at)\n", 315 | "VALUES (1, 'John Doe', 'johnny', 'en', '2020-01-01');\n", 316 | "\n", 317 | "\n", 318 | "INSERT INTO users\n", 319 | "(telegram_id, full_name, username, language_code, created_at, referrer_id)\n", 320 | "VALUES (2, 'Jane Doe', 'jane', 'en', '2020-01-02', 1);\n", 321 | "\n", 322 | "\n", 323 | "2023-05-08 16:57:16,936 INFO sqlalchemy.engine.Engine [generated in 0.00046s] {}\n", 324 | "2023-05-08 16:57:16,949 INFO sqlalchemy.engine.Engine COMMIT\n" 325 | ] 326 | } 327 | ], 328 | "source": [ 329 | "# Create tables and insert some data\n", 330 | "\n", 331 | "from sqlalchemy import text # import the `text()` function\n", 332 | "\n", 333 | "query = text(\"\"\"\n", 334 | "\n", 335 | "CREATE TABLE users\n", 336 | "(\n", 337 | " telegram_id BIGINT PRIMARY KEY,\n", 338 | " full_name VARCHAR(255) NOT NULL,\n", 339 | " username VARCHAR(255),\n", 340 | " language_code VARCHAR(255) NOT NULL,\n", 341 | " created_at TIMESTAMP DEFAULT NOW(),\n", 342 | " referrer_id BIGINT,\n", 343 | " FOREIGN KEY (referrer_id)\n", 344 | " REFERENCES users (telegram_id)\n", 345 | " ON DELETE SET NULL\n", 346 | ");\n", 347 | "\n", 348 | "\n", 349 | "INSERT INTO users\n", 350 | " (telegram_id, full_name, username, language_code, created_at)\n", 351 | "VALUES (1, 'John Doe', 'johnny', 'en', '2020-01-01');\n", 352 | "\n", 353 | "\n", 354 | "INSERT INTO users\n", 355 | "(telegram_id, full_name, username, language_code, created_at, referrer_id)\n", 356 | "VALUES (2, 'Jane Doe', 'jane', 'en', '2020-01-02', 1);\n", 357 | "\n", 358 | "\"\"\")\n", 359 | "with session_pool() as session: # create a session\n", 360 | " session.execute(query) # execute the query\n", 361 | " session.commit() # commit the changes" 362 | ], 363 | "metadata": { 364 | "collapsed": false, 365 | "ExecuteTime": { 366 | "end_time": "2023-05-08T13:57:17.181001500Z", 367 | "start_time": "2023-05-08T13:57:16.935500200Z" 368 | } 369 | } 370 | }, 371 | { 372 | "cell_type": "markdown", 373 | "source": [ 374 | "##### Explanation:\n", 375 | "\n", 376 | "You can see that firstly the transaction is started, then the query is executed,\n", 377 | "then we print the result and finally the transaction is rollbacked automatically.\n", 378 | "This is because we didn't commit any changes.\n", 379 | "\n", 380 | "## Extracting data from rows\n", 381 | "\n", 382 | "You can also print every row of the result as a dictionary, or get columns as attributes:" 383 | ], 384 | "metadata": { 385 | "collapsed": false 386 | } 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": 15, 391 | "outputs": [ 392 | { 393 | "name": "stdout", 394 | "output_type": "stream", 395 | "text": [ 396 | "2023-05-08 17:00:40,987 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 397 | "2023-05-08 17:00:40,988 INFO sqlalchemy.engine.Engine SELECT * FROM users LIMIT 2\n", 398 | "2023-05-08 17:00:40,988 INFO sqlalchemy.engine.Engine [cached since 347.4s ago] {}\n", 399 | "ID: 1, Full name: John Doe, Username: johnny\n", 400 | "\n", 401 | "ID: 2, Full name: Jane Doe, Username: jane\n", 402 | "\n", 403 | "2023-05-08 17:00:40,992 INFO sqlalchemy.engine.Engine ROLLBACK\n" 404 | ] 405 | } 406 | ], 407 | "source": [ 408 | "with session_pool() as session: # create a session\n", 409 | " result = session.execute(query) # execute the query\n", 410 | "\n", 411 | " for row in result:\n", 412 | " # print(dict(row)) # print every row as a dictionary\n", 413 | " # print() # print a new line\n", 414 | " print(f'ID: {row.telegram_id}, Full name: {row.full_name}, Username: {row.username}') # print as attributes\n", 415 | " print() # print a new line\n" 416 | ], 417 | "metadata": { 418 | "collapsed": false, 419 | "ExecuteTime": { 420 | "end_time": "2023-05-08T14:00:41.211807600Z", 421 | "start_time": "2023-05-08T14:00:40.989303500Z" 422 | } 423 | } 424 | }, 425 | { 426 | "cell_type": "markdown", 427 | "source": [ 428 | "See, now you know that row-objects may be transformed to different types of objects.\n", 429 | "\n", 430 | "## Working with Result\n", 431 | "\n", 432 | "There are different ways you can retrieve the result of a query. Not all results you need are in a form of a table.\n", 433 | "\n", 434 | "Sometimes you want to retrieve a single row, or a single column at once.\n", 435 | "\n", 436 | "You can do this by calling different methods of the `Result` object:\n", 437 | "\n", 438 | "- `all()` - returns all rows of the result as a list of tuples.\n", 439 | "- `first()` - returns the first row of the result.\n", 440 | "- `scalar()` - returns the first column of the first row of the result.\n", 441 | "\n", 442 | "> **Note:**\n", 443 | "> You can apply these methods only ONCE. The result will be consumed after the first call, and you'll get\n", 444 | "> an error: `sqlalchemy.exc.ResourceClosedError: This result object is closed.`\n", 445 | "\n", 446 | "#### Example 2\n", 447 | "\n", 448 | "Let's get all rows of the result as a list of tuples:" 449 | ], 450 | "metadata": { 451 | "collapsed": false 452 | } 453 | }, 454 | { 455 | "cell_type": "code", 456 | "execution_count": 16, 457 | "outputs": [ 458 | { 459 | "name": "stdout", 460 | "output_type": "stream", 461 | "text": [ 462 | "2023-05-08 17:01:37,104 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 463 | "2023-05-08 17:01:37,105 INFO sqlalchemy.engine.Engine SELECT * FROM users\n", 464 | "2023-05-08 17:01:37,106 INFO sqlalchemy.engine.Engine [generated in 0.00060s] {}\n", 465 | "all_rows=[(1, 'John Doe', 'johnny', 'en', datetime.datetime(2020, 1, 1, 0, 0), None), (2, 'Jane Doe', 'jane', 'en', datetime.datetime(2020, 1, 2, 0, 0), 1)]\n", 466 | "2023-05-08 17:01:37,108 INFO sqlalchemy.engine.Engine ROLLBACK\n" 467 | ] 468 | } 469 | ], 470 | "source": [ 471 | "query = text(\"SELECT * FROM users\")\n", 472 | "with session_pool() as session:\n", 473 | " result = session.execute(query)\n", 474 | " all_rows = result.all()\n", 475 | " print(f'{all_rows=}')\n" 476 | ], 477 | "metadata": { 478 | "collapsed": false, 479 | "ExecuteTime": { 480 | "end_time": "2023-05-08T14:01:37.340338600Z", 481 | "start_time": "2023-05-08T14:01:37.112841300Z" 482 | } 483 | } 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "source": [ 488 | "#### Example 3\n", 489 | "\n", 490 | "Let's get the first row of the result:" 491 | ], 492 | "metadata": { 493 | "collapsed": false 494 | } 495 | }, 496 | { 497 | "cell_type": "code", 498 | "execution_count": 17, 499 | "outputs": [ 500 | { 501 | "name": "stdout", 502 | "output_type": "stream", 503 | "text": [ 504 | "2023-05-08 17:02:10,028 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 505 | "2023-05-08 17:02:10,028 INFO sqlalchemy.engine.Engine SELECT * FROM users\n", 506 | "2023-05-08 17:02:10,029 INFO sqlalchemy.engine.Engine [cached since 32.92s ago] {}\n", 507 | "first_row=(1, 'John Doe', 'johnny', 'en', datetime.datetime(2020, 1, 1, 0, 0), None)\n", 508 | "2023-05-08 17:02:10,031 INFO sqlalchemy.engine.Engine ROLLBACK\n" 509 | ] 510 | } 511 | ], 512 | "source": [ 513 | "query = text(\"SELECT * FROM users\")\n", 514 | "with session_pool() as session:\n", 515 | " result = session.execute(query)\n", 516 | " first_row = result.first()\n", 517 | " print(f'{first_row=}')" 518 | ], 519 | "metadata": { 520 | "collapsed": false, 521 | "ExecuteTime": { 522 | "end_time": "2023-05-08T14:02:10.255874300Z", 523 | "start_time": "2023-05-08T14:02:10.036372900Z" 524 | } 525 | } 526 | }, 527 | { 528 | "cell_type": "markdown", 529 | "source": [ 530 | "#### Example 4\n", 531 | "\n", 532 | "Let's get the first column of the first row of the result:\n", 533 | "> **Note:**\n", 534 | ">\n", 535 | "> If you want to pass the parameters to the query, you can name it as a keyword argument (`:id`) and pass it as a\n", 536 | "> keyword argument to the `params` method.\n", 537 | ">\n", 538 | "> **Never insert a parameter into a string by formatting methods, your query will be vulnerable to SQL injection.**" 539 | ], 540 | "metadata": { 541 | "collapsed": false 542 | } 543 | }, 544 | { 545 | "cell_type": "code", 546 | "execution_count": 18, 547 | "outputs": [ 548 | { 549 | "name": "stdout", 550 | "output_type": "stream", 551 | "text": [ 552 | "2023-05-08 17:03:11,120 INFO sqlalchemy.engine.Engine BEGIN (implicit)\n", 553 | "2023-05-08 17:03:11,121 INFO sqlalchemy.engine.Engine SELECT full_name FROM users WHERE telegram_id = %(id)s\n", 554 | "2023-05-08 17:03:11,122 INFO sqlalchemy.engine.Engine [generated in 0.00083s] {'id': 1}\n", 555 | "full_name='John Doe'\n", 556 | "2023-05-08 17:03:11,125 INFO sqlalchemy.engine.Engine ROLLBACK\n" 557 | ] 558 | } 559 | ], 560 | "source": [ 561 | "query = text(\"SELECT full_name FROM users WHERE telegram_id = :id\").params(id=1)\n", 562 | "with session_pool() as session:\n", 563 | " result = session.execute(query)\n", 564 | " full_name = result.scalar()\n", 565 | " print(f'{full_name=}')\n" 566 | ], 567 | "metadata": { 568 | "collapsed": false, 569 | "ExecuteTime": { 570 | "end_time": "2023-05-08T14:03:11.354523300Z", 571 | "start_time": "2023-05-08T14:03:11.129021400Z" 572 | } 573 | } 574 | }, 575 | { 576 | "cell_type": "markdown", 577 | "source": [ 578 | "Great! Now you know how to retrieve data from a result." 579 | ], 580 | "metadata": { 581 | "collapsed": false 582 | } 583 | } 584 | ], 585 | "metadata": { 586 | "kernelspec": { 587 | "display_name": "Python 3", 588 | "language": "python", 589 | "name": "python3" 590 | }, 591 | "language_info": { 592 | "codemirror_mode": { 593 | "name": "ipython", 594 | "version": 2 595 | }, 596 | "file_extension": ".py", 597 | "mimetype": "text/x-python", 598 | "name": "python", 599 | "nbconvert_exporter": "python", 600 | "pygments_lexer": "ipython2", 601 | "version": "2.7.6" 602 | } 603 | }, 604 | "nbformat": 4, 605 | "nbformat_minor": 0 606 | } 607 | -------------------------------------------------------------------------------- /README.MD: -------------------------------------------------------------------------------- 1 | # SQL Tutorial 2 | 3 | This is an English text-version of lessons [published on Youtube (in Russian)](https://www.youtube.com/playlist?list=PLwVBSkoL97Q3q7zHVJLojlu1xs7_xdJ0c), which will introduce you to the world of PostgreSQL and SQL Syntax, in the context of [Telegram bots](https://core.telegram.org/bots/api). 4 | 5 | ### This is only the half of all lessons and this part does not cover the lessons where we connect SQLAlchemy with our bots. 6 | 7 | So if you liked this part and want to integrate **SQL**, **PostgreSQL**, **Alembic** and **SQLAlchemy** into your bots, you can purchase a full bot-development course on [my website](https://botfather.dev/) and you'll get an access to the second part of the tutorial (text and videos). 8 | _(That part is still in development and I'll remove this message when it is available)._ 9 | -------------------------------------------------------------------------------- /img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Latand/SQL-Tutorial/da30d7b54f21f8673076d88a656b30d765a5e158/img.png --------------------------------------------------------------------------------