├── LICENSE ├── README.md ├── img ├── 01.jpg ├── 02.jpg ├── ERD.png ├── ERD2.png └── logo.png └── mysql.ipynb /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Craig 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![CraigDoesData][logo]][link] 2 | 3 | [logo]: ./img/logo.png 4 | [link]: https://www.craigdoesdata.com/ 5 | 6 | 7 | # MySQL + Python for Data Analysts 8 | 9 | Using [MySQL Connector](https://dev.mysql.com/doc/connector-python/en/) and Python to implement a database on MySQL Server, and to create, read, update and delete data in that database. 10 | 11 | #### Project status - Complete 12 | 13 | 14 | ## Introduction 15 | 16 | This repo accompanies [this tutorial]() I wrote for [freeCodeCamp](https://www.freecodecamp.org/). 17 | 18 | The [notebook](https://github.com/thecraigd/Python_SQL/blob/master/mysql.ipynb) takes the reader step-by-step through all the processes involved with using Python and the [MySQL Connector](https://dev.mysql.com/doc/connector-python/en/) to perform the standard [CRUD functions](https://stackify.com/what-are-crud-operations/) on a database running on [MySQL Server](https://dev.mysql.com/downloads/mysql/). 19 | 20 | This repository uses code and data from [this repository](https://github.com/thecraigd/SQL_School_Tutorial/) which accompanies my [series of SQL Tutorials](https://towardsdatascience.com/tagged/sql-series) on [Towards Data Science](https://towardsdatascience.com/). If you are not familiar with relational databases and SQL, I recommend going through those tutorials first. 21 | 22 | We will be implementing the same code to build the below database for the International Language School, but this time doing it via Python using MySQL Connector. 23 | 24 | 25 | [![Entity Relationship Diagram for Database][erd]][link1] 26 | 27 | [erd]: ./img/ERD.png 28 | [link1]: https://towardsdatascience.com/designing-a-relational-database-and-creating-an-entity-relationship-diagram-89c1c19320b2 29 | 30 | ----------------- 31 | 32 | 33 | ### Methods used 34 | * Defining functions in Python 35 | * Database Implementation 36 | * Creating, Reading, Updating and Deleting data using SQL and Python 37 | 38 | 39 | ### Technologies used 40 | * [MySQL Community Server](https://dev.mysql.com/downloads/mysql/) 41 | * [MySQL Python Connector](https://dev.mysql.com/doc/connector-python/en/) 42 | * [PopSQL](https://popsql.com/) 43 | * [Jupyter Notebook](https://jupyter.org/) 44 | * [pandas](https://pandas.pydata.org/) 45 | 46 | ### Data sources 47 | The data used were created by myself for an earlier [project](https://github.com/thecraigd/SQL_School_Tutorial). 48 | Some participant names were created using [Fake Name Generator](https://www.fakenamegenerator.com/). 49 | 50 | 51 | ### Getting started 52 | 53 | 1. Clone this repo (for help see this [tutorial](https://help.github.com/articles/cloning-a-repository/)). 54 | 2. All code is contained within the [Jupyter Notebook](https://github.com/thecraigd/Python_SQL/blob/master/mysql.ipynb) for this project, stored in the root folder as mysql.ipynb 55 | 56 | 57 | ## Featured Notebooks 58 | * [mysql](https://github.com/thecraigd/Python_SQL/blob/master/mysql.ipynb) 59 | 60 | * [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/thecraigd/Python_SQL/master) 61 | 62 | ## Contact 63 | All feedback is warmly received. Craig Dickson can be contacted at [craigdoesdata.com](https://www.craigdoesdata.com/contact.html). 64 | -------------------------------------------------------------------------------- /img/01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thecraigd/Python_SQL/4821dc0d477ecac8bd4bb01017406ae4513e9cf5/img/01.jpg -------------------------------------------------------------------------------- /img/02.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thecraigd/Python_SQL/4821dc0d477ecac8bd4bb01017406ae4513e9cf5/img/02.jpg -------------------------------------------------------------------------------- /img/ERD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thecraigd/Python_SQL/4821dc0d477ecac8bd4bb01017406ae4513e9cf5/img/ERD.png -------------------------------------------------------------------------------- /img/ERD2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thecraigd/Python_SQL/4821dc0d477ecac8bd4bb01017406ae4513e9cf5/img/ERD2.png -------------------------------------------------------------------------------- /img/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thecraigd/Python_SQL/4821dc0d477ecac8bd4bb01017406ae4513e9cf5/img/logo.png -------------------------------------------------------------------------------- /mysql.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "[![CraigDoesData][logo]][link]\n", 8 | "\n", 9 | "[logo]: ./img/logo.png\n", 10 | "[link]: https://www.craigdoesdata.de/\n", 11 | "\n", 12 | "\n", 13 | "# MySQL + Python for Data Analysts\n", 14 | "\n", 15 | "Using [MySQL Connector](https://dev.mysql.com/doc/connector-python/en/) and Python to implement a database on MySQL Server, and to create, read, update and delete data in that database.\n", 16 | "\n", 17 | "#### Project status - Complete\n", 18 | "\n", 19 | "\n", 20 | "## Introduction\n", 21 | "\n", 22 | "This notebook accompanies [this tutorial]() I wrote for [freeCodeCamp](https://www.freecodecamp.org/).\n", 23 | "\n", 24 | "The notebook takes the reader step-by-step through all the processes involved with using Python and the [MySQL Connector](https://dev.mysql.com/doc/connector-python/en/) to perform the standard [CRUD functions](https://stackify.com/what-are-crud-operations/) on a database running on [MySQL Server](https://dev.mysql.com/downloads/mysql/).\n", 25 | "\n", 26 | "This notebook uses code and data from [this repository](https://github.com/thecraigd/SQL_School_Tutorial/) which accompanies my [series of SQL Tutorials](https://towardsdatascience.com/tagged/sql-series) on [Towards Data Science](https://towardsdatascience.com/). If you are not familiar with relational databases and SQL, I recommend going through those tutorials first.\n", 27 | "\n", 28 | "We will be implementing the same code to build the below database for the International Language School, but this time doing it via Python using MySQL Connector.\n", 29 | "\n", 30 | "\n", 31 | "[![Entity Relationship Diagram for Database][erd]][link1]\n", 32 | "\n", 33 | "[erd]: ./img/ERD.png\n", 34 | "[link1]: https://towardsdatascience.com/designing-a-relational-database-and-creating-an-entity-relationship-diagram-89c1c19320b2\n", 35 | "\n", 36 | "-----------------\n", 37 | "\n", 38 | "\n", 39 | "### Methods used\n", 40 | "* Defining functions in Python\n", 41 | "* Database Implementation\n", 42 | "* Creating, Reading, Updating and Deleting data using SQL and Python\n", 43 | "\n", 44 | "\n", 45 | "### Technologies used\n", 46 | "* [MySQL Community Server](https://dev.mysql.com/downloads/mysql/)\n", 47 | "* [MySQL Python Connector](https://dev.mysql.com/doc/connector-python/en/)\n", 48 | "* [PopSQL](https://popsql.com/)\n", 49 | "* [Jupyter Notebook](https://jupyter.org/)\n", 50 | "* [pandas](https://pandas.pydata.org/)\n", 51 | "\n", 52 | "### Data sources\n", 53 | "The data used were created by myself for an earlier [project](https://github.com/thecraigd/SQL_School_Tutorial). \n", 54 | "Some participant names were created using [Fake Name Generator](https://www.fakenamegenerator.com/).\n", 55 | "\n", 56 | "\n", 57 | "\n", 58 | "---------------------\n", 59 | "\n", 60 | "\n", 61 | "## Table of Contents\n", 62 | "\n", 63 | "#### 1. Library Import\n", 64 | "* 1.1 Library Import\n", 65 | "\n", 66 | "#### 2. Connect to Server and Create Database\n", 67 | "* 2.1 Define Server Connection Function \n", 68 | "* 2.2 Create a New Database\n", 69 | "* 2.3 Modify Server Connection Function, Create Database Connection Function\n", 70 | "* 2.4 Define Query Execution Function\n", 71 | "\n", 72 | "#### 3. Creating Tables\n", 73 | "* 3.1 Create Teacher Table\n", 74 | "* 3.2 Create Remaining Tables\n", 75 | "* 3.3 Define Foreign Key Relationships\n", 76 | "\n", 77 | "#### 4. Populating Tables\n", 78 | "* 4.1 Populate Teacher Table\n", 79 | "* 4.2 Populate Remaining Tables\n", 80 | "\n", 81 | "#### 5. Reading Data\n", 82 | "* 5.1 Define Data Reading Function\n", 83 | "* 5.2 - Read Data from Database\n", 84 | "* 5.3 - Formatting Output into a List\n", 85 | "* 5.4 - Formatting Output into a List of Lists\n", 86 | "* 5.5 - Formatting Output into a pandas DataFrame\n", 87 | "\n", 88 | "#### 6. Updating Records\n", 89 | "* 6.1 Updating Client Address\n", 90 | "\n", 91 | "#### 7. Deleting Records\n", 92 | "* 7.1 Deleting a Course\n", 93 | "* 7.2 Restoring the Course\n", 94 | "\n", 95 | "#### 8. Creating Records from Lists\n", 96 | "* 8.1 Create Execute List Query Function\n", 97 | "* 8.2 Add New Teachers\n", 98 | "\n", 99 | "#### 9. Conclusion\n", 100 | "* 9.1 Conclusion\n", 101 | "\n", 102 | "--------------\n", 103 | "\n", 104 | "### 1. Import Libraries\n", 105 | "\n", 106 | "##### 1.1 - Import Libraries\n", 107 | "\n", 108 | "Our first step is to import [MySQL Connector](https://dev.mysql.com/doc/connector-python/en/) and [pandas](https://pandas.pydata.org/)." 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 1, 114 | "metadata": {}, 115 | "outputs": [], 116 | "source": [ 117 | "import mysql.connector\n", 118 | "from mysql.connector import Error\n", 119 | "import pandas as pd" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "-------------------\n", 127 | "\n", 128 | "### 2. Connect to Server and Create Database\n", 129 | "\n", 130 | "##### 2.1 - Define Server Connection Function \n", 131 | "\n", 132 | "Next we want to define a function in python which connects to our MySQL Server (for info on how to set this up, see [my previous article](https://towardsdatascience.com/coding-and-implementing-a-relational-database-using-mysql-d9bc69be90f5). To do this we use the [mysql.connector.connect()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysql-connector-connect.html) method.\n", 133 | "\n", 134 | "**N.B. Update the variable 'pw' with the root password for your MySQL Server! Otherwise the connection cannot be made.**" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 2, 140 | "metadata": {}, 141 | "outputs": [ 142 | { 143 | "name": "stdout", 144 | "output_type": "stream", 145 | "text": [ 146 | "MySQL Database connection successful\n" 147 | ] 148 | } 149 | ], 150 | "source": [ 151 | "def create_server_connection(host_name, user_name, user_password):\n", 152 | " connection = None\n", 153 | " try:\n", 154 | " connection = mysql.connector.connect(\n", 155 | " host=host_name,\n", 156 | " user=user_name,\n", 157 | " passwd=user_password\n", 158 | " )\n", 159 | " print(\"MySQL Database connection successful\")\n", 160 | " except Error as err:\n", 161 | " print(f\"Error: '{err}'\")\n", 162 | "\n", 163 | " return connection\n", 164 | "\n", 165 | "pw = \"******\" # IMPORTANT! Put your MySQL Terminal password here.\n", 166 | "db = \"school\" # This is the name of the database we will create in the next step - call it whatever you like.\n", 167 | "\n", 168 | "connection = create_server_connection(\"localhost\", \"root\", pw)" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "##### 2.2 - Create a New Database\n", 176 | "\n", 177 | "Now we define a function to create a new database on our server. Here we are using [cursor.execute()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html) to execute a [CREATE DATABASE](https://dev.mysql.com/doc/refman/8.0/en/creating-database.html) SQL command." 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": 3, 183 | "metadata": {}, 184 | "outputs": [ 185 | { 186 | "name": "stdout", 187 | "output_type": "stream", 188 | "text": [ 189 | "Database created successfully\n" 190 | ] 191 | } 192 | ], 193 | "source": [ 194 | "def create_database(connection, query):\n", 195 | " cursor = connection.cursor()\n", 196 | " try:\n", 197 | " cursor.execute(query)\n", 198 | " print(\"Database created successfully\")\n", 199 | " except Error as err:\n", 200 | " print(f\"Error: '{err}'\")\n", 201 | "\n", 202 | "create_database_query = \"CREATE DATABASE school\"\n", 203 | "create_database(connection, create_database_query)" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "metadata": {}, 209 | "source": [ 210 | "##### 2.3 - Modify Server Connection Function, Create Database Connection Function\n", 211 | "\n", 212 | "Now that we've created a DB, let's modify our create_server_connection function to create a new function for connecting directly to that DB. This will prove more useful than just connecting to our server." 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 4, 218 | "metadata": {}, 219 | "outputs": [], 220 | "source": [ 221 | "def create_db_connection(host_name, user_name, user_password, db_name):\n", 222 | " connection = None\n", 223 | " try:\n", 224 | " connection = mysql.connector.connect(\n", 225 | " host=host_name,\n", 226 | " user=user_name,\n", 227 | " passwd=user_password,\n", 228 | " database=db_name\n", 229 | " )\n", 230 | " print(\"MySQL Database connection successful\")\n", 231 | " except Error as err:\n", 232 | " print(f\"Error: '{err}'\")\n", 233 | "\n", 234 | " return connection" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": {}, 240 | "source": [ 241 | "##### 2.4 - Define Query Execution Function\n", 242 | "\n", 243 | "The final step of this stage is to create a function which will allow us to execute queries written in SQL. This is going to be extremely useful!\n", 244 | "\n", 245 | "Again, we use [cursor.execute()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html) to execute our commands." 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": 5, 251 | "metadata": {}, 252 | "outputs": [], 253 | "source": [ 254 | "def execute_query(connection, query):\n", 255 | " cursor = connection.cursor()\n", 256 | " try:\n", 257 | " cursor.execute(query)\n", 258 | " connection.commit()\n", 259 | " print(\"Query successful\")\n", 260 | " except Error as err:\n", 261 | " print(f\"Error: '{err}'\")" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "-------------------\n", 269 | "\n", 270 | "### 3. Creating Tables\n", 271 | "\n", 272 | "##### 3.1 - Create Teacher Table\n", 273 | "\n", 274 | "Now let's create our first table inside our DB, using our newly defined functions." 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": 6, 280 | "metadata": {}, 281 | "outputs": [ 282 | { 283 | "name": "stdout", 284 | "output_type": "stream", 285 | "text": [ 286 | "MySQL Database connection successful\n", 287 | "Query successful\n" 288 | ] 289 | } 290 | ], 291 | "source": [ 292 | "# Assign our SQL command to a python variable using triple quotes to create a multi-line string\n", 293 | "create_teacher_table = \"\"\"\n", 294 | "CREATE TABLE teacher (\n", 295 | " teacher_id INT PRIMARY KEY,\n", 296 | " first_name VARCHAR(40) NOT NULL,\n", 297 | " last_name VARCHAR(40) NOT NULL,\n", 298 | " language_1 VARCHAR(3) NOT NULL,\n", 299 | " language_2 VARCHAR(3),\n", 300 | " dob DATE,\n", 301 | " tax_id INT UNIQUE,\n", 302 | " phone_no VARCHAR(20)\n", 303 | " );\n", 304 | " \"\"\"\n", 305 | "\n", 306 | "connection = create_db_connection(\"localhost\", \"root\", pw, db) # Connect to the Database\n", 307 | "execute_query(connection, create_teacher_table) # Execute our defined query" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": {}, 313 | "source": [ 314 | "If we check in our MySQL terminal using SHOW TABLES; we see the following:\n", 315 | "\n", 316 | "![MySQL Terminal Screen confirming table creation](./img/01.jpg)\n", 317 | "\n", 318 | "Success! The teacher table has been created.\n", 319 | "\n", 320 | "##### 3.2 - Create Remaining Tables\n", 321 | "\n", 322 | "Now let's create the rest of our tables." 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": 7, 328 | "metadata": {}, 329 | "outputs": [ 330 | { 331 | "name": "stdout", 332 | "output_type": "stream", 333 | "text": [ 334 | "MySQL Database connection successful\n", 335 | "Query successful\n", 336 | "Query successful\n", 337 | "Query successful\n" 338 | ] 339 | } 340 | ], 341 | "source": [ 342 | "create_client_table = \"\"\"\n", 343 | "CREATE TABLE client (\n", 344 | " client_id INT PRIMARY KEY,\n", 345 | " client_name VARCHAR(40) NOT NULL,\n", 346 | " address VARCHAR(60) NOT NULL,\n", 347 | " industry VARCHAR(20)\n", 348 | ");\n", 349 | " \"\"\"\n", 350 | "\n", 351 | "create_participant_table = \"\"\"\n", 352 | "CREATE TABLE participant (\n", 353 | " participant_id INT PRIMARY KEY,\n", 354 | " first_name VARCHAR(40) NOT NULL,\n", 355 | " last_name VARCHAR(40) NOT NULL,\n", 356 | " phone_no VARCHAR(20),\n", 357 | " client INT\n", 358 | ");\n", 359 | "\"\"\"\n", 360 | "\n", 361 | "create_course_table = \"\"\"\n", 362 | "CREATE TABLE course (\n", 363 | " course_id INT PRIMARY KEY,\n", 364 | " course_name VARCHAR(40) NOT NULL,\n", 365 | " language VARCHAR(3) NOT NULL,\n", 366 | " level VARCHAR(2),\n", 367 | " course_length_weeks INT,\n", 368 | " start_date DATE,\n", 369 | " in_school BOOLEAN,\n", 370 | " teacher INT,\n", 371 | " client INT\n", 372 | ");\n", 373 | "\"\"\"\n", 374 | "\n", 375 | "\n", 376 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 377 | "execute_query(connection, create_client_table)\n", 378 | "execute_query(connection, create_participant_table)\n", 379 | "execute_query(connection, create_course_table)" 380 | ] 381 | }, 382 | { 383 | "cell_type": "markdown", 384 | "metadata": {}, 385 | "source": [ 386 | "##### 3.3 - Define Foreign Key Relationships\n", 387 | "\n", 388 | "Now altering the tables to create Foreign Key relationships (see the accompanying [SQL Tutorial Series](https://towardsdatascience.com/tagged/sql-series) on [Towards Data Science](https://towardsdatascience.com/) for the background on all of this), and creating our final table, takes_course" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": 8, 394 | "metadata": {}, 395 | "outputs": [ 396 | { 397 | "name": "stdout", 398 | "output_type": "stream", 399 | "text": [ 400 | "MySQL Database connection successful\n", 401 | "Query successful\n", 402 | "Query successful\n", 403 | "Query successful\n", 404 | "Query successful\n" 405 | ] 406 | } 407 | ], 408 | "source": [ 409 | "alter_participant = \"\"\"\n", 410 | "ALTER TABLE participant\n", 411 | "ADD FOREIGN KEY(client)\n", 412 | "REFERENCES client(client_id)\n", 413 | "ON DELETE SET NULL;\n", 414 | "\"\"\"\n", 415 | "\n", 416 | "alter_course = \"\"\"\n", 417 | "ALTER TABLE course\n", 418 | "ADD FOREIGN KEY(teacher)\n", 419 | "REFERENCES teacher(teacher_id)\n", 420 | "ON DELETE SET NULL;\n", 421 | "\"\"\"\n", 422 | "\n", 423 | "alter_course_again = \"\"\"\n", 424 | "ALTER TABLE course\n", 425 | "ADD FOREIGN KEY(client)\n", 426 | "REFERENCES client(client_id)\n", 427 | "ON DELETE SET NULL;\n", 428 | "\"\"\"\n", 429 | "\n", 430 | "create_takescourse_table = \"\"\"\n", 431 | "CREATE TABLE takes_course (\n", 432 | " participant_id INT,\n", 433 | " course_id INT,\n", 434 | " PRIMARY KEY(participant_id, course_id),\n", 435 | " FOREIGN KEY(participant_id) REFERENCES participant(participant_id) ON DELETE CASCADE, -- it makes no sense to keep this rtelation when a participant or course is no longer in the system, hence why CASCADE this time\n", 436 | " FOREIGN KEY(course_id) REFERENCES course(course_id) ON DELETE CASCADE\n", 437 | ");\n", 438 | "\"\"\"\n", 439 | "\n", 440 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 441 | "execute_query(connection, alter_participant)\n", 442 | "execute_query(connection, alter_course)\n", 443 | "execute_query(connection, alter_course_again)\n", 444 | "execute_query(connection, create_takescourse_table)\n" 445 | ] 446 | }, 447 | { 448 | "cell_type": "markdown", 449 | "metadata": {}, 450 | "source": [ 451 | "-----------------\n", 452 | "\n", 453 | "### 4. Populate Tables\n", 454 | "\n", 455 | "The next step is to populate our tables with data, starting with the teacher table\n", 456 | "\n", 457 | "##### 4.1 - Populate Teacher Table\n", 458 | "\n", 459 | "Here we again assign a multi-line string with our SQL command to a variable, and then call our create_db_connection and execute_query functions." 460 | ] 461 | }, 462 | { 463 | "cell_type": "code", 464 | "execution_count": 9, 465 | "metadata": {}, 466 | "outputs": [ 467 | { 468 | "name": "stdout", 469 | "output_type": "stream", 470 | "text": [ 471 | "MySQL Database connection successful\n", 472 | "Query successful\n" 473 | ] 474 | } 475 | ], 476 | "source": [ 477 | "pop_teacher = \"\"\"\n", 478 | "INSERT INTO teacher VALUES\n", 479 | "(1, 'James', 'Smith', 'ENG', NULL, '1985-04-20', 12345, '+491774553676'),\n", 480 | "(2, 'Stefanie', 'Martin', 'FRA', NULL, '1970-02-17', 23456, '+491234567890'), \n", 481 | "(3, 'Steve', 'Wang', 'MAN', 'ENG', '1990-11-12', 34567, '+447840921333'),\n", 482 | "(4, 'Friederike', 'Müller-Rossi', 'DEU', 'ITA', '1987-07-07', 45678, '+492345678901'),\n", 483 | "(5, 'Isobel', 'Ivanova', 'RUS', 'ENG', '1963-05-30', 56789, '+491772635467'),\n", 484 | "(6, 'Niamh', 'Murphy', 'ENG', 'IRI', '1995-09-08', 67890, '+491231231232');\n", 485 | "\"\"\"\n", 486 | "\n", 487 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 488 | "execute_query(connection, pop_teacher)" 489 | ] 490 | }, 491 | { 492 | "cell_type": "markdown", 493 | "metadata": {}, 494 | "source": [ 495 | "Let's take a look in MySQL terminal:\n", 496 | "\n", 497 | "![MySQL Terminal Screen confirming table population](./img/02.jpg)\n", 498 | "\n", 499 | "Success! We can see the table has been properly filled with all of our data.\n", 500 | "\n", 501 | "##### 4.2 - Populate Remaining Tables\n", 502 | "\n", 503 | "Now, let's populate the remaining tables." 504 | ] 505 | }, 506 | { 507 | "cell_type": "code", 508 | "execution_count": 10, 509 | "metadata": {}, 510 | "outputs": [ 511 | { 512 | "name": "stdout", 513 | "output_type": "stream", 514 | "text": [ 515 | "MySQL Database connection successful\n", 516 | "Query successful\n", 517 | "Query successful\n", 518 | "Query successful\n", 519 | "Query successful\n" 520 | ] 521 | } 522 | ], 523 | "source": [ 524 | "pop_client = \"\"\"\n", 525 | "INSERT INTO client VALUES\n", 526 | "(101, 'Big Business Federation', '123 Falschungstraße, 10999 Berlin', 'NGO'),\n", 527 | "(102, 'eCommerce GmbH', '27 Ersatz Allee, 10317 Berlin', 'Retail'),\n", 528 | "(103, 'AutoMaker AG', '20 Künstlichstraße, 10023 Berlin', 'Auto'),\n", 529 | "(104, 'Banko Bank', '12 Betrugstraße, 12345 Berlin', 'Banking'),\n", 530 | "(105, 'WeMoveIt GmbH', '138 Arglistweg, 10065 Berlin', 'Logistics');\n", 531 | "\"\"\"\n", 532 | "\n", 533 | "pop_participant = \"\"\"\n", 534 | "INSERT INTO participant VALUES\n", 535 | "(101, 'Marina', 'Berg','491635558182', 101),\n", 536 | "(102, 'Andrea', 'Duerr', '49159555740', 101),\n", 537 | "(103, 'Philipp', 'Probst', '49155555692', 102),\n", 538 | "(104, 'René', 'Brandt', '4916355546', 102),\n", 539 | "(105, 'Susanne', 'Shuster', '49155555779', 102),\n", 540 | "(106, 'Christian', 'Schreiner', '49162555375', 101),\n", 541 | "(107, 'Harry', 'Kim', '49177555633', 101),\n", 542 | "(108, 'Jan', 'Nowak', '49151555824', 101),\n", 543 | "(109, 'Pablo', 'Garcia', '49162555176', 101),\n", 544 | "(110, 'Melanie', 'Dreschler', '49151555527', 103),\n", 545 | "(111, 'Dieter', 'Durr', '49178555311', 103),\n", 546 | "(112, 'Max', 'Mustermann', '49152555195', 104),\n", 547 | "(113, 'Maxine', 'Mustermann', '49177555355', 104),\n", 548 | "(114, 'Heiko', 'Fleischer', '49155555581', 105);\n", 549 | "\"\"\"\n", 550 | "\n", 551 | "pop_course = \"\"\"\n", 552 | "INSERT INTO course VALUES\n", 553 | "(12, 'English for Logistics', 'ENG', 'A1', 10, '2020-02-01', TRUE, 1, 105),\n", 554 | "(13, 'Beginner English', 'ENG', 'A2', 40, '2019-11-12', FALSE, 6, 101),\n", 555 | "(14, 'Intermediate English', 'ENG', 'B2', 40, '2019-11-12', FALSE, 6, 101),\n", 556 | "(15, 'Advanced English', 'ENG', 'C1', 40, '2019-11-12', FALSE, 6, 101),\n", 557 | "(16, 'Mandarin für Autoindustrie', 'MAN', 'B1', 15, '2020-01-15', TRUE, 3, 103),\n", 558 | "(17, 'Français intermédiaire', 'FRA', 'B1', 18, '2020-04-03', FALSE, 2, 101),\n", 559 | "(18, 'Deutsch für Anfänger', 'DEU', 'A2', 8, '2020-02-14', TRUE, 4, 102),\n", 560 | "(19, 'Intermediate English', 'ENG', 'B2', 10, '2020-03-29', FALSE, 1, 104),\n", 561 | "(20, 'Fortgeschrittenes Russisch', 'RUS', 'C1', 4, '2020-04-08', FALSE, 5, 103);\n", 562 | "\"\"\"\n", 563 | "\n", 564 | "pop_takescourse = \"\"\"\n", 565 | "INSERT INTO takes_course VALUES\n", 566 | "(101, 15),\n", 567 | "(101, 17),\n", 568 | "(102, 17),\n", 569 | "(103, 18),\n", 570 | "(104, 18),\n", 571 | "(105, 18),\n", 572 | "(106, 13),\n", 573 | "(107, 13),\n", 574 | "(108, 13),\n", 575 | "(109, 14),\n", 576 | "(109, 15),\n", 577 | "(110, 16),\n", 578 | "(110, 20),\n", 579 | "(111, 16),\n", 580 | "(114, 12),\n", 581 | "(112, 19),\n", 582 | "(113, 19);\n", 583 | "\"\"\"\n", 584 | "\n", 585 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 586 | "execute_query(connection, pop_client)\n", 587 | "execute_query(connection, pop_participant)\n", 588 | "execute_query(connection, pop_course)\n", 589 | "execute_query(connection, pop_takescourse)" 590 | ] 591 | }, 592 | { 593 | "cell_type": "markdown", 594 | "metadata": {}, 595 | "source": [ 596 | "--------------\n", 597 | "\n", 598 | "### 5. Reading Data\n", 599 | "\n", 600 | "##### 5.1 - Define Data Reading Function\n", 601 | "\n", 602 | "Now that we have populated our tables, it's time to start creating read queries. To do this, we will need a new function." 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": 11, 608 | "metadata": {}, 609 | "outputs": [], 610 | "source": [ 611 | "def read_query(connection, query):\n", 612 | " cursor = connection.cursor()\n", 613 | " result = None\n", 614 | " try:\n", 615 | " cursor.execute(query)\n", 616 | " result = cursor.fetchall()\n", 617 | " return result\n", 618 | " except Error as err:\n", 619 | " print(f\"Error: '{err}'\")" 620 | ] 621 | }, 622 | { 623 | "cell_type": "markdown", 624 | "metadata": {}, 625 | "source": [ 626 | "##### 5.2 - Read Data from Database\n", 627 | "\n", 628 | "Let's try this with a simple query to begin with." 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "execution_count": 12, 634 | "metadata": {}, 635 | "outputs": [ 636 | { 637 | "name": "stdout", 638 | "output_type": "stream", 639 | "text": [ 640 | "MySQL Database connection successful\n", 641 | "(1, 'James', 'Smith', 'ENG', None, datetime.date(1985, 4, 20), 12345, '+491774553676')\n", 642 | "(2, 'Stefanie', 'Martin', 'FRA', None, datetime.date(1970, 2, 17), 23456, '+491234567890')\n", 643 | "(3, 'Steve', 'Wang', 'MAN', 'ENG', datetime.date(1990, 11, 12), 34567, '+447840921333')\n", 644 | "(4, 'Friederike', 'Müller-Rossi', 'DEU', 'ITA', datetime.date(1987, 7, 7), 45678, '+492345678901')\n", 645 | "(5, 'Isobel', 'Ivanova', 'RUS', 'ENG', datetime.date(1963, 5, 30), 56789, '+491772635467')\n", 646 | "(6, 'Niamh', 'Murphy', 'ENG', 'IRI', datetime.date(1995, 9, 8), 67890, '+491231231232')\n" 647 | ] 648 | } 649 | ], 650 | "source": [ 651 | "q1 = \"\"\"\n", 652 | "SELECT *\n", 653 | "FROM teacher;\n", 654 | "\"\"\"\n", 655 | "\n", 656 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 657 | "results = read_query(connection, q1)\n", 658 | "\n", 659 | "for result in results:\n", 660 | " print(result)" 661 | ] 662 | }, 663 | { 664 | "cell_type": "markdown", 665 | "metadata": {}, 666 | "source": [ 667 | "And here are some more queries to try." 668 | ] 669 | }, 670 | { 671 | "cell_type": "code", 672 | "execution_count": 13, 673 | "metadata": {}, 674 | "outputs": [ 675 | { 676 | "name": "stdout", 677 | "output_type": "stream", 678 | "text": [ 679 | "MySQL Database connection successful\n", 680 | "(13, 'Beginner English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin')\n", 681 | "(14, 'Intermediate English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin')\n", 682 | "(15, 'Advanced English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin')\n", 683 | "(17, 'Français intermédiaire', 'FRA', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin')\n", 684 | "(19, 'Intermediate English', 'ENG', 'Banko Bank', '12 Betrugstraße, 12345 Berlin')\n", 685 | "(20, 'Fortgeschrittenes Russisch', 'RUS', 'AutoMaker AG', '20 Künstlichstraße, 10023 Berlin')\n" 686 | ] 687 | } 688 | ], 689 | "source": [ 690 | "q2 = \"\"\"\n", 691 | "SELECT last_name, dob\n", 692 | "FROM teacher;\n", 693 | "\"\"\"\n", 694 | "\n", 695 | "q3 = \"\"\"\n", 696 | "SELECT *\n", 697 | "FROM course\n", 698 | "WHERE language = 'ENG'\n", 699 | "ORDER BY start_date DESC;\n", 700 | "\"\"\"\n", 701 | "\n", 702 | "q4 = \"\"\"\n", 703 | "SELECT first_name, last_name, phone_no\n", 704 | "FROM teacher\n", 705 | "WHERE dob < '1990-01-01';\n", 706 | "\"\"\"\n", 707 | "\n", 708 | "q5 = \"\"\"\n", 709 | "SELECT course.course_id, course.course_name, course.language, client.client_name, client.address\n", 710 | "FROM course\n", 711 | "JOIN client\n", 712 | "ON course.client = client.client_id\n", 713 | "WHERE course.in_school = FALSE;\n", 714 | "\"\"\"\n", 715 | "\n", 716 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 717 | "results = read_query(connection, q5)\n", 718 | "\n", 719 | "for result in results:\n", 720 | " print(result)" 721 | ] 722 | }, 723 | { 724 | "cell_type": "markdown", 725 | "metadata": {}, 726 | "source": [ 727 | "##### 5.3 - Formatting Output into a List\n", 728 | "\n", 729 | "Now we can assign the results to a list, to use further in our python applications or scripts.\n", 730 | "\n", 731 | "The following code returns the results of our query as a list of tuples." 732 | ] 733 | }, 734 | { 735 | "cell_type": "code", 736 | "execution_count": 14, 737 | "metadata": {}, 738 | "outputs": [ 739 | { 740 | "name": "stdout", 741 | "output_type": "stream", 742 | "text": [ 743 | "[(13, 'Beginner English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'), (14, 'Intermediate English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'), (15, 'Advanced English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'), (17, 'Français intermédiaire', 'FRA', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'), (19, 'Intermediate English', 'ENG', 'Banko Bank', '12 Betrugstraße, 12345 Berlin'), (20, 'Fortgeschrittenes Russisch', 'RUS', 'AutoMaker AG', '20 Künstlichstraße, 10023 Berlin')]\n" 744 | ] 745 | } 746 | ], 747 | "source": [ 748 | "#Initialise empty list\n", 749 | "from_db = []\n", 750 | "\n", 751 | "# Loop over the results and append them into our list, different styles\n", 752 | "\n", 753 | "# Returns a list of tuples\n", 754 | "for result in results:\n", 755 | " result = result\n", 756 | " from_db.append(result)\n", 757 | " \n", 758 | "print(from_db)" 759 | ] 760 | }, 761 | { 762 | "cell_type": "markdown", 763 | "metadata": {}, 764 | "source": [ 765 | "##### 5.4 - Formatting Output into a List of Lists\n", 766 | "\n", 767 | "If we want to, we can make this return a list of lists instead, like so:" 768 | ] 769 | }, 770 | { 771 | "cell_type": "code", 772 | "execution_count": 15, 773 | "metadata": {}, 774 | "outputs": [ 775 | { 776 | "name": "stdout", 777 | "output_type": "stream", 778 | "text": [ 779 | "[[13, 'Beginner English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'], [14, 'Intermediate English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'], [15, 'Advanced English', 'ENG', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'], [17, 'Français intermédiaire', 'FRA', 'Big Business Federation', '123 Falschungstraße, 10999 Berlin'], [19, 'Intermediate English', 'ENG', 'Banko Bank', '12 Betrugstraße, 12345 Berlin'], [20, 'Fortgeschrittenes Russisch', 'RUS', 'AutoMaker AG', '20 Künstlichstraße, 10023 Berlin']]\n" 780 | ] 781 | } 782 | ], 783 | "source": [ 784 | "# Returns a list of lists\n", 785 | "from_db = []\n", 786 | "\n", 787 | "for result in results:\n", 788 | " result = list(result)\n", 789 | " from_db.append(result)\n", 790 | " \n", 791 | "print(from_db)" 792 | ] 793 | }, 794 | { 795 | "cell_type": "markdown", 796 | "metadata": {}, 797 | "source": [ 798 | "##### 5.5 - Formatting Output into a pandas [DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html)\n", 799 | "\n", 800 | "With a little more work (creating a list with our column names), we can create a pandas DataFrame like so:" 801 | ] 802 | }, 803 | { 804 | "cell_type": "code", 805 | "execution_count": 16, 806 | "metadata": {}, 807 | "outputs": [ 808 | { 809 | "data": { 810 | "text/html": [ 811 | "
\n", 812 | "\n", 825 | "\n", 826 | " \n", 827 | " \n", 828 | " \n", 829 | " \n", 830 | " \n", 831 | " \n", 832 | " \n", 833 | " \n", 834 | " \n", 835 | " \n", 836 | " \n", 837 | " \n", 838 | " \n", 839 | " \n", 840 | " \n", 841 | " \n", 842 | " \n", 843 | " \n", 844 | " \n", 845 | " \n", 846 | " \n", 847 | " \n", 848 | " \n", 849 | " \n", 850 | " \n", 851 | " \n", 852 | " \n", 853 | " \n", 854 | " \n", 855 | " \n", 856 | " \n", 857 | " \n", 858 | " \n", 859 | " \n", 860 | " \n", 861 | " \n", 862 | " \n", 863 | " \n", 864 | " \n", 865 | " \n", 866 | " \n", 867 | " \n", 868 | " \n", 869 | " \n", 870 | " \n", 871 | " \n", 872 | " \n", 873 | " \n", 874 | " \n", 875 | " \n", 876 | " \n", 877 | " \n", 878 | " \n", 879 | " \n", 880 | " \n", 881 | " \n", 882 | " \n", 883 | " \n", 884 | " \n", 885 | " \n", 886 | "
course_idcourse_namelanguageclient_nameaddress
013Beginner EnglishENGBig Business Federation123 Falschungstraße, 10999 Berlin
114Intermediate EnglishENGBig Business Federation123 Falschungstraße, 10999 Berlin
215Advanced EnglishENGBig Business Federation123 Falschungstraße, 10999 Berlin
317Français intermédiaireFRABig Business Federation123 Falschungstraße, 10999 Berlin
419Intermediate EnglishENGBanko Bank12 Betrugstraße, 12345 Berlin
520Fortgeschrittenes RussischRUSAutoMaker AG20 Künstlichstraße, 10023 Berlin
\n", 887 | "
" 888 | ], 889 | "text/plain": [ 890 | " course_id course_name language client_name \\\n", 891 | "0 13 Beginner English ENG Big Business Federation \n", 892 | "1 14 Intermediate English ENG Big Business Federation \n", 893 | "2 15 Advanced English ENG Big Business Federation \n", 894 | "3 17 Français intermédiaire FRA Big Business Federation \n", 895 | "4 19 Intermediate English ENG Banko Bank \n", 896 | "5 20 Fortgeschrittenes Russisch RUS AutoMaker AG \n", 897 | "\n", 898 | " address \n", 899 | "0 123 Falschungstraße, 10999 Berlin \n", 900 | "1 123 Falschungstraße, 10999 Berlin \n", 901 | "2 123 Falschungstraße, 10999 Berlin \n", 902 | "3 123 Falschungstraße, 10999 Berlin \n", 903 | "4 12 Betrugstraße, 12345 Berlin \n", 904 | "5 20 Künstlichstraße, 10023 Berlin " 905 | ] 906 | }, 907 | "metadata": {}, 908 | "output_type": "display_data" 909 | } 910 | ], 911 | "source": [ 912 | "# Returns a list of lists and then creates a pandas DataFrame\n", 913 | "from_db = []\n", 914 | "\n", 915 | "for result in results:\n", 916 | " result = list(result)\n", 917 | " from_db.append(result)\n", 918 | "\n", 919 | "\n", 920 | "columns = [\"course_id\", \"course_name\", \"language\", \"client_name\", \"address\"]\n", 921 | "df = pd.DataFrame(from_db, columns=columns)\n", 922 | "\n", 923 | "display(df)" 924 | ] 925 | }, 926 | { 927 | "cell_type": "markdown", 928 | "metadata": {}, 929 | "source": [ 930 | "### 6. Updating Records\n", 931 | "\n", 932 | "Sometimes we will need to update our Database. We can do this very easily using our execute_query function alongside the SQL [UPDATE](https://dev.mysql.com/doc/refman/8.0/en/update.html) statement.\n", 933 | "\n", 934 | "##### 6.1 - Updating Client Address\n", 935 | "\n", 936 | "The School receives notification that the Big Business Federation has moved office, and now they are located at 23 Fingiertweg, 14534 Berlin. We can change that in our database like so:" 937 | ] 938 | }, 939 | { 940 | "cell_type": "code", 941 | "execution_count": 17, 942 | "metadata": {}, 943 | "outputs": [ 944 | { 945 | "name": "stdout", 946 | "output_type": "stream", 947 | "text": [ 948 | "MySQL Database connection successful\n", 949 | "Query successful\n" 950 | ] 951 | } 952 | ], 953 | "source": [ 954 | "update = \"\"\"\n", 955 | "UPDATE client \n", 956 | "SET address = '23 Fingiertweg, 14534 Berlin' \n", 957 | "WHERE client_id = 101;\n", 958 | "\"\"\"\n", 959 | "\n", 960 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 961 | "execute_query(connection, update)" 962 | ] 963 | }, 964 | { 965 | "cell_type": "markdown", 966 | "metadata": {}, 967 | "source": [ 968 | "Let's see if that worked." 969 | ] 970 | }, 971 | { 972 | "cell_type": "code", 973 | "execution_count": 18, 974 | "metadata": {}, 975 | "outputs": [ 976 | { 977 | "name": "stdout", 978 | "output_type": "stream", 979 | "text": [ 980 | "MySQL Database connection successful\n", 981 | "(101, 'Big Business Federation', '23 Fingiertweg, 14534 Berlin', 'NGO')\n" 982 | ] 983 | } 984 | ], 985 | "source": [ 986 | "q1 = \"\"\"\n", 987 | "SELECT *\n", 988 | "FROM client\n", 989 | "WHERE client_id = 101;\n", 990 | "\"\"\"\n", 991 | "\n", 992 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 993 | "results = read_query(connection, q1)\n", 994 | "\n", 995 | "for result in results:\n", 996 | " print(result)" 997 | ] 998 | }, 999 | { 1000 | "cell_type": "markdown", 1001 | "metadata": {}, 1002 | "source": [ 1003 | "Excellent!\n", 1004 | "\n", 1005 | "Notice that as well as using \"WHERE client_id = 101\" in the UPDATE query, we could also have used \"WHERE client_name = 'Big Business Federation'\" or \"WHERE address = '123 Falschungstraße, 10999 Berlin'\" (or even \"WHERE address LIKE '%Falschung%'\"). The important thing is that the WHERE clause allows us to uniquely identify the record we want to update. \n", 1006 | "\n", 1007 | "Running the query without a WHERE clause would update all address records in the table, which is very much not what we wanted to do.\n", 1008 | "\n", 1009 | "-----------------\n", 1010 | "\n", 1011 | "### 7. Deleting Records\n", 1012 | "\n", 1013 | "##### 7.1 - Deleting a Course\n", 1014 | "\n", 1015 | "We can also use our execute_query function to delete records, by using DELETE FROM.\n", 1016 | "\n", 1017 | "Let's try this with our course table. First let's remind ourselves of the courses contained in the table." 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": 19, 1023 | "metadata": {}, 1024 | "outputs": [ 1025 | { 1026 | "name": "stdout", 1027 | "output_type": "stream", 1028 | "text": [ 1029 | "MySQL Database connection successful\n", 1030 | "(12, 'English for Logistics', 'ENG', 'A1', 10, datetime.date(2020, 2, 1), 1, 1, 105)\n", 1031 | "(13, 'Beginner English', 'ENG', 'A2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1032 | "(14, 'Intermediate English', 'ENG', 'B2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1033 | "(15, 'Advanced English', 'ENG', 'C1', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1034 | "(16, 'Mandarin für Autoindustrie', 'MAN', 'B1', 15, datetime.date(2020, 1, 15), 1, 3, 103)\n", 1035 | "(17, 'Français intermédiaire', 'FRA', 'B1', 18, datetime.date(2020, 4, 3), 0, 2, 101)\n", 1036 | "(18, 'Deutsch für Anfänger', 'DEU', 'A2', 8, datetime.date(2020, 2, 14), 1, 4, 102)\n", 1037 | "(19, 'Intermediate English', 'ENG', 'B2', 10, datetime.date(2020, 3, 29), 0, 1, 104)\n", 1038 | "(20, 'Fortgeschrittenes Russisch', 'RUS', 'C1', 4, datetime.date(2020, 4, 8), 0, 5, 103)\n" 1039 | ] 1040 | } 1041 | ], 1042 | "source": [ 1043 | "q1 = \"\"\"\n", 1044 | "SELECT *\n", 1045 | "FROM course;\n", 1046 | "\"\"\"\n", 1047 | "\n", 1048 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1049 | "results = read_query(connection, q1)\n", 1050 | "\n", 1051 | "from_db = []\n", 1052 | "\n", 1053 | "for result in results:\n", 1054 | " print(result)" 1055 | ] 1056 | }, 1057 | { 1058 | "cell_type": "markdown", 1059 | "metadata": {}, 1060 | "source": [ 1061 | "Let's delete the course with course_id 20 - 'Fortgeschrittenes Russisch'. For this we will use the [DELETE](https://dev.mysql.com/doc/refman/8.0/en/delete.html) SQL command." 1062 | ] 1063 | }, 1064 | { 1065 | "cell_type": "code", 1066 | "execution_count": 20, 1067 | "metadata": {}, 1068 | "outputs": [ 1069 | { 1070 | "name": "stdout", 1071 | "output_type": "stream", 1072 | "text": [ 1073 | "MySQL Database connection successful\n", 1074 | "Query successful\n" 1075 | ] 1076 | } 1077 | ], 1078 | "source": [ 1079 | "delete_course = \"\"\"\n", 1080 | "DELETE FROM course WHERE course_id = 20;\n", 1081 | "\"\"\"\n", 1082 | "\n", 1083 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1084 | "execute_query(connection, delete_course)" 1085 | ] 1086 | }, 1087 | { 1088 | "cell_type": "markdown", 1089 | "metadata": {}, 1090 | "source": [ 1091 | "Let's confirm that the course is gone." 1092 | ] 1093 | }, 1094 | { 1095 | "cell_type": "code", 1096 | "execution_count": 21, 1097 | "metadata": {}, 1098 | "outputs": [ 1099 | { 1100 | "name": "stdout", 1101 | "output_type": "stream", 1102 | "text": [ 1103 | "MySQL Database connection successful\n", 1104 | "(12, 'English for Logistics', 'ENG', 'A1', 10, datetime.date(2020, 2, 1), 1, 1, 105)\n", 1105 | "(13, 'Beginner English', 'ENG', 'A2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1106 | "(14, 'Intermediate English', 'ENG', 'B2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1107 | "(15, 'Advanced English', 'ENG', 'C1', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1108 | "(16, 'Mandarin für Autoindustrie', 'MAN', 'B1', 15, datetime.date(2020, 1, 15), 1, 3, 103)\n", 1109 | "(17, 'Français intermédiaire', 'FRA', 'B1', 18, datetime.date(2020, 4, 3), 0, 2, 101)\n", 1110 | "(18, 'Deutsch für Anfänger', 'DEU', 'A2', 8, datetime.date(2020, 2, 14), 1, 4, 102)\n", 1111 | "(19, 'Intermediate English', 'ENG', 'B2', 10, datetime.date(2020, 3, 29), 0, 1, 104)\n" 1112 | ] 1113 | } 1114 | ], 1115 | "source": [ 1116 | "q1 = \"\"\"\n", 1117 | "SELECT *\n", 1118 | "FROM course;\n", 1119 | "\"\"\"\n", 1120 | "\n", 1121 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1122 | "results = read_query(connection, q1)\n", 1123 | "\n", 1124 | "from_db = []\n", 1125 | "\n", 1126 | "for result in results:\n", 1127 | " print(result)" 1128 | ] 1129 | }, 1130 | { 1131 | "cell_type": "markdown", 1132 | "metadata": {}, 1133 | "source": [ 1134 | "It's gone! Good work.\n", 1135 | "\n", 1136 | "##### 7.2 - Restoring the Course\n", 1137 | "\n", 1138 | "Let's put that course back - it's a perfectly good course." 1139 | ] 1140 | }, 1141 | { 1142 | "cell_type": "code", 1143 | "execution_count": 22, 1144 | "metadata": {}, 1145 | "outputs": [ 1146 | { 1147 | "name": "stdout", 1148 | "output_type": "stream", 1149 | "text": [ 1150 | "MySQL Database connection successful\n", 1151 | "Query successful\n", 1152 | "MySQL Database connection successful\n", 1153 | "(12, 'English for Logistics', 'ENG', 'A1', 10, datetime.date(2020, 2, 1), 1, 1, 105)\n", 1154 | "(13, 'Beginner English', 'ENG', 'A2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1155 | "(14, 'Intermediate English', 'ENG', 'B2', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1156 | "(15, 'Advanced English', 'ENG', 'C1', 40, datetime.date(2019, 11, 12), 0, 6, 101)\n", 1157 | "(16, 'Mandarin für Autoindustrie', 'MAN', 'B1', 15, datetime.date(2020, 1, 15), 1, 3, 103)\n", 1158 | "(17, 'Français intermédiaire', 'FRA', 'B1', 18, datetime.date(2020, 4, 3), 0, 2, 101)\n", 1159 | "(18, 'Deutsch für Anfänger', 'DEU', 'A2', 8, datetime.date(2020, 2, 14), 1, 4, 102)\n", 1160 | "(19, 'Intermediate English', 'ENG', 'B2', 10, datetime.date(2020, 3, 29), 0, 1, 104)\n", 1161 | "(20, 'Fortgeschrittenes Russisch', 'RUS', 'C1', 4, datetime.date(2020, 4, 8), 0, 5, 103)\n" 1162 | ] 1163 | } 1164 | ], 1165 | "source": [ 1166 | "restore_russian = \"\"\"\n", 1167 | "INSERT INTO course VALUES\n", 1168 | "(20, 'Fortgeschrittenes Russisch', 'RUS', 'C1', 4, '2020-04-08', FALSE, 5, 103);\n", 1169 | "\"\"\"\n", 1170 | "\n", 1171 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1172 | "execute_query(connection, restore_russian)\n", 1173 | "\n", 1174 | "\n", 1175 | "q1 = \"\"\"\n", 1176 | "SELECT *\n", 1177 | "FROM course;\n", 1178 | "\"\"\"\n", 1179 | "\n", 1180 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1181 | "results = read_query(connection, q1)\n", 1182 | "\n", 1183 | "from_db = []\n", 1184 | "\n", 1185 | "for result in results:\n", 1186 | " print(result)" 1187 | ] 1188 | }, 1189 | { 1190 | "cell_type": "markdown", 1191 | "metadata": {}, 1192 | "source": [ 1193 | "Excellent.\n", 1194 | "\n", 1195 | "This also works with DROP TABLE if we want to get rid of a whole table at once. We won't do that in this tutorial, but feel free to try it yourself. In this notebook we can always create and populate the table again - in a real production environment we will need to be much more careful with the DELETE and DROP operations. \n", 1196 | "\n", 1197 | "And always remember to back up our database!\n", 1198 | "\n", 1199 | "----------------\n", 1200 | "\n", 1201 | "### 8. Creating Records from Lists\n", 1202 | "\n", 1203 | "We saw in Section 4 that we can use the SQL [INSERT](https://dev.mysql.com/doc/refman/8.0/en/insert.html) command in our execute_query function to insert records into our tables.\n", 1204 | "\n", 1205 | "MySQL Connector also gives us a way to do this in a more 'pythonic' fashion, using a list of tuples as our input, where each tuple contains the data we wish to insert into our table. This is extremely useful for updating our database with data which may have been generated by an application we have written in Python, such as logs of user activity on a social media app, for example.\n", 1206 | "\n", 1207 | "To do this, we will use the [executemany()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-executemany.html) method, instead of the simpler [execute()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html) method we have been using thus far.\n", 1208 | "\n", 1209 | "This method is also more secure if our database is open to our users at any point, as it helps to prevent against [SQL Injection](https://en.wikipedia.org/wiki/SQL_injection) attacks, which can damage or even destroy our whole database.\n", 1210 | "\n", 1211 | "##### 8.1 - Create Execute List Query Function\n", 1212 | "\n", 1213 | "To see how we can do this, let's add a couple of new teachers to our teacher table.\n", 1214 | "\n", 1215 | "Fist let's modify our execute_query function to use [executemany()](https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-executemany.html) and to accept one more argument." 1216 | ] 1217 | }, 1218 | { 1219 | "cell_type": "code", 1220 | "execution_count": 23, 1221 | "metadata": {}, 1222 | "outputs": [], 1223 | "source": [ 1224 | "def execute_list_query(connection, sql, val):\n", 1225 | " cursor = connection.cursor()\n", 1226 | " try:\n", 1227 | " cursor.executemany(sql, val)\n", 1228 | " connection.commit()\n", 1229 | " print(\"Query successful\")\n", 1230 | " except Error as err:\n", 1231 | " print(f\"Error: '{err}'\")" 1232 | ] 1233 | }, 1234 | { 1235 | "cell_type": "markdown", 1236 | "metadata": {}, 1237 | "source": [ 1238 | "##### 8.2 - Add New Teachers\n", 1239 | "\n", 1240 | "Now let's create a list containing the data for our new teachers (each stored within a [tuple](https://www.w3schools.com/python/python_tuples.asp)), and the SQL command to perform our action.\n", 1241 | "\n", 1242 | "Notice that the SQL command requires a '%s' placeholder for each of the columns we wish to act upon, so in this case we need 8 for the 8 columns we wish to add values for." 1243 | ] 1244 | }, 1245 | { 1246 | "cell_type": "code", 1247 | "execution_count": 24, 1248 | "metadata": {}, 1249 | "outputs": [ 1250 | { 1251 | "name": "stdout", 1252 | "output_type": "stream", 1253 | "text": [ 1254 | "MySQL Database connection successful\n", 1255 | "Query successful\n" 1256 | ] 1257 | } 1258 | ], 1259 | "source": [ 1260 | "sql = '''\n", 1261 | " INSERT INTO teacher (teacher_id, first_name, last_name, language_1, language_2, dob, tax_id, phone_no) \n", 1262 | " VALUES (%s, %s, %s, %s, %s, %s, %s, %s)\n", 1263 | " '''\n", 1264 | "val = [\n", 1265 | " (7, 'Hank', 'Dodson', 'ENG', None, '1991-12-23', 11111, '+491772345678'), \n", 1266 | " (8, 'Sue', 'Perkins', 'MAN', 'ENG', '1976-02-02', 22222, '+491443456432')\n", 1267 | "]\n", 1268 | "\n", 1269 | "\n", 1270 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1271 | "execute_list_query(connection, sql, val)" 1272 | ] 1273 | }, 1274 | { 1275 | "cell_type": "code", 1276 | "execution_count": 25, 1277 | "metadata": {}, 1278 | "outputs": [ 1279 | { 1280 | "name": "stdout", 1281 | "output_type": "stream", 1282 | "text": [ 1283 | "MySQL Database connection successful\n", 1284 | "(1, 'James', 'Smith', 'ENG', None, datetime.date(1985, 4, 20), 12345, '+491774553676')\n", 1285 | "(2, 'Stefanie', 'Martin', 'FRA', None, datetime.date(1970, 2, 17), 23456, '+491234567890')\n", 1286 | "(3, 'Steve', 'Wang', 'MAN', 'ENG', datetime.date(1990, 11, 12), 34567, '+447840921333')\n", 1287 | "(4, 'Friederike', 'Müller-Rossi', 'DEU', 'ITA', datetime.date(1987, 7, 7), 45678, '+492345678901')\n", 1288 | "(5, 'Isobel', 'Ivanova', 'RUS', 'ENG', datetime.date(1963, 5, 30), 56789, '+491772635467')\n", 1289 | "(6, 'Niamh', 'Murphy', 'ENG', 'IRI', datetime.date(1995, 9, 8), 67890, '+491231231232')\n", 1290 | "(7, 'Hank', 'Dodson', 'ENG', None, datetime.date(1991, 12, 23), 11111, '+491772345678')\n", 1291 | "(8, 'Sue', 'Perkins', 'MAN', 'ENG', datetime.date(1976, 2, 2), 22222, '+491443456432')\n" 1292 | ] 1293 | } 1294 | ], 1295 | "source": [ 1296 | "q1 = \"\"\"\n", 1297 | "SELECT *\n", 1298 | "FROM teacher;\n", 1299 | "\"\"\"\n", 1300 | "\n", 1301 | "connection = create_db_connection(\"localhost\", \"root\", pw, db)\n", 1302 | "results = read_query(connection, q1)\n", 1303 | "\n", 1304 | "from_db = []\n", 1305 | "\n", 1306 | "for result in results:\n", 1307 | " print(result)" 1308 | ] 1309 | }, 1310 | { 1311 | "cell_type": "markdown", 1312 | "metadata": {}, 1313 | "source": [ 1314 | "Welcome to the ILS, Hank and Sue!\n", 1315 | "\n", 1316 | "This method can allow us to create new records in our database (or read, update or delete existing records) using a python list as our input. It is difficult to overstate how useful this can be when we are working with Python and SQL together.\n", 1317 | "\n", 1318 | "--------------------\n", 1319 | "\n", 1320 | "### 9. Conclusion\n", 1321 | "\n", 1322 | "##### 9.1 - Conclusion\n", 1323 | "\n", 1324 | "We have covered a lot of ground in this tutorial. From using Python and MySQL Connector to create an entirely new database in MySQL Server, creating tables, defining their relationships to one another and populating them with data. We have covered how to [Create, Read, Update and Delete](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) data in our database.\n", 1325 | "\n", 1326 | "We have looked at how to extract data from existing databases and load them into pandas DataFrames, ready for analysis and further work taking advantage of all the possibilities offered by the [PyData stack](https://www.pluralsight.com/guides/a-lap-around-the-pydata-stack). Going the other direction, we have also learned how to take data generated by our Python scripts and applications, and write those into a database where they can be safely stored for later retrieval.\n", 1327 | "\n", 1328 | "I hope it is clear just how powerful each of these programming languages can be for Data Analysts. Using them together makes them even stronger.\n", 1329 | "\n", 1330 | "----------------------------\n", 1331 | "\n", 1332 | "[![CraigDoesData][logo]][link]\n", 1333 | "\n", 1334 | "[logo]: ./img/logo.png\n", 1335 | "[link]: https://www.craigdoesdata.de/" 1336 | ] 1337 | } 1338 | ], 1339 | "metadata": { 1340 | "kernelspec": { 1341 | "display_name": "Python 3", 1342 | "language": "python", 1343 | "name": "python3" 1344 | }, 1345 | "language_info": { 1346 | "codemirror_mode": { 1347 | "name": "ipython", 1348 | "version": 3 1349 | }, 1350 | "file_extension": ".py", 1351 | "mimetype": "text/x-python", 1352 | "name": "python", 1353 | "nbconvert_exporter": "python", 1354 | "pygments_lexer": "ipython3", 1355 | "version": "3.7.4" 1356 | } 1357 | }, 1358 | "nbformat": 4, 1359 | "nbformat_minor": 4 1360 | } 1361 | --------------------------------------------------------------------------------