├── DSU.png ├── README.md ├── machine_learning_workshop └── titanic_survival.ipynb ├── python_bootcamp_2020 ├── README.md ├── banner.jpg ├── smart_questioning_guidelines.md ├── week_1 │ ├── README.md │ ├── day_1 │ │ ├── Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11 │ │ │ ├── Untitled 1.png │ │ │ ├── Untitled 10.png │ │ │ ├── Untitled 11.png │ │ │ ├── Untitled 2.png │ │ │ ├── Untitled 3.png │ │ │ ├── Untitled 4.png │ │ │ ├── Untitled 5.png │ │ │ ├── Untitled 6.png │ │ │ ├── Untitled 7.png │ │ │ ├── Untitled 8.png │ │ │ ├── Untitled 9.png │ │ │ └── Untitled.png │ │ └── README.md │ ├── day_2 │ │ ├── .ipynb_checkpoints │ │ │ └── bootcamp_day_2 -checkpoint.ipynb │ │ ├── Lecture Notes ab4cded2c8ea400186b447dadfc415bf │ │ │ ├── Untitled 1.png │ │ │ └── Untitled.png │ │ ├── README.md │ │ ├── bootcamp_day_2 .ipynb │ │ └── debug.log │ └── day_3 │ │ └── README.md ├── week_2 │ ├── Assignment 26ad56b85d694c8397104cf0eac24689 │ │ ├── New_Project_(13).png │ │ ├── Untitled 1.png │ │ └── Untitled.png │ ├── README.md │ ├── day_1 │ │ ├── Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec │ │ │ ├── Untitled 1.png │ │ │ ├── Untitled 2.png │ │ │ └── Untitled.png │ │ ├── README.md │ │ ├── read.py │ │ ├── read_csv.py │ │ ├── read_dict.py │ │ ├── sample.csv │ │ ├── walk.py │ │ ├── walk_exclude.py │ │ ├── write.py │ │ ├── write_dict.py │ │ └── write_file.py │ ├── day_2 │ │ ├── Lecture Notes 13778c78ac534a93aa4b6a6c8104467c │ │ │ ├── New_Project_(11).png │ │ │ ├── New_Project_(12).png │ │ │ ├── Untitled 1.png │ │ │ ├── Untitled 2.png │ │ │ ├── Untitled 3.png │ │ │ └── Untitled.png │ │ ├── README.md │ │ ├── quotes.csv │ │ └── quotes_scraper.py │ └── day_3 │ │ ├── AJAX.ipynb │ │ ├── README.md │ │ ├── lecture notes - 2 3 b111b8ff3373478abf32962786aa1994 │ │ ├── header.png │ │ ├── inspect.png │ │ └── parameters.png │ │ └── sessions.ipynb ├── week_3 │ ├── README.md │ ├── day_1 │ │ ├── Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a │ │ │ ├── Untitled 1.png │ │ │ ├── Untitled.png │ │ │ └── image_(2).png │ │ ├── README.md │ │ ├── Untitled.ipynb │ │ └── react_calc.py │ ├── day_2 │ │ ├── Lecture Notes 8078ffb6e5f84399b2d64c93984199b4 │ │ │ ├── Untitled 1.png │ │ │ ├── Untitled 2.png │ │ │ ├── Untitled 3.png │ │ │ ├── Untitled 4.png │ │ │ └── Untitled.png │ │ ├── README.md │ │ ├── fb_script.py │ │ └── soundcloud_bot.ipynb │ └── day_3 │ │ ├── Lecture Notes 121d9a943593432e8769f200aa9621cf │ │ └── Untitled.png │ │ ├── README.md │ │ ├── fb_bot.ipynb │ │ ├── soundcloud_pause.ipynb │ │ ├── soundcloud_script_EC.py │ │ └── soundcloud_script_sleep.py └── week_4 │ ├── README.md │ ├── day_2 │ ├── MongoDB Atlas Configuration │ │ ├── README.md │ │ └── Screenshots │ │ │ ├── 1.png │ │ │ ├── 10.png │ │ │ ├── 11.png │ │ │ ├── 12.png │ │ │ ├── 13.png │ │ │ ├── 14.png │ │ │ ├── 2.png │ │ │ ├── 3.png │ │ │ ├── 4.png │ │ │ ├── 5.png │ │ │ ├── 6.png │ │ │ ├── 7.png │ │ │ ├── 8.png │ │ │ └── 9.png │ ├── README.md │ ├── main.py │ └── templates │ │ ├── Home.html │ │ ├── Login.html │ │ ├── Register.html │ │ └── includes │ │ ├── _formhelpers.html │ │ └── _messages.html │ └── day_3 │ ├── README.md │ └── presentation.pdf └── version_control_with_git └── lecture_notes.md /DSU.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/DSU.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU 2 | 3 |

4 | 5 |

6 | Main repository containing all relevant documents and codebase for Developer Student Club at DSU powered by Google Developers 7 | 8 | ## Sessions 9 | 10 | - [Version Control With Git](#version-control-with-git) 11 | - [Machine Learning Workshop](#machine-learning-workshop) 12 | - [Python Bootcamp 2020](#python-bootcamp-2020) 13 | 14 | ## Resources 15 | 16 | - ## Version Control With Git 17 | 18 | - [Lecture notes](version_control_with_git/lecture_notes.md) 19 | - [Youtube playlist](https://www.youtube.com/watch?v=oMntxxpwDqA&list=PLLyazdzLgFw7YKIV7tPhwruafPl388OoN) 20 | 21 | - ## Machine Learning Workshop 22 | 23 | - [Notebook](machine_learning_workshop/titanic_survival.ipynb) 24 | - [Youtube playlist](https://www.youtube.com/watch?v=2GfOG5AO0rc) 25 | - [Workshop Slides](https://docs.google.com/presentation/d/1MF5V6WfjW88ipWDzUKJ1IKoAkKEIn3u3rvJwSBAqIuY/edit#slide=id.ga0a280794d_0_15) 26 | 27 | - ## Python Bootcamp 2020 28 | - [YouTube playlist](https://www.youtube.com/playlist?list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&fbclid=IwAR0NGiQMWnytKUn9Je5BP4BWro-hcTxoEbuw8nNNpbECKGKFG6et6yz4-co) 29 | - [Lecture notes](python_bootcamp_2020/README.md) 30 | -------------------------------------------------------------------------------- /python_bootcamp_2020/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | Repository to hold DSC@DSU Python Bootcamp 2020 resources. 5 | 6 | ## Lecture Notes 7 | 8 | | Date | Week | Topics | Lecture Notes | Recorded YouTube Link | 9 | | :------------ | ---: | :-----------------------------------------------: | :--------------------------------: | ----------------------------------------------------------------------------------------------------------- | 10 | | 4th Dec 2020 | 1 | Installation, IDEs, runtime | [Lecture Notes 1.1](week_1/day_1/) | [Youtube 1.1](https://youtu.be/DmSf5VO43Ns) | 11 | | 5th Dec 2020 | | PEP8, datatypes, scope, loops, conditions | [Lecture Notes 1.2](week_1/day_2/) | [Youtube 1.2](https://youtu.be/NDaaIrbczy0) | 12 | | 6th Dec 2020 | | \_\_name\_\_ variable, Dicts, lists, collections | [Lecture Notes 1.3](week_1/day_3/) | [Youtube 1.3](https://youtu.be/G5rQ4u0psOg) | 13 | | | | | | | 14 | | 11th Dec 2020 | 2 | File handling, CSV reading and writing | [Lecture Notes 2.1](week_2/day_1) | [Youtube 2.1](https://youtu.be/pdzKoAYcGv8) | 15 | | 12th Dec 2020 | | Web Scraping with `requests` and `BeautifulSoup4` | [Lecture Notes 2.2](week_2/day_2) | [Youtube 2.2](https://youtu.be/I8eHzbSNEiY) | 16 | | 13th Dec 2020 | | Handling web sessions and cookies | [Lecture Notes 2.3](week_2/day_3) | [Youtube 2.3](https://youtu.be/dZhU5smYl0I) | 17 | | | | | | | 18 | | 18th Dec 2020 | 3 | Web crawlers and bots using Selenium | [Lecture Notes 3.1](week_3/day_1) | [Youtube 3.1](https://www.youtube.com/watch?v=yTWhLgR5RNc&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=7) | 19 | | 19th Dec 2020 | | Interacting with websites using Bots | [Lecture Notes 3.2](week_3/day_2) | [Youtube 3.2](https://www.youtube.com/watch?v=dN1DyMJf9hs&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=8) | 20 | | 20th Dec 2020 | | Creating Facebook bot | [Lecture Notes 3.3](week_3/day_3) | [Youtube 3.3](https://www.youtube.com/watch?v=aZJJCbG7gXA&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=9) | 21 | | | | | | | 22 | | 25th Dec 2020 | 4 | Desktop apps with PyQt w/ Shakeeb Siddiqui | On their way | [Youtube 4.1](https://www.youtube.com/watch?v=i9_Pxff9Yy4&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=10) | 23 | | 26th Dec 2020 | | Creating basic web apps with Flask w/ Taha Khan | [Lecture Notes 4.2](week_4/day_2) | [Youtube 4.2](https://www.youtube.com/watch?v=ZKC8LrrOUaY&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=11) | 24 | | 27th Dec 2020 | | Roadmap ahead | [Lecture Notes 4.3](week_4/day_3) | [Youtube 4.3](https://www.youtube.com/watch?v=nbrRA9coJF8&list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&index=12) | 25 | 26 | ## Instructors 27 | 28 | 29 | 30 | 44 | 58 | 59 |
31 | 32 | 37 |
38 | Tarun Kumar 39 |
40 |
41 | Lead Instructor 42 |
43 |
45 | 46 | 51 |
52 | Bahawal Baloch 53 |
54 |
55 | Co-Lead Instructor 56 |
57 |
60 | 61 | ## [Smart Questioning Guidelines](smart_questioning_guidelines.md) 62 | 63 | ## Resources 64 | 65 | - [YouTube playlist](https://www.youtube.com/playlist?list=PLLyazdzLgFw4SsqxcJzmoKZ94juVjEJZG&fbclid=IwAR0NGiQMWnytKUn9Je5BP4BWro-hcTxoEbuw8nNNpbECKGKFG6et6yz4-co) 66 | - [Facebook page](https://www.facebook.com/DeveloperStudentClubDHASuffaUniversity) 67 | - [DHA Suffa University DSC Chapter](https://dsc.community.dev/dha-suffa-university/) 68 | -------------------------------------------------------------------------------- /python_bootcamp_2020/banner.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/banner.jpg -------------------------------------------------------------------------------- /python_bootcamp_2020/smart_questioning_guidelines.md: -------------------------------------------------------------------------------- 1 | # Smart Questioning Guidelines 2 | 3 | The key to proper learning is not studying, rather practicing and if you're practicing correctly you should be running into glitches, errors, crashes and weird corner cases. 4 | 5 | While we as instructors are always present to help you out of such cases, it's mandatory that you have gone through the following steps first, otherwise your question may not be answered or taken seriously. 6 | 7 | 1. Have either attended all the sessions or watched all the prior recordings 8 | 1. If the issue is related with Git/Github, you must've watched [Tarun's Git Series](https://www.youtube.com/watch?v=oMntxxpwDqA&list=PLLyazdzLgFw7YKIV7tPhwruafPl388OoN) 9 | 2. Have tried to Google the exact scenario you're in. Most of the times pasting in exact error yields valuable results. 10 | 3. Have tried asking on the main WhatsApp group in the format mentioned below and no one has answered you. 11 | 4. If the question is assignment related, there's longer than 4 hours left in submission of the assignment 12 | 13 | Now asking questions that may yield better responses is an art too. Google doesn't always give you perfect results and that's because you're asking questions like "program no run" which are inherently dumb, have little context and no information on what you tried to do. 14 | 15 | Fortunately, there's a simple fix for that. Before asking any question, be it on any platform, try to have the following: 16 | 17 | 1. Copy and paste the entire error message and your code to https://pastebin.com/ or just send the code file over 18 | 2. Explain what you’ve already tried to do to solve your problem. This tells people you’ve already put in some work to figure things out on your own. 19 | 3. Specify the point at which the error happens. Does it occur at the very start of the program or only after you do a certain action or line of code? 20 | 21 | Sources: 22 | 23 | - [Al Sweigart, Python course](https://automatetheboringstuff.com/2e/chapter0/) -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | Repository to hold DSC@DSU Python Bootcamp 2020 resources. 5 | 6 | ## Week 1 | Basics (IDE, Datatypes, Structures) 7 | 8 | - ### [Day 1](day_1/) 9 | - Installation 10 | - Runtime environments (`Jupyter Notebook`, `Shell`, `Script`) 11 | - Procedure of Bootcamp 12 | - ### [Day 2](day_2/) 13 | - PEP8 overview, naming conventions 14 | - Primitive datatypes, variable scopes, dynamic typing 15 | - Programming structures (`if`/`else`, `for/while`, `input`, output) 16 | - ### [Day 3](day_3/) 17 | - Functions 18 | - Collection data structures (Dicts, Lists, Tuples) 19 | - Mutability 20 | - Iterating over collections 21 | 22 | ## Assignments 23 | 24 | 1. Define a function to print a string diagonally, for example: 25 | 26 | Tarun will be printed as: 27 | 28 | ![https://s3.us-west-2.amazonaws.com/secure.notion-static.com/ba165be3-eab3-43f3-9399-211180238841/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAT73L2G45O3KS52Y5%2F20201206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20201206T191214Z&X-Amz-Expires=86400&X-Amz-Signature=b85fda20d735dfdfd9ebfd409d440d93bed5391ad5ca4bd1dc594c30fd616c35&X-Amz-SignedHeaders=host&response-content-disposition=filename%20%3D%22Untitled.png%22](https://s3.us-west-2.amazonaws.com/secure.notion-static.com/ba165be3-eab3-43f3-9399-211180238841/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAT73L2G45O3KS52Y5%2F20201206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20201206T191214Z&X-Amz-Expires=86400&X-Amz-Signature=b85fda20d735dfdfd9ebfd409d440d93bed5391ad5ca4bd1dc594c30fd616c35&X-Amz-SignedHeaders=host&response-content-disposition=filename%20%3D%22Untitled.png%22) 29 | 30 | Optional (recommended): 31 | 32 | - Try printing reverse diagonal as well. 33 | 34 |
35 | 2. Create a program to take as input 5 student records in the following format: 36 | 37 | ``` 38 | **roll_num** | **name** | **age** | **marks**(out of 100) 39 | ``` 40 | 41 | And then output the records in a tabular form with class average, class highest and class lowest at end in the following format. 42 | 43 | - Use dictionaries (list of dictionaries in exact) 44 | - Insert atleast 5 records 45 | - Input must be user-given 46 | - (Optional) validate the user input, i.e marks aren't greater 100 and other such validations you think there might be 47 | 48 |
49 |
50 | 51 | 3. A function that will print lyrics of given song with 1 second delay between each line. 52 | 53 | - Use time.sleep() 54 | - Use split() function of string 55 | 56 | Note: 57 | 58 | - You can use main function to predefine anything but make sure you divide everything you can in functions. 59 | - Submit your github folder link and your output on Google Classroom 60 | - Make sure you fork [https://github.com/sinnytk/Python-Bootcamp-DSC-DSU](https://github.com/sinnytk/Python-Bootcamp-DSC-DSU), clone the repo locally and create your scripts in the respective week's folder and push it. 61 | - Make sure in each folder there's a [readme.md](http://readme.md) file with code output and brief explanation of what you did and why you did it (not necessary, but good practice) 62 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 10.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 11.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 2.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 3.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 4.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 5.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 6.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 7.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 8.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled 9.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_1/Lecture Notes 06f7e20d1f064197ad83d6ed5d75fc11/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_1/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 |

4 | 5 | # Python Bootcamp | DSC@DSU 6 | 7 | ## Week 1, Day 1 8 | 9 | ### Index 10 | 11 | - [Python Installation](<###\ Python\ Installation>) 12 | - [Path Configuration (Windows only)](<###\ Path\ Configuration\ (Windows\ only)>) 13 | - [What is PIP?](<###\ What\ is\ PIP?>) 14 | - [How does Python code run?](<###\ How\ does\ Python\ code\ run?>) 15 | - [Running Python: **Interactive shell (terminal)**](<###\ Running\ Python:\ **Interactive\ shell\ (terminal)**>) 16 | - [Running Python: **Interactive Python notebooks (Jupyter Notebook/Google Colab)**](<###\ Running Python: **Interactive Python notebooks (Jupyter Notebook/Google Colab)**>) 17 | - [Running Python: **Scripts**](<###\ Running\ Python:\ **Scripts**>) 18 | 19 | By _Tarun Kumar_, Python@DSU Lead and Core Team member 20 | 21 | With hopes of imparting my knowledge in a befitting manner to you, we start the bootcamp. The main aim of this series will be to become a source I wish I had when I was stuck in the dive in barrier people usually have when starting something new. So let's get started without any further ado? 22 | 23 | Python holds quite some importance as a language in software development, considering it keeps rocking the "Most Wanted" language developers want to work with in **_Stackoverflow Developer Survey_** of [2020](https://insights.stackoverflow.com/survey/2020#most-loved-dreaded-and-wanted), [2019](https://insights.stackoverflow.com/survey/2019#most-loved-dreaded-and-wanted), [2018](https://insights.stackoverflow.com/survey/2018#most-loved-dreaded-and-wanted) 24 | 25 | And seeing as you came upon here, you're probably one of them. 26 | 27 | [Installing Python](<###\ Python\ Installation\ and\ Setup>) 28 | 29 | ### Python Installation 30 | 31 | Now you might hear people suggesting that you can install Anaconda package as it installs everything that you may require in the near future to learn and work with Python? While that might be true for mathematicians, scientists or less tech-savvy who just want to start running a script without caring much about tinkering a bit to configure stuff, **_hackers_** like us don't do it that way. 32 | 33 | Since OSX comes pre-installed with Python and people using Linux distros already know how to apt-get their packages, I would keep this limited to Windows users. 34 | 35 | 1. To begin installation in a vanilla way, you can download the respective installer from [Download Python | Python.org](https://www.python.org/downloads/) 36 | 2. Run the installer 37 | 38 | 39 | 40 | Remember the path being mentioned here. 41 | 42 | The default install path is in the AppData/ directory of the current Windows user. 43 | 44 | Keep "Add Python 3.8 to PATH" unselected 45 | 46 | 3. Voila, you have Python installed on your Windows machine. 47 | 48 | ### Path Configuration (Windows only) 49 | 50 | However you cannot start programming with Python yet, atleast not on a system-wide level. To do so, you must configure your PATH so that Windows knows what Python is. 51 | 52 | To do so: 53 | 54 | 1. Press windows key 55 | 56 | 2. Search for "environment variables" and open "Edit the system environment variables" 57 | 58 | 59 | 60 | 3. Click the environment variables button: 61 | 62 | 63 | 64 | 4. In the bottom dialog(system variables), click on the variable with name PATH and click edit 65 | 66 | 67 | 68 | 5. Click on new button and paste in the following 69 | 70 | 1) Path where we installed Python in step 2, for me it was: 71 | 72 | ```bash 73 | C:\Users\Tarun\AppData\Local\Programs\Python\Python39 74 | ``` 75 | 76 | 77 | 78 | 2. Same path as above with the addition of **\Scripts** at the end. 79 | 80 | ```bash 81 | C:\Users\Tarun\AppData\Local\Programs\Python\Python39\Scripts 82 | ``` 83 | 84 | 85 | 86 | 6. Voila, you can now run Python in your system. To verify that you've done everything correctly just open a command prompt window or PowerShell window and just type in: 87 | 88 | ```bash 89 | python --version 90 | ``` 91 | 92 | You shall get a response similar to this, 93 | 94 | 95 | 96 | if you don't, make sure you've followed the steps correctly, alternatively you can watch the video for today's lecture. 97 | 98 | ### What is PIP? 99 | 100 | In a modern language, libraries and frameworks allow you to use existing codebase to fast-track your development. You don't necessarily need to reinvent the wheel for most cases if the existing solution has had years of development and is probably faster than whatever you could write. 101 | 102 | So in such cases, you may use existing code that is written and deployed in modules called **_packages_** that you can conveniently import into your own codebase and start using the **_functions, classes_** and **_objects_** that it provides. 103 | 104 | Now to apply a simple interface for the installation and configuration of such packages, Python uses a package manager called **_pip_** which comes pre-installed with a Python installation. 105 | 106 | To verify **_pip_** is installed: 107 | 108 | ```bash 109 | pip --version 110 | ``` 111 | 112 | Output: 113 | 114 | 115 | 116 | To install any package with pip, you just do: 117 | 118 | ```bash 119 | pip install package-name 120 | ``` 121 | 122 | To find existing packages that people have created for Python, you can check out **[Py**thon **P**ackage **I**ndex](https://pypi.org/) 123 | 124 | ### How does Python code run? 125 | 126 | In the programming world, **Python** belongs to a family of interpreted (dynamically typed) languages like _JavaScript_, _PHP, Ruby, Perl_ etc. 127 | 128 | The advantage of being dynamically typed is that Python does not require the programmer to keep track of variable datatypes and throughout the life of a program, the variable may change not only it's value but also it's datatype. This makes writing code a breeze. 129 | 130 | Now the term used to identify programs written in dynamically typed languages (like Python) is "script" and we'll be using the term from here on out. 131 | 132 | Now to run Python code, we have three options. 133 | 134 | ### Running Python: **Interactive shell (terminal)** 135 | 136 | Simply open a command prompt or terminal window and type in `python` and you'll enter an interactive shell: 137 | 138 | 139 | 140 | This can act as a workbench for you to test your code logic before writing it into a script. For example slicing a string, playing with custom formulas etcetera. 141 | 142 | ## Running Python: **Interactive Python notebooks (Jupyter Notebook/Google Colab)** 143 | 144 | **Jupyter Notebooks** are perhaps one of the most comprehensive way to write manuals, documentation, labs, presentations and visualizations using only Python. 145 | 146 | To get started with using **Jupyter Notebook**, you need to install the **Jupyter** module using **pip** 147 | 148 | Just open CMD/Terminal and write: 149 | 150 | ```bash 151 | pip install jupyter 152 | ``` 153 | 154 | Now this may take a little while to complete, let it since it's downloading some large dependencies. 155 | 156 | Now Jupyter Notebook run on an server environment because it runs a containerized Python kernel which you don't need to confuse yourself with, so just remember to run a local instance of Jupyter Notebook you need to type the following in terminal after installing (using command shown above): 157 | 158 | ```bash 159 | jupyter notebook 160 | ``` 161 | 162 | This'll launch a [localhost](http://localhost) server and you can view it at [http://localhost:8888/tree](http://localhost:8888/tree) on your browser (running the above command opens it directly) 163 | 164 | Now just create a new file: 165 | 166 | 167 | 168 | You can start writing code in individual cells and execute by pressing `shift + enter` 169 | 170 | 171 | 172 | The new file you just created will have the extension `.ipynb` which signifies that the file is an Interactive Python Notebook. 173 | 174 | Besides you can use Jupyter Notebooks to write Markdown code as well to provide proper documentation to your presentation. The possibilities are limitless on sharing code that not only you can understand, but you can provide proper documentation so that even a layman may understand what you're trying to do in a code section. 175 | 176 | Take a look at this notebook to see the possibilities: 177 | 178 | [Exploratory Computing with Python](https://nbviewer.jupyter.org/github/mbakker7/exploratory_computing_with_python/blob/master/notebook1_basics_plotting/py_exploratory_comp_1_sol.ipynb) 179 | 180 | If you face the following issue after running `jupyter notebook` on your Windows system: 181 | 182 | 183 | 184 | You can fix it by installing the Visual C++ redistributable files from [here](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads) 185 | 186 | ### Running Python: **Scripts** 187 | 188 | Now this is how you'll be spending most of your time on Python. Building scripts that do stuff without any flashy interactive outputs or markdown to explain code. 189 | 190 | To write Python scripts, you can: 191 | 192 | - use any code editor (Visual Studio Code, Sublime Text, Notepad++ or even Vim if you're badass enough) 193 | - use IDEs (PyCharm, IDLE, Spyder) 194 | 195 | I personally love to work on projects that span through multiple languages and frameworks, so Visual Studio Code works for me due to having the lightness of a code editor and great extension support. 196 | 197 | You can write code on Word or Google Docs for all Python cares (make sure you convert the code to plaintext first though because it may raise syntax errors otherwise :p ). 198 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/.ipynb_checkpoints/bootcamp_day_2 -checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Let's Start\n", 8 | "### Variables\n", 9 | "- dynamically type" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 14, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "# int a = 5 C/C++ or java\n", 19 | "a = '5.0'" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 15, 25 | "metadata": {}, 26 | "outputs": [ 27 | { 28 | "data": { 29 | "text/plain": [ 30 | "str" 31 | ] 32 | }, 33 | "execution_count": 15, 34 | "metadata": {}, 35 | "output_type": "execute_result" 36 | } 37 | ], 38 | "source": [ 39 | "type(a)" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "### Primitive data type\n", 47 | " 1. int\n", 48 | " 2. float\n", 49 | " 3. string(str)\n", 50 | " 4. bool\n" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 1, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "a = 5.0\n", 60 | "print(type(a))\n", 61 | "a = 5 \n", 62 | "print(type(a))" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 24, 68 | "metadata": {}, 69 | "outputs": [ 70 | { 71 | "data": { 72 | "text/plain": [ 73 | "False" 74 | ] 75 | }, 76 | "execution_count": 24, 77 | "metadata": {}, 78 | "output_type": "execute_result" 79 | } 80 | ], 81 | "source": [ 82 | "bool('')" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "### Global vs local scope\n", 90 | "\n", 91 | "In most programming languages, variables, classes and objects (and sometimes functions) have scope where they may be referenced and mutated, Python also has a global and local scope.\n", 92 | "\n", 93 | "- Global variables as their name suggests may be accessed through out the program, however cannot be referenced from a function's local scope.\n", 94 | "- Local variables exist within a function's scope and cannot be accessed outside that function." 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 35, 100 | "metadata": {}, 101 | "outputs": [ 102 | { 103 | "name": "stdout", 104 | "output_type": "stream", 105 | "text": [ 106 | "25\n" 107 | ] 108 | } 109 | ], 110 | "source": [ 111 | "x = 5 \n", 112 | "def func():\n", 113 | " global x\n", 114 | " x = x * 5\n", 115 | " print(x)\n", 116 | "func()" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 41, 122 | "metadata": {}, 123 | "outputs": [ 124 | { 125 | "name": "stdout", 126 | "output_type": "stream", 127 | "text": [ 128 | "x in local local\n", 129 | "x in global local\n" 130 | ] 131 | } 132 | ], 133 | "source": [ 134 | "x = 'global'\n", 135 | "def my_func():\n", 136 | " global x\n", 137 | " x=\"local\"\n", 138 | " print('x in local',x)\n", 139 | "my_func()\n", 140 | "print('x in global',x)" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": {}, 146 | "source": [ 147 | "### Printing in Python" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 57, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "name": "stdout", 157 | "output_type": "stream", 158 | "text": [ 159 | "hello world local\n" 160 | ] 161 | } 162 | ], 163 | "source": [ 164 | "print(f'hello world {x}')" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": 2, 170 | "metadata": {}, 171 | "outputs": [ 172 | { 173 | "name": "stdout", 174 | "output_type": "stream", 175 | "text": [ 176 | "python bootcamp\n" 177 | ] 178 | } 179 | ], 180 | "source": [ 181 | "print ('python '+ 'bootcamp')" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "### Input in python " 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 77, 194 | "metadata": {}, 195 | "outputs": [ 196 | { 197 | "name": "stdout", 198 | "output_type": "stream", 199 | "text": [ 200 | "Enter number:5\n" 201 | ] 202 | } 203 | ], 204 | "source": [ 205 | "x = float(input('Enter number:'))" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "### Casting types" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 3, 218 | "metadata": {}, 219 | "outputs": [ 220 | { 221 | "name": "stdout", 222 | "output_type": "stream", 223 | "text": [ 224 | "5\n" 225 | ] 226 | } 227 | ], 228 | "source": [ 229 | "a = 5.0 \n", 230 | "a = int(a)\n", 231 | "print(a)" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": 5, 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "name": "stdout", 241 | "output_type": "stream", 242 | "text": [ 243 | "582\n", 244 | "582.0\n" 245 | ] 246 | } 247 | ], 248 | "source": [ 249 | "b = '582'\n", 250 | "print(int(b))\n", 251 | "print(float(b))" 252 | ] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "### Swapping variables " 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": 86, 264 | "metadata": {}, 265 | "outputs": [], 266 | "source": [ 267 | "a = 5\n", 268 | "b = 4" 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": 96, 274 | "metadata": {}, 275 | "outputs": [], 276 | "source": [ 277 | "a , b = b , a " 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": 97, 283 | "metadata": {}, 284 | "outputs": [ 285 | { 286 | "name": "stdout", 287 | "output_type": "stream", 288 | "text": [ 289 | "8 9\n" 290 | ] 291 | } 292 | ], 293 | "source": [ 294 | "print(a,b)" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "## Control Structure\n", 302 | "### if else\n" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 6, 308 | "metadata": {}, 309 | "outputs": [ 310 | { 311 | "name": "stdout", 312 | "output_type": "stream", 313 | "text": [ 314 | "enter a:5\n", 315 | "enter b:8\n", 316 | "a is greater than b\n" 317 | ] 318 | } 319 | ], 320 | "source": [ 321 | "a = input('enter a:')\n", 322 | "b = input('enter b:')\n", 323 | "if a > b :\n", 324 | " print('a is greater than b')\n", 325 | "elif b > a :\n", 326 | " print('b is greater than a')\n", 327 | "else:\n", 328 | " print('both are equal')" 329 | ] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": {}, 334 | "source": [ 335 | "## Loops\n", 336 | "- for \n", 337 | "- while \n", 338 | "### loop keywords \n", 339 | "- break \n", 340 | "- continue \n", 341 | "### range function\n", 342 | "- first let's look at range function! \n" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": 8, 348 | "metadata": {}, 349 | "outputs": [ 350 | { 351 | "name": "stdout", 352 | "output_type": "stream", 353 | "text": [ 354 | "0-1-2-3-4-5-6-7-8-9-10-11-12-13-14-15-16-17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-33-34-35-36-37-38-39-40-41-42-43-44-45-46-47-48-49-50-51-52-53-54-55-56-57-58-59-60-61-62-63-64-65-66-67-68-69-70-71-72-73-74-75-76-77-78-79-80-81-82-83-84-85-86-87-88-89-90-91-92-93-94-95-96-97-98-99-" 355 | ] 356 | } 357 | ], 358 | "source": [ 359 | "for i in range(100):\n", 360 | " print(i,end='-')" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": 126, 366 | "metadata": {}, 367 | "outputs": [ 368 | { 369 | "name": "stdout", 370 | "output_type": "stream", 371 | "text": [ 372 | "1 5 8 6 " 373 | ] 374 | } 375 | ], 376 | "source": [ 377 | "for i in [1,5,8,6]:\n", 378 | " print(i,end=' ')" 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": 135, 384 | "metadata": {}, 385 | "outputs": [ 386 | { 387 | "name": "stdout", 388 | "output_type": "stream", 389 | "text": [ 390 | "p y t " 391 | ] 392 | } 393 | ], 394 | "source": [ 395 | "for i in 'python':\n", 396 | " if i == 'h':\n", 397 | " break\n", 398 | " print(i,end=' ')\n", 399 | " " 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 9, 405 | "metadata": {}, 406 | "outputs": [ 407 | { 408 | "name": "stdout", 409 | "output_type": "stream", 410 | "text": [ 411 | "\n", 412 | "* \n", 413 | "* * \n", 414 | "* * * \n", 415 | "* * * * \n", 416 | "* * * * * \n" 417 | ] 418 | } 419 | ], 420 | "source": [ 421 | "rows=6\n", 422 | "for i in range(rows):\n", 423 | " for j in range(i):\n", 424 | " print('*',end=\" \")\n", 425 | " print(\"\")" 426 | ] 427 | }, 428 | { 429 | "cell_type": "code", 430 | "execution_count": 143, 431 | "metadata": {}, 432 | "outputs": [ 433 | { 434 | "name": "stdout", 435 | "output_type": "stream", 436 | "text": [ 437 | "0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 " 438 | ] 439 | } 440 | ], 441 | "source": [ 442 | "for i in range(50):\n", 443 | " if i==25:\n", 444 | " break\n", 445 | " print(i,end=' ')" 446 | ] 447 | }, 448 | { 449 | "cell_type": "code", 450 | "execution_count": 10, 451 | "metadata": {}, 452 | "outputs": [ 453 | { 454 | "name": "stdout", 455 | "output_type": "stream", 456 | "text": [ 457 | "enter string:585\n", 458 | "it's a palindrome\n" 459 | ] 460 | } 461 | ], 462 | "source": [ 463 | "pf = input(\"enter string:\")\n", 464 | "fp = ''\n", 465 | "for i in range(len(pf)-1,-1,-1):\n", 466 | " fp+=pf[i]\n", 467 | "if(pf==fp):\n", 468 | " print(\"it's a palindrome\")\n", 469 | "else:\n", 470 | " print(\"it's a not palindrome\")" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 157, 476 | "metadata": {}, 477 | "outputs": [ 478 | { 479 | "name": "stdout", 480 | "output_type": "stream", 481 | "text": [ 482 | "it'\n" 483 | ] 484 | } 485 | ], 486 | "source": [ 487 | "print('it\\'')" 488 | ] 489 | }, 490 | { 491 | "cell_type": "code", 492 | "execution_count": 11, 493 | "metadata": {}, 494 | "outputs": [ 495 | { 496 | "data": { 497 | "text/plain": [ 498 | "48" 499 | ] 500 | }, 501 | "execution_count": 11, 502 | "metadata": {}, 503 | "output_type": "execute_result" 504 | } 505 | ], 506 | "source": [ 507 | "len('len will return no. of characters in this string')" 508 | ] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "execution_count": null, 513 | "metadata": {}, 514 | "outputs": [], 515 | "source": [] 516 | } 517 | ], 518 | "metadata": { 519 | "kernelspec": { 520 | "display_name": "Python 3", 521 | "language": "python", 522 | "name": "python3" 523 | }, 524 | "language_info": { 525 | "codemirror_mode": { 526 | "name": "ipython", 527 | "version": 3 528 | }, 529 | "file_extension": ".py", 530 | "mimetype": "text/x-python", 531 | "name": "python", 532 | "nbconvert_exporter": "python", 533 | "pygments_lexer": "ipython3", 534 | "version": "3.7.6" 535 | } 536 | }, 537 | "nbformat": 4, 538 | "nbformat_minor": 4 539 | } 540 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/Lecture Notes ab4cded2c8ea400186b447dadfc415bf/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_2/Lecture Notes ab4cded2c8ea400186b447dadfc415bf/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/Lecture Notes ab4cded2c8ea400186b447dadfc415bf/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_1/day_2/Lecture Notes ab4cded2c8ea400186b447dadfc415bf/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 |

4 | 5 | # Python Bootcamp | DSC@DSU 6 | 7 | ## Week 1, Day 2 8 | 9 | By _Bahawal Baloch_, Python@DSU Co-Lead and Core Team member 10 | 11 | ## Naming Convention 12 | 13 | The standard naming convention to be followed through out the Python developers. This is the PEP8 naming convention, for more detailed code guidelines: [PEP8](https://www.python.org/dev/peps/pep-0008/). 14 | 15 | [Naming Convention Table](https://www.notion.so/8df8f18fd31340d8a65b2abcf0ebdc91) 16 | 17 | ## Variables 18 | 19 | - Dynamically typed 20 | 21 | ```python 22 | x = "Hello world" 23 | x = 1.25 24 | x = 5 25 | ``` 26 | 27 | - No need to initialize variables with data types, like so: 28 | 29 | ```cpp 30 | int x = 5; 31 | ``` 32 | 33 | ### Primitive Datatypes 34 | 35 | 1. int 36 | 2. float 37 | 3. string(str) 38 | 4. bool 39 | 40 | ```python 41 | bool(True) 42 | True 43 | 44 | bool(False) 45 | False 46 | 47 | bool(1) 48 | True 49 | 50 | bool(0) 51 | False 52 | 53 | bool(None) 54 | False 55 | 56 | bool([]) 57 | False 58 | 59 | bool('') 60 | False 61 | ``` 62 | 63 | ### Global vs local scope 64 | 65 | In most programming languages, variables, classes and objects (and sometimes functions) have scope where they may be referenced and mutated, Python also has a global and local scope. 66 | 67 | - Global variables as their name suggests may be accessed through out the program, however cannot be referenced from a function's local scope. 68 | - Local variables exist within a function's scope and cannot be accessed outside that function. 69 | 70 | ```python 71 | x = "global" 72 | def foo(): 73 | print("x inside:", x) 74 | foo() 75 | print("x outside:", x) 76 | # Output 77 | # x inside: global 78 | # x outside: global 79 | ``` 80 | 81 | ```python 82 | x = 5 83 | 84 | def foo(): 85 | x = x * 2 86 | print(x) 87 | 88 | foo() 89 | 90 | # Output 91 | # UnboundLocalError: local variable 'x' referenced before assignment 92 | ``` 93 | 94 | - Using the `global` keyword you can mutate a global variable in a local scope. 95 | 96 | ```python 97 | x = 5 98 | 99 | def foo(): 100 | global x 101 | x = x * 2 102 | print(x) 103 | 104 | foo() 105 | 106 | # Output 107 | # 10 108 | ``` 109 | 110 | ### Printing in Python 111 | 112 | - F-strings 113 | - Format() 114 | 115 | ### Input 116 | 117 | ![Lecture%20Notes%20ab4cded2c8ea400186b447dadfc415bf/Untitled.png](Lecture%20Notes%20ab4cded2c8ea400186b447dadfc415bf/Untitled.png) 118 | 119 | ### Casting types 120 | 121 | ![Lecture%20Notes%20ab4cded2c8ea400186b447dadfc415bf/Untitled%201.png](Lecture%20Notes%20ab4cded2c8ea400186b447dadfc415bf/Untitled%201.png) 122 | 123 | ### Swapping variables without third variable 124 | 125 | In conventional language, you require a intermediate variable to swap variables (unless you involve some integer mathematics) 126 | 127 | ```python 128 | x=1 129 | y=2 130 | 131 | temp = x 132 | x = y 133 | y = temp 134 | # y becomes 1 and x becomes 2 135 | ``` 136 | 137 | ```python 138 | x = 5 139 | y = 8 140 | print(x,y) 141 | x,y = y,x 142 | print(x,y) 143 | ``` 144 | 145 | ## Control Structure 146 | 147 | - `if` 148 | 149 | conditional statement with change the control of program 150 | 151 | if condition is true: 152 | 153 | do this 154 | 155 | - `else` 156 | 157 | else if a block of code which is written after an if statement and only execute if the "if" condition is not satisfied 158 | 159 | - `elif` 160 | 161 | short for else if 162 | 163 | used to make ladder if else condition 164 | 165 | ```python 166 | x = int(input()) 167 | if x > 5 : 168 | print("x is greater than 5") 169 | elif x < 5 : 170 | print("x is less than 5") 171 | else : 172 | print("x is equal to 5") 173 | 174 | ``` 175 | 176 | ## Loops 177 | 178 | - `for` 179 | 180 | - Use case scenario is when number of iterations are **_known_** 181 | 182 | ```python 183 | for i in range(100): #do something 100 times, increment value in i 184 | ``` 185 | 186 | - What does `range` function do? 187 | 188 | A function which generates a list (we'll discuss lists in detail tomorrow) in the format required: _(start, end, step)_ 189 | 190 | start default is 0, step default is 1 191 | 192 | ```python 193 | range(5) 194 | # [0,1,2,3,4] 195 | ``` 196 | 197 | ```python 198 | range(1,5) 199 | # [1,2,3,4] 200 | ``` 201 | 202 | ```python 203 | range(0,5,2): 204 | # [0,2,4] 205 | ``` 206 | 207 | - Iterating over a collection (again, we'll discuss in detail in next lecture) 208 | 209 | ```python 210 | l = [1,2,3,4] 211 | for i in l: 212 | print(i) 213 | 214 | # 1,2,3,4 215 | ``` 216 | 217 | - `while` 218 | 219 | - Use case scenario is when number of iterations are **_unknown_** 220 | 221 | ```python 222 | x = 0 223 | while x < 100: 224 | print(x) 225 | x+=1 226 | 227 | # same output as **for i in range(100)** 228 | ``` 229 | 230 | - `break`, `continue` 231 | 232 | Are control statements to be used when you want to skip an iteration based on some condition or break the loop operation entirely 233 | 234 | - Give some use case of `continue` 235 | 236 | ```python 237 | for letter in 'Python': 238 | if letter == 'h': 239 | continue 240 | print('current letter :',letter) 241 | ``` 242 | 243 | - Give some use case of `break` 244 | 245 | ```python 246 | for letter in 'Python': 247 | if letter == 'h': 248 | break 249 | print('current letter :',letter) 250 | ``` 251 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/bootcamp_day_2 .ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Let's Start\n", 8 | "### Variables\n", 9 | "- dynamically type" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 14, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "# int a = 5 C/C++ or java\n", 19 | "a = '5.0'" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 15, 25 | "metadata": {}, 26 | "outputs": [ 27 | { 28 | "data": { 29 | "text/plain": [ 30 | "str" 31 | ] 32 | }, 33 | "execution_count": 15, 34 | "metadata": {}, 35 | "output_type": "execute_result" 36 | } 37 | ], 38 | "source": [ 39 | "type(a)" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "### Primitive data type\n", 47 | " 1. int\n", 48 | " 2. float\n", 49 | " 3. string(str)\n", 50 | " 4. bool\n" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 1, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "a = 5.0\n", 60 | "print(type(a))\n", 61 | "a = 5 \n", 62 | "print(type(a))" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 24, 68 | "metadata": {}, 69 | "outputs": [ 70 | { 71 | "data": { 72 | "text/plain": [ 73 | "False" 74 | ] 75 | }, 76 | "execution_count": 24, 77 | "metadata": {}, 78 | "output_type": "execute_result" 79 | } 80 | ], 81 | "source": [ 82 | "bool('')" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "### Global vs local scope\n", 90 | "\n", 91 | "In most programming languages, variables, classes and objects (and sometimes functions) have scope where they may be referenced and mutated, Python also has a global and local scope.\n", 92 | "\n", 93 | "- Global variables as their name suggests may be accessed through out the program, however cannot be referenced from a function's local scope.\n", 94 | "- Local variables exist within a function's scope and cannot be accessed outside that function." 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 35, 100 | "metadata": {}, 101 | "outputs": [ 102 | { 103 | "name": "stdout", 104 | "output_type": "stream", 105 | "text": [ 106 | "25\n" 107 | ] 108 | } 109 | ], 110 | "source": [ 111 | "x = 5 \n", 112 | "def func():\n", 113 | " global x\n", 114 | " x = x * 5\n", 115 | " print(x)\n", 116 | "func()" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 41, 122 | "metadata": {}, 123 | "outputs": [ 124 | { 125 | "name": "stdout", 126 | "output_type": "stream", 127 | "text": [ 128 | "x in local local\n", 129 | "x in global local\n" 130 | ] 131 | } 132 | ], 133 | "source": [ 134 | "x = 'global'\n", 135 | "def my_func():\n", 136 | " global x\n", 137 | " x=\"local\"\n", 138 | " print('x in local',x)\n", 139 | "my_func()\n", 140 | "print('x in global',x)" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": {}, 146 | "source": [ 147 | "### Printing in Python" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 57, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "name": "stdout", 157 | "output_type": "stream", 158 | "text": [ 159 | "hello world local\n" 160 | ] 161 | } 162 | ], 163 | "source": [ 164 | "print(f'hello world {x}')" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": 2, 170 | "metadata": {}, 171 | "outputs": [ 172 | { 173 | "name": "stdout", 174 | "output_type": "stream", 175 | "text": [ 176 | "python bootcamp\n" 177 | ] 178 | } 179 | ], 180 | "source": [ 181 | "print ('python '+ 'bootcamp')" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "### Input in python " 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 77, 194 | "metadata": {}, 195 | "outputs": [ 196 | { 197 | "name": "stdout", 198 | "output_type": "stream", 199 | "text": [ 200 | "Enter number:5\n" 201 | ] 202 | } 203 | ], 204 | "source": [ 205 | "x = float(input('Enter number:'))" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "### Casting types" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 3, 218 | "metadata": {}, 219 | "outputs": [ 220 | { 221 | "name": "stdout", 222 | "output_type": "stream", 223 | "text": [ 224 | "5\n" 225 | ] 226 | } 227 | ], 228 | "source": [ 229 | "a = 5.0 \n", 230 | "a = int(a)\n", 231 | "print(a)" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": 5, 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "name": "stdout", 241 | "output_type": "stream", 242 | "text": [ 243 | "582\n", 244 | "582.0\n" 245 | ] 246 | } 247 | ], 248 | "source": [ 249 | "b = '582'\n", 250 | "print(int(b))\n", 251 | "print(float(b))" 252 | ] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "### Swapping variables " 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": 86, 264 | "metadata": {}, 265 | "outputs": [], 266 | "source": [ 267 | "a = 5\n", 268 | "b = 4" 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": 96, 274 | "metadata": {}, 275 | "outputs": [], 276 | "source": [ 277 | "a , b = b , a " 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": 97, 283 | "metadata": {}, 284 | "outputs": [ 285 | { 286 | "name": "stdout", 287 | "output_type": "stream", 288 | "text": [ 289 | "8 9\n" 290 | ] 291 | } 292 | ], 293 | "source": [ 294 | "print(a,b)" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "## Control Structure\n", 302 | "### if else\n" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 6, 308 | "metadata": {}, 309 | "outputs": [ 310 | { 311 | "name": "stdout", 312 | "output_type": "stream", 313 | "text": [ 314 | "enter a:5\n", 315 | "enter b:8\n", 316 | "a is greater than b\n" 317 | ] 318 | } 319 | ], 320 | "source": [ 321 | "a = input('enter a:')\n", 322 | "b = input('enter b:')\n", 323 | "if a > b :\n", 324 | " print('a is greater than b')\n", 325 | "elif b > a :\n", 326 | " print('b is greater than a')\n", 327 | "else:\n", 328 | " print('both are equal')" 329 | ] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": {}, 334 | "source": [ 335 | "## Loops\n", 336 | "- for \n", 337 | "- while \n", 338 | "### loop keywords \n", 339 | "- break \n", 340 | "- continue \n", 341 | "### range function\n", 342 | "- first let's look at range function! \n" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": 8, 348 | "metadata": {}, 349 | "outputs": [ 350 | { 351 | "name": "stdout", 352 | "output_type": "stream", 353 | "text": [ 354 | "0-1-2-3-4-5-6-7-8-9-10-11-12-13-14-15-16-17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-33-34-35-36-37-38-39-40-41-42-43-44-45-46-47-48-49-50-51-52-53-54-55-56-57-58-59-60-61-62-63-64-65-66-67-68-69-70-71-72-73-74-75-76-77-78-79-80-81-82-83-84-85-86-87-88-89-90-91-92-93-94-95-96-97-98-99-" 355 | ] 356 | } 357 | ], 358 | "source": [ 359 | "for i in range(100):\n", 360 | " print(i,end='-')" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": 126, 366 | "metadata": {}, 367 | "outputs": [ 368 | { 369 | "name": "stdout", 370 | "output_type": "stream", 371 | "text": [ 372 | "1 5 8 6 " 373 | ] 374 | } 375 | ], 376 | "source": [ 377 | "for i in [1,5,8,6]:\n", 378 | " print(i,end=' ')" 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": 135, 384 | "metadata": {}, 385 | "outputs": [ 386 | { 387 | "name": "stdout", 388 | "output_type": "stream", 389 | "text": [ 390 | "p y t " 391 | ] 392 | } 393 | ], 394 | "source": [ 395 | "for i in 'python':\n", 396 | " if i == 'h':\n", 397 | " break\n", 398 | " print(i,end=' ')\n", 399 | " " 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 9, 405 | "metadata": {}, 406 | "outputs": [ 407 | { 408 | "name": "stdout", 409 | "output_type": "stream", 410 | "text": [ 411 | "\n", 412 | "* \n", 413 | "* * \n", 414 | "* * * \n", 415 | "* * * * \n", 416 | "* * * * * \n" 417 | ] 418 | } 419 | ], 420 | "source": [ 421 | "rows=6\n", 422 | "for i in range(rows):\n", 423 | " for j in range(i):\n", 424 | " print('*',end=\" \")\n", 425 | " print(\"\")" 426 | ] 427 | }, 428 | { 429 | "cell_type": "code", 430 | "execution_count": 143, 431 | "metadata": {}, 432 | "outputs": [ 433 | { 434 | "name": "stdout", 435 | "output_type": "stream", 436 | "text": [ 437 | "0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 " 438 | ] 439 | } 440 | ], 441 | "source": [ 442 | "for i in range(50):\n", 443 | " if i==25:\n", 444 | " break\n", 445 | " print(i,end=' ')" 446 | ] 447 | }, 448 | { 449 | "cell_type": "code", 450 | "execution_count": 10, 451 | "metadata": {}, 452 | "outputs": [ 453 | { 454 | "name": "stdout", 455 | "output_type": "stream", 456 | "text": [ 457 | "enter string:585\n", 458 | "it's a palindrome\n" 459 | ] 460 | } 461 | ], 462 | "source": [ 463 | "pf = input(\"enter string:\")\n", 464 | "fp = ''\n", 465 | "for i in range(len(pf)-1,-1,-1):\n", 466 | " fp+=pf[i]\n", 467 | "if(pf==fp):\n", 468 | " print(\"it's a palindrome\")\n", 469 | "else:\n", 470 | " print(\"it's a not palindrome\")" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 157, 476 | "metadata": {}, 477 | "outputs": [ 478 | { 479 | "name": "stdout", 480 | "output_type": "stream", 481 | "text": [ 482 | "it'\n" 483 | ] 484 | } 485 | ], 486 | "source": [ 487 | "print('it\\'')" 488 | ] 489 | }, 490 | { 491 | "cell_type": "code", 492 | "execution_count": 11, 493 | "metadata": {}, 494 | "outputs": [ 495 | { 496 | "data": { 497 | "text/plain": [ 498 | "48" 499 | ] 500 | }, 501 | "execution_count": 11, 502 | "metadata": {}, 503 | "output_type": "execute_result" 504 | } 505 | ], 506 | "source": [ 507 | "len('len will return no. of characters in this string')" 508 | ] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "execution_count": 19, 513 | "metadata": {}, 514 | "outputs": [ 515 | { 516 | "name": "stdout", 517 | "output_type": "stream", 518 | "text": [ 519 | "B B\n", 520 | " a a \n", 521 | " h h \n", 522 | " a a \n", 523 | " w w \n", 524 | " a a \n", 525 | " l l \n", 526 | " \n", 527 | " K K \n", 528 | " h \n", 529 | " a a \n", 530 | " n n \n", 531 | " \n", 532 | " B B \n", 533 | " a a \n", 534 | " l l \n", 535 | " o o \n", 536 | " c c \n", 537 | "h h\n" 538 | ] 539 | } 540 | ], 541 | "source": [ 542 | "N = \"Bahawal Khan Baloch\"\n", 543 | "for i in range(len(N)):\n", 544 | " for j in range(len(N)):\n", 545 | " if (i == j) or ((len(N) - i -1) == j):\n", 546 | " print(N[i], end = '')\n", 547 | " else:\n", 548 | " print(' ', end = '')\n", 549 | " print('')" 550 | ] 551 | }, 552 | { 553 | "cell_type": "code", 554 | "execution_count": 13, 555 | "metadata": {}, 556 | "outputs": [ 557 | { 558 | "name": "stdout", 559 | "output_type": "stream", 560 | "text": [ 561 | "\"Ek Hazar Janen Aisi Mujh pe Sab Fida In Sab Ko Al-Wida Aur Ek Hazar Baten Aisi,\n", 562 | "Sab Lagti Hain Saheeh Lekin Main Manoon Ga Naheen \n", 563 | "Kyunk Kuch Aise Hain Sawal Kih Jin Ke Hain Jawab \n", 564 | "Lafzon Men Mumkin Hi Naheen \n", 565 | "Aur Keh Doon Main Agar Sab Pagal Is Qadar Koyi Mane Ga Naheen \n", 566 | "Bas Ab Dil Khirki Ko Kholo, In Darwazon Ko Kholo Aur yeh Darwaze \n", 567 | "Hain Aise Kih yeh Khul Jaen Agar Toh Band yeh Honge Na Kabhi Oh heart, \n", 568 | "Aur Darya Kuch Hain Aise Jo Nah Bahne The Kabhi Jab Tak Ham Doobe The \n", 569 | "Naheen Aur yeh Zanjeeren Hain Aisi yeh Na Khulni Theen Kabhi Agar Ham Jhukte\n", 570 | "Hi Naheen Aur Manzar yeh Hain Aise yeh Nah Dikhayi Dene The Kabhi \n", 571 | "Andar Ham Ghuste Agar Naheen Bas Ab Dil Khirki Ko Kholo, In Darwazon Ko Bolo Bas Ab Dil Khirki Ko Kholo, \n", 572 | "In Darwazon Ko Kholo Had we not entered within. \n", 573 | "Aur Han Yeh Sahraon Men Cheekho Yeh Wadiyon Men Seekho Kih Ab Dil Khirki Ko Kholo \n", 574 | "Aur Go Ham Lagte Hain Hairan, Akele Pareshan Lekin Ham Khush Hain Meri Jaan,\n", 575 | "Aur Teri Ankhen Bhi Udas, Lambe Bhikre Tere Bal Lekin Yeh Sach Hai Meri Jan \n", 576 | "Kih Phool Khilte Hi Naheen Aur Ham Milte Nah Kabhi Agar Ham Gumte Hi Naheen \n", 577 | "Aur Ek Hazar Janen Aisi Mujh Pe Sab Fida In Sab Ko Mahdi Ma’loof Ka Al-Wida Bas Ab Dil Khirki Ko Kholo, \n", 578 | "In Darwazon Ko Bolo\n" 579 | ] 580 | } 581 | ], 582 | "source": [ 583 | "from time import sleep\n", 584 | "lyrics =\"\"\"\"Ek Hazar Janen Aisi Mujh pe Sab Fida In Sab Ko Al-Wida Aur Ek Hazar Baten Aisi,\n", 585 | "Sab Lagti Hain Saheeh Lekin Main Manoon Ga Naheen \n", 586 | "Kyunk Kuch Aise Hain Sawal Kih Jin Ke Hain Jawab \n", 587 | "Lafzon Men Mumkin Hi Naheen \n", 588 | "Aur Keh Doon Main Agar Sab Pagal Is Qadar Koyi Mane Ga Naheen \n", 589 | "Bas Ab Dil Khirki Ko Kholo, In Darwazon Ko Kholo Aur yeh Darwaze \n", 590 | "Hain Aise Kih yeh Khul Jaen Agar Toh Band yeh Honge Na Kabhi Oh heart, \n", 591 | "Aur Darya Kuch Hain Aise Jo Nah Bahne The Kabhi Jab Tak Ham Doobe The \n", 592 | "Naheen Aur yeh Zanjeeren Hain Aisi yeh Na Khulni Theen Kabhi Agar Ham Jhukte\n", 593 | "Hi Naheen Aur Manzar yeh Hain Aise yeh Nah Dikhayi Dene The Kabhi \n", 594 | "Andar Ham Ghuste Agar Naheen Bas Ab Dil Khirki Ko Kholo, In Darwazon Ko Bolo Bas Ab Dil Khirki Ko Kholo, \n", 595 | "In Darwazon Ko Kholo Had we not entered within. \n", 596 | "Aur Han Yeh Sahraon Men Cheekho Yeh Wadiyon Men Seekho Kih Ab Dil Khirki Ko Kholo \n", 597 | "Aur Go Ham Lagte Hain Hairan, Akele Pareshan Lekin Ham Khush Hain Meri Jaan,\n", 598 | "Aur Teri Ankhen Bhi Udas, Lambe Bhikre Tere Bal Lekin Yeh Sach Hai Meri Jan \n", 599 | "Kih Phool Khilte Hi Naheen Aur Ham Milte Nah Kabhi Agar Ham Gumte Hi Naheen \n", 600 | "Aur Ek Hazar Janen Aisi Mujh Pe Sab Fida In Sab Ko Mahdi Ma’loof Ka Al-Wida Bas Ab Dil Khirki Ko Kholo, \n", 601 | "In Darwazon Ko Bolo\"\"\"\n", 602 | "lines = lyrics.splitlines()\n", 603 | "for i in lines:\n", 604 | " print(i)\n", 605 | " sleep(1)" 606 | ] 607 | }, 608 | { 609 | "cell_type": "code", 610 | "execution_count": null, 611 | "metadata": {}, 612 | "outputs": [], 613 | "source": [] 614 | } 615 | ], 616 | "metadata": { 617 | "kernelspec": { 618 | "display_name": "Python 3", 619 | "language": "python", 620 | "name": "python3" 621 | }, 622 | "language_info": { 623 | "codemirror_mode": { 624 | "name": "ipython", 625 | "version": 3 626 | }, 627 | "file_extension": ".py", 628 | "mimetype": "text/x-python", 629 | "name": "python", 630 | "nbconvert_exporter": "python", 631 | "pygments_lexer": "ipython3", 632 | "version": "3.7.6" 633 | } 634 | }, 635 | "nbformat": 4, 636 | "nbformat_minor": 4 637 | } 638 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_1/day_2/debug.log: -------------------------------------------------------------------------------- 1 | [1208/230824.649:ERROR:directory_reader_win.cc(43)] FindFirstFile: The system cannot find the path specified. (0x3) 2 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/New_Project_(13).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/New_Project_(13).png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/Assignment 26ad56b85d694c8397104cf0eac24689/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | Repository to hold DSC@DSU Python Bootcamp 2020 resources. 5 | 6 | - ## Week 2 | File system, API and web scraping 7 | 8 | - ### [Day 1](day_1/) 9 | 10 | - Opening and handling files with different modes 11 | - `OS` module to handle paths and directories 12 | - Writing, reading structured file formats like CSV 13 | 14 | - ### [Day 2](day_2/) 15 | 16 | - Introduction to HTTP Requests and `requests` library 17 | - Working with Public APIs 18 | - Parsing HTML using `BeautifulSoup4` 19 | - Accessing DOM tree using `CSS selectors` 20 | 21 | - ### [Day 3](day_3/) 22 | - Handling dynamic web content scraping (`AJAX`) 23 | - Using web sessions to handle cookies 24 | - User login, authentication 25 | 26 | ## Assignment 27 | 28 | ### 1. Take as user input a folder path and print all the files in descending order of their respective sizes 29 | 30 | Hints: 31 | 32 | - You can create a tuple for each file (filename, filesize) or perhaps even a dictionary(filename, filesize, root etc). You'll be creating a list of those dictionaries or tuples. 33 | - We have so far discussed sorting a list of primitive types (strings, ints, floats) but how do you go about sorting a list of tuples or dictionaries.. or even a list of lists? How does the sorting function know what list is greater or lesser than the other list? Google it. 34 | 35 | **Optional** : Give option for users to remove these files 36 | 37 | ### 2. Build a Facebook bot that will retrieve the number of likes a Facebook page. Input Facebook page handles using CSVs 38 | 39 | - The script will take as input a CSV with a list of Facebook page handles 40 | 41 | - The handle is FB Page's url, not the name. 42 | 43 | ![Assignment%2026ad56b85d694c8397104cf0eac24689/Untitled.png](Assignment%2026ad56b85d694c8397104cf0eac24689/Untitled.png) 44 | 45 | For example, **DeveloperStudentClubDHASuffaUniversity** 46 | 47 | - And the output will be another CSV (or update original CSV) 48 | 49 | ![Assignment%2026ad56b85d694c8397104cf0eac24689/New_Project_(13).png]() 50 | 51 | ### 3. Build a school scraper that'll scrap 50 schools' info and generate a CSV. 52 | 53 | - Locate the API endpoint of the following education directory 54 | 55 | [https://directory.ntschools.net/#/schools](https://directory.ntschools.net/#/schools) 56 | 57 | - Using BS4 only for the main page and JSON + Requests libraries for interacting with those endpoints 58 | - Produce a well structured CSV with the following fields: 59 | 1. School name 60 | 2. Single lined formatted physical address 61 | 3. Principal/Admin Name 62 | 4. Principal/Admin Position 63 | 5. Principal/Admin Email 64 | 6. School Telephone number 65 | 66 | Sample output: 67 | 68 | ![Assignment%2026ad56b85d694c8397104cf0eac24689/Untitled%201.png](Assignment%2026ad56b85d694c8397104cf0eac24689/Untitled%201.png) 69 | 70 | ### 4 (optional, submission not required): Try accessing your university's dashboard where your assignments and resources are uploaded. Alternatively try accessing any site which requires you to access 71 | 72 | - You can use main function to predefine anything but make sure you divide everything you can in functions. 73 | - Submit your github folder link and your output on Google Classroom 74 | - Make sure you fork [https://github.com/sinnytk/Python-Bootcamp-DSC-DSU](https://github.com/sinnytk/Python-Bootcamp-DSC-DSU), clone the repo locally and create your scripts in the respective week's folder and push it. 75 | - Make sure in each folder there's a [readme.md](http://readme.md) file with code output and brief explanation of what you did and why you did it (not necessary, but good practice) 76 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled 2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled 2.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_1/Lecture Notes 84b61ffb043f4d6f9a0513bee92d74ec/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | 5 | ## Week 2, Day 1 (File system, filing and CSV) 6 | 7 | By _Tarun Kumar_, **Python@DSU Lead** and Core Team **DSC@DSU** 8 | 9 | As we move towards the usefulness of a programming language in real-life applications, filing is one of the important aspect that comes to our mind. 10 | 11 | If you come languages like C/C++, Java or JavaScript even, filing is relatively easier in Python. 12 | 13 | But before we get into handling individual files, let's explore how we interact with an operating system's file system using the `os` module. 14 | 15 | `os` is a builtin Python library that as the name suggests contains various functions and attributes to work with your operating system. 16 | 17 | ### Functions with default parameters 18 | 19 | I forgot to discuss default parameters in the previous lecture notes. Simply, these are parameters that may take a default value for a function if they're not passed through the calling function. 20 | 21 | e.g 22 | 23 | ```bash 24 | >>> def foo(name="Tarun"): 25 | ... print(name) 26 | ... 27 | >>> foo("Kumar") 28 | Kumar 29 | >>> foo() 30 | Tarun 31 | ``` 32 | 33 | ### Working with directories in `os` 34 | 35 | One thing to remember is when you execute any Python script, interactive notebook or shell, the working directory defaults to the folder where the script exists. We can use this working directory to relatively refer to different directories in the file system. 36 | 37 | To start working with the `os` module, we need to import the library using a `import os` statement at top of our script. 38 | 39 | Some cool functions that `os` library has: 40 | 41 | - `os.getcwd()` 42 | 43 | Return a string representing the current working directory. 44 | 45 | ```bash 46 | >>> os.getcwd() 47 | '/home/tarun' 48 | ``` 49 | 50 | - `os.listdir(path=".")` 51 | 52 | Lists all the files and folders of the given path, by default returns the files and folders of current working directory. 53 | 54 | ```bash 55 | >>> os.listdir() 56 | ['week_3', 'smart_questioning_guidelines.md', 57 | 'week_2', 'banner.jpg', 'test.html', 'week_4', 58 | 'overview_timetable.md', 'README.md', 'timetable.md', 'week_1'] 59 | 60 | >>> sorted(os.listdir()) # sorted() function sorts a given list 61 | ['README.md', 'banner.jpg', 'overview_timetable.md', 62 | 'smart_questioning_guidelines.md', 'test.html', 'timetable.md', 63 | 'week_1', 'week_2', 'week_3', 'week_4'] 64 | ``` 65 | 66 | - `os.walk(top=".")` 67 | 68 | Walk a given path in top-down (or bottom-up) fashion. In other words, print all files and folders in root then go into each folder and repeat. 69 | 70 | ```python 71 | for folder, subfolders, filenames in os.walk('/home/tarun/Tarun/DSC@DSU'): 72 | print(f'The current folder is {folder}') 73 | 74 | for subfolder in subfolders: 75 | print(f'SUBFOLDER OF {folder}: {subfolder}') 76 | 77 | for filename in filenames: 78 | print(f'FILE INSIDE {folder} : {filename}') 79 | 80 | print("\n") 81 | ``` 82 | 83 | ```bash 84 | The current folder is /home/tarun/Tarun/DSC@DSU 85 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: Git_assets 86 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: Social Media Kit 87 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: SilabGit 88 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: .ipynb_checkpoints 89 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: github_repos 90 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: Hit-With-Git-Feedback 91 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: dsc-certificate-generator 92 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: notion_lecture_notes_tools 93 | SUBFOLDER OF /home/tarun/Tarun/DSC@DSU: hit_with_git_github 94 | FILE INSIDE /home/tarun/Tarun/DSC@DSU : Frame-34609.png 95 | FILE INSIDE /home/tarun/Tarun/DSC@DSU : Git.osp 96 | FILE INSIDE /home/tarun/Tarun/DSC@DSU : Final.mp4 97 | ``` 98 | 99 | Skipping specific folders or directories in `os.walk`: 100 | 101 | While traversing in topdown fashion(default behavior of os.walk), you can modify the subfolders list **in-place** to filter out that subdirectory from being traversed. 102 | 103 | list[:] ensures we reference the original list instead of assigning the variable with the new list's memory. 104 | 105 | ```bash 106 | >>> lA = [1,2,3] 107 | >>> lB = [4,5,6] 108 | >>> lC = [7,8,9] 109 | >>> lB = lA 110 | >>> lA.append(5) 111 | >>> lB 112 | [1, 2, 3, 5] 113 | >>> lA 114 | [1, 2, 3, 5] 115 | >>> lA = lC 116 | >>> lA 117 | [7, 8, 9] 118 | >>> lB 119 | [1, 2, 3, 5] 120 | 121 | >>> lA = [1,2,3] 122 | >>> lB = [4,5,6] 123 | >>> lC = [7,8,9] 124 | >>> lB = lA 125 | >>> lA.append(5) 126 | >>> lB 127 | [1, 2, 3, 5] 128 | >>> lA 129 | [1, 2, 3, 5] 130 | >>> lA[:] = lC 131 | >>> lA 132 | [7, 8, 9] 133 | >>> lB 134 | [7, 8, 9] 135 | ``` 136 | 137 | ```bash 138 | for folder, subfolders, filenames in os.walk('/home/tarun/Tarun/DSC@DSU'): 139 | subfolders[:] = [subfolder for subfolder in subfolders if subfolder!='.git'] 140 | print(f'The current folder is {folder}') 141 | 142 | for subfolder in subfolders: 143 | print(f'SUBFOLDER OF {folder}: {subfolder}') 144 | 145 | for filename in filenames: 146 | print(f'FILE INSIDE {folder} : {filename}') 147 | 148 | print("\n") 149 | ``` 150 | 151 | Class activity: 152 | 153 | - Sort files based on their extensions 154 | - Use os 155 | 156 | ### Handling different OS paths using `pathlib` 157 | 158 | As you might've noticed from my notes, I primarily use Linux which uses forward slashes `/` as representation of path directory, while Windows systems use back slashes `\` 159 | 160 | - You should use forward slashes with pathlib functions. The Path() object will convert forward slashes into the correct kind of slash for the current operating system. Nice! 161 | 162 | To read up more on the module, I found this Medium article to be helpful: [https://medium.com/@ageitgey/python-3-quick-tip-the-easy-way-to-deal-with-file-paths-on-windows-mac-and-linux-11a072b58d5f](https://medium.com/@ageitgey/python-3-quick-tip-the-easy-way-to-deal-with-file-paths-on-windows-mac-and-linux-11a072b58d5f) 163 | 164 | ### Reading files with `open` 165 | 166 | Reading files in Python is extremely easy, you can open an existing file using a function called `*open(filepath)*` 167 | 168 | ```bash 169 | >>> f = open('timetable.md') 170 | >>> f 171 | <_io.TextIOWrapper name='timetable.md' mode='r' encoding='UTF-8'> 172 | 173 | >>> f = open("timetable2.md") 174 | Traceback (most recent call last): 175 | File "", line 1, in 176 | FileNotFoundError: [Errno 2] No such file or directory: 'timetable2.md' 177 | ``` 178 | 179 | **By default**, the file (if it exists in the current working directory or specified path) opens up in read mode, that is you can only read the file contents but not modify them. 180 | 181 | ### Different file modes 182 | 183 | `"r"` - Read - Default value. Opens a file for reading, error if the file does not exist 184 | 185 | `"a"` - Append - Opens a file for appending, creates the file if it does not exist 186 | 187 | `"w"` - Write - Opens a file for writing, creates the file if it does not exist 188 | 189 | `"x"` - Create - Creates the specified file, returns an error if the file exists, you can also write in the created file, but not read 190 | 191 | Some advanced modes to remember: 192 | 193 | - Read and write the file (but also truncate or create if doesn't exist): `w+` 194 | - Read and write the file (without truncate or creation): `r+` 195 | - Read and write the file (create file but append at last if it exists): `a+` 196 | 197 | ### Reading files 198 | 199 | You can read a file at once using `file.read()` 200 | 201 | **Class activity: do the song example with filing this time.** 202 | 203 | ### CSV Files 204 | 205 | Humans seek structure in whatever they use and CSVs are the easiest and most commonly used flat database storage structure today. 206 | 207 | The magic's in the name. Comma-separated value files are one of the best ways to represent and store tables of data, like so. 208 | 209 | ![Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled.png](Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled.png) 210 | 211 | To view CSV files you can use Excel, LibreOffice Calc, Google Sheets or any other spreadsheet parsing reader. 212 | 213 | Which may look like this: 214 | 215 | ![Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled%201.png](Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled%201.png) 216 | 217 | In raw format, CSV files look like this: 218 | 219 | ![Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled%202.png](Lecture%20Notes%2084b61ffb043f4d6f9a0513bee92d74ec/Untitled%202.png) 220 | 221 | So it means you can write files as you normally would using loops, but this time only appending commas at the end of each separation, right? 222 | 223 | Let's try it. 224 | 225 | ```bash 226 | >>> f = open("sample.csv") 227 | >>> a = f.read() 228 | >>> a 229 | ',Name,Age,Semester\n,Tarun,201,7\n,Bahawal,22,7\n' 230 | 231 | >>> f = open("sample.csv","a") 232 | >>> f.write(",Zain,26,7\n") # file comma is to leave first col empty 233 | 234 | >>> f = open("sample.csv") 235 | >>> a = f.read() 236 | >>> a 237 | ',Name,Age,Semester\n,Tarun,21,7\n,Bahawal,22,7\n,Zain,26,7\n' 238 | ``` 239 | 240 | **Class activity: output multiplication tables** 241 | 242 | ### `with` keyword 243 | 244 | Sometimes we require access to a resource only for single time or for a limited access. Python provides a with keyword to automatically delete resources after they're consumed within a block of a code. 245 | 246 | Like so. 247 | 248 | ### `csv` module 249 | 250 | As common as CSV reading and writing are, Python developers decided to include a builtin module to support and simplify CSV operations using the `csv` module. 251 | 252 | ### Reading CSV files with `csv.reader` 253 | 254 | ```python 255 | import csv 256 | 257 | with open('sample.csv') as csv_file: 258 | csv_reader = csv.reader(csv_file, delimiter=',') 259 | line_count = 0 260 | for row in csv_reader: 261 | if line_count == 0: 262 | print(f'Column names are {row}') 263 | line_count += 1 264 | else: 265 | print( 266 | f'\t{row[0]} is {row[1]} years old, and is currently in semester {row[2]}.') 267 | line_count += 1 268 | ``` 269 | 270 | ```bash 271 | Column names are ['', 'Name', 'Age', 'Semester'] 272 | Tarun is 21 years old, and is currently in semester 7. 273 | Bahawal is 22 years old, and is currently in semester 7. 274 | Zain is 26 years old, and is currently in semester 7. 275 | ``` 276 | 277 | ### Writing csv files with `csv.writer` 278 | 279 | You can also write into CSV files in a similar fashion. 280 | 281 | ```python 282 | import csv 283 | 284 | with open('sample.csv', mode='a') as friends: 285 | friends_writer = csv.writer(friends, delimiter=',') 286 | 287 | friends_writer.writerow(['', 'Hamza Fayyaz', '22', '7']) 288 | friends_writer.writerow(['', 'Syed Ali', '25', '7']) 289 | ``` 290 | 291 | ### Reading and writing into `dict` with `DictReader` and `DictWriter`: 292 | 293 | CSV module also provides classes to read and write using dictionaries, which make for a ton of readability and convenient data pipeline modeling. 294 | 295 | - Using `DictReader` to automatically read columns and create dictionaries of each row: 296 | 297 | ```bash 298 | >>> import csv 299 | >>> with open('sample.csv', mode='r') as friends: 300 | ... friends_reader = csv.DictReader(friends) 301 | ... for friend in friends_reader: 302 | ... print(friend) 303 | ... 304 | {'': '', 'Name': 'Tarun', 'Age': '21', 'Semester': '7'} 305 | {'': '', 'Name': 'Bahawal', 'Age': '22', 'Semester': '7'} 306 | {'': '', 'Name': 'Zain', 'Age': '26', 'Semester': '7'} 307 | {'': '', 'Name': 'Zain', 'Age': '26', 'Semester': '7'} 308 | {'': '', 'Name': 'Hamza Fayyaz', 'Age': '22', 'Semester': '7'} 309 | {'': '', 'Name': 'Syed Ali', 'Age': '25', 'Semester': '7'} 310 | 311 | #first key and value is empty due to empty column 312 | >>> friends_reader.fieldnames 313 | ['', 'Name', 'Age', 'Semester'] 314 | ``` 315 | 316 | - Using `DictWriter` to automatically map a dictionary to corresponding columns 317 | 318 | ```bash 319 | >>> with open('sample.csv', mode='a') as friends: 320 | ... friends_writer = csv.DictWriter( 321 | ... friends, fieldnames=["", "Name", "Age", "Semester"]) 322 | ... friends_writer.writerow( 323 | ... {"Age": "22", "Name": "Maaz Ahmed", "Semester": "7"}) 324 | ... 325 | # Notice how the dict may miss some column names and is order-agnostic 326 | # Even then, the output is correctly written 327 | {'': '', 'Name': 'Tarun', 'Age': '21', 'Semester': '7'} 328 | {'': '', 'Name': 'Bahawal', 'Age': '22', 'Semester': '7'} 329 | {'': '', 'Name': 'Zain', 'Age': '26', 'Semester': '7'} 330 | {'': '', 'Name': 'Zain', 'Age': '26', 'Semester': '7'} 331 | {'': '', 'Name': 'Hamza Fayyaz', 'Age': '22', 'Semester': '7'} 332 | {'': '', 'Name': 'Syed Ali', 'Age': '25', 'Semester': '7'} 333 | {'': '', 'Name': 'Maaz Ahmed', 'Age': '22', 'Semester': '7'} 334 | ``` 335 | 336 | ### Ending notes 337 | 338 | - You can also use Pandas to clear most of the overhead for writing and reading CSVs, you can explore such here: [https://towardsdatascience.com/pandas-dataframe-playing-with-csv-files-944225d19ff](https://towardsdatascience.com/pandas-dataframe-playing-with-csv-files-944225d19ff) 339 | 340 | ### Some practice activities: 341 | 342 | - Try to do all previous assignments using CSVs 343 | - Can you write diagonally like we did in previous assignment in CSV? 344 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/read.py: -------------------------------------------------------------------------------- 1 | import csv 2 | 3 | with open('sample.csv') as csv_file: 4 | csv_reader = csv.reader(csv_file, delimiter=',') 5 | line_count = 0 6 | for row in csv_reader: 7 | if line_count == 0: 8 | print(f'Column names are {row}') 9 | line_count += 1 10 | else: 11 | print( 12 | f'\t{row[1]} is {row[2]} years old, and is currently in semester {row[3]}.') 13 | line_count += 1 14 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/read_csv.py: -------------------------------------------------------------------------------- 1 | import csv 2 | 3 | with open("sample.csv") as sample_file: 4 | reader = csv.reader(sample_file) 5 | for row in reader: 6 | print(row) 7 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/read_dict.py: -------------------------------------------------------------------------------- 1 | import csv 2 | 3 | with open('sample.csv', mode='r') as friends: 4 | friends_reader = csv.DictReader(friends) 5 | for friend in friends_reader: 6 | print(friend["Name"]) 7 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/sample.csv: -------------------------------------------------------------------------------- 1 | ,Name,Age,Semester 2 | ,Tarun,21,7 3 | ,Bahawal,22,7 4 | ,Zain,26,7 5 | ,Zain,26,7 6 | ,Hamza Fayyaz,22,7 7 | ,Syed Ali,25,7 8 | ,Maaz Ahmed,22,7 9 | ,Maaz Ahmed,22,7 10 | ,Maaz Ahmed,22,7 11 | ,Tarun Kumar,21,7 12 | ,Zaid Bin Harris,30,10 13 | ,Zaid Bin Harris,30,10 14 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/walk.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | for folder, subfolders, filenames in os.walk('/home/tarun/Tarun/DSC@DSU'): 4 | print(f'The current folder is {folder}') 5 | 6 | for subfolder in subfolders: 7 | print(f'SUBFOLDER OF {folder}: {subfolder}') 8 | 9 | for filename in filenames: 10 | print(f'FILE INSIDE {folder} : {filename}') 11 | 12 | print("\n") -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/walk_exclude.py: -------------------------------------------------------------------------------- 1 | import os 2 | for dirpath, dirnames, filenames in os.walk('/home/tarun/Tarun/DSC@DSU'): 3 | dirnames[:] = [subfolder for subfolder in dirnames if subfolder!='.git'] 4 | print(f'The current dirpath is {dirpath}') 5 | 6 | for subfolder in dirnames: 7 | print(f'SUBFOLDER OF {dirpath}: {subfolder}') 8 | 9 | for filename in filenames: 10 | print(f'FILE INSIDE {dirpath} : {filename}') 11 | 12 | print("\n") -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/write.py: -------------------------------------------------------------------------------- 1 | import csv 2 | 3 | with open('sample.csv', mode='a') as friends: 4 | friends_writer = csv.writer(friends, delimiter=',') 5 | 6 | friends_writer.writerow(['', 'Hamza Fayyaz', '22', '7']) 7 | friends_writer.writerow(['', 'Syed Ali', '25', '7']) 8 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/write_dict.py: -------------------------------------------------------------------------------- 1 | import csv 2 | 3 | with open('sample.csv', mode='a') as friends: 4 | friends_writer = csv.DictWriter( 5 | friends, fieldnames=["", "Name", "Age", "Semester"]) 6 | friends_writer.writerow( 7 | {"Age": "30", "Name": "Zaid Bin Harris", "Semester": "10"}) 8 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_1/write_file.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | 3 | with open("test.txt") as song_file: 4 | print(song_file.closed) 5 | song_lyrics = song_file.read() 6 | # for line in song_lyrics.split("\n"): 7 | # print(line) 8 | # sleep(1) 9 | 10 | print(song_file.closed) -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/New_Project_(11).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/New_Project_(11).png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/New_Project_(12).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/New_Project_(12).png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 2.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled 3.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_2/Lecture Notes 13778c78ac534a93aa4b6a6c8104467c/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | # Python Bootcamp | DSC@DSU 4 | 5 |

6 | 7 | ## Week 2, Day 2 8 | 9 | By _Tarun Kumar_, **Python@DSU Lead** and Core Team **DSC@DSU** 10 | 11 | Connection with internet is one of the important things for any human in today's time, same goes for the applications and software that you write. 12 | 13 | ### Hypertext Transfer Protocol (HTTP) 14 | 15 | Any connection over the internet is carried out through the **H**yperText **T**ransfer **P**rotocol and as protocols go, they define a set of rules so that communication is easier. 16 | 17 | Think any human communication, we communicate using a verbal protocol and in that verbal protocol there's a language used and that language has some a set of rules called grammar. 18 | 19 | In the same fashion, to communicate over the web, you use HTTP. 20 | 21 | HTTP protocol supports the transfer of any hypermedia (interactive multimedia) in a client-server protocol using **requests**(client request of a resource or status from server) and **responses** (server's response after processing the request). 22 | 23 | To see the different transfers that happen between a client and server, go to any website on your browser and check **_Developer Tools -> Network Tab_** 24 | 25 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled.png](Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled.png) 26 | 27 | You may notice when you click on any one request there, you'll see some header information. 28 | 29 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%201.png](Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%201.png) 30 | 31 | What's important for you know the requested **url, request method** used \***\*and \*\***the **status code** returned. 32 | 33 | In HTTP protcol, there are different request methods which help you categorize the requests and respond from the server accordingly. I won't be getting into too much detail on all the different methods and status codes, for that you can read up this [cheatsheet](https://cheatography.com/kstep/cheat-sheets/http-status-codes/). 34 | 35 | **GET** (retrieving some resource) and **POST** (sending and storing some resource) \*\*\*\*are the commonly used request methods and is what you'll be dealing with on the daily. 36 | 37 | GET gets a resource while with POST you send in resources. 38 | 39 | Now covering HTTP is not the purpose of this bootcamp, so you can read it up on [here](https://www.freecodecamp.org/news/restful-services-part-i-http-in-a-nutshell-aab3bfedd131/) or any other resource you find good. 40 | 41 | ### Using `requests` library 42 | 43 | Similar to JavaScript, if you can think of something there's probably a package or library for it (and I sincerely hope that you guys create a library for the masses one day too). 44 | 45 | `requests` is a wonderful library to use HTTP and interact with APIs and websites and what we'll be using throughout the bootcamp. 46 | 47 | Let's get started on working with it? 48 | 49 | First you'll need to install the library if you haven't already: 50 | 51 | ```bash 52 | pip install requests 53 | ``` 54 | 55 | Then import into your script or shell using 56 | 57 | ```bash 58 | import requests 59 | ``` 60 | 61 | And voila!, you're ready to make Python talk with the internet. 62 | 63 | ### Using `requests` on APIs 64 | 65 | To test out how you can emulate HTTP requests with requests library, let's take out any API from [https://any-api.com/](https://any-api.com/) a repository of open-access APIs. 66 | 67 | I'll be using Discog's API which is a music release database. 68 | 69 | From reading up on their [documentation](https://www.discogs.com/developers/#page:database,header:database-artist), I have access to 25 unregistered requests for 60 minutes. Let's try it out. 70 | 71 | - The base url is: [https://api.discogs.com/](https://api.discogs.com/) 72 | - Then the API provides access to different resources like releases, artists and other entities they have in their database along with the request and response schema. 73 | 74 | Let's try to get all Led Zepplin releases: 75 | 76 | - I found that to get releases, the endpoint is: 77 | - `/artists/{artist_id}/releases{?sort,sort_order}` 78 | - The artist ID can easily be retrieved by Googling "Discog [artist name]" and I found Led Zeppelin's to be `34278` 79 | 80 | Now let's emulate a `GET` on the aforementioned endpoint: 81 | 82 | ```bash 83 | import requests 84 | base_url = "https://api.discogs.com/" 85 | response = requests.get(f"{base_url}/artists/34278/releases") 86 | >>> response.status_code 87 | 200 88 | >>> response.content 89 | b'{"pagination": {"page": 1, "pages": 43, "per_page": 50, "items": 2118, "urls":... 90 | # Response content truncated 91 | 92 | ``` 93 | 94 | Evidently, I get a Python dictionary like response from the API. This type of schema is called **J**avaScript **O**bject **N**otation or more commonly **JSON**. This is how REST APIs communicate with each other, to read up more what REST APIs are and what JSONs are: you can either watch [Mosh's video](https://www.youtube.com/watch?v=SLwpqD8n3d0), read up on this [FreeCodeCamp writeup](https://www.freecodecamp.org/news/rest-api-tutorial-rest-client-rest-service-and-api-calls-explained-with-code-examples/) or Google around to find whatever seems to follow your pace because REST APIs are an important concept modern web development and something every Computer Scientist should be aware of. 95 | 96 | Now we got the response, however as the response is a huge one and not something we can scroll through and understand the schema of, let's use an online tool called [JSON Formatter](https://jsonformatter.org/) which can provide us a good overview of the schema. 97 | 98 | 1. Just open the site jsonformatter.org 99 | 100 | 2. Paste in the response body. 101 | 102 | 3. Click "Beautify" 103 | 104 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%202.png](Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%202.png) 105 | 106 | You can also change the "code" format to "tree" for a better indented view in the right column. 107 | 108 | ### Using `json` to parse API responses 109 | 110 | To convert that response byte string to a local Python dictionary, you can use the `json` module like so. 111 | 112 | ```bash 113 | 114 | >>> import json 115 | >>> data = json.loads(response.content) 116 | >>> data["releases"][7]['title'] 117 | 'Babe I’m Gonna Leave You / Dazed & Confused' 118 | ``` 119 | 120 | ### Scraping website data using `requests` 121 | 122 | Now APIs are a direct communication platform between a website and the server and hence they're heavily structured, however APIs are not something you usually have access to (Discog for example only allowed 25 accesses per hour). 123 | 124 | But what about websites? You don't really have someone stopping you from accessing a website and it's contents right? (unless of course the website requires logging which Bahawal covers in the next session). 125 | 126 | Let's try to do a `GET` on a normal website, my Facebook account: [https://www.facebook.com/sinnytk](https://www.facebook.com/sinnytk) 127 | 128 | ```bash 129 | >>> response = requests.get("https://www.facebook.com/sinnytk") 130 | >>> response.content 131 | b'\n>> with open("sinnytk.html","w") as file: 140 | ... file.write(response.content.decode()) 141 | ... 142 | 353363 143 | ``` 144 | 145 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/New_Project_(11).png]() 146 | 147 | You can see the difference (Urdu language being returned). The reason behind this is that web browsers are capable of executing the included JavaScript scripts in the HTML files, however a normal HTTP GET only retrieves the HTML resource without executing the underlying CSS file linking, JavaScript execution and other cascaded events that normal web browsers can do. 148 | 149 | This result can be more evident by doing the same GET request and saving response content on [https://ahfarmer.github.io/calculator/](https://ahfarmer.github.io/calculator/), a React based single page application which is completely JavaScript generated. 150 | 151 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/New_Project_(12).png]() 152 | 153 | ### Parsing and querying HTML code with `BeautifulSoup` 154 | 155 | Now imagine the previous example of retrieving my Facebook profile. Suppose I want to get my username or alternative name? 156 | 157 | How do I go about it? There are some solutions like: 158 | 159 | - Using Regular Expressions to search for a specific pattern in text 160 | - Use string functions like split or find 161 | 162 | But yet again, we have a cool library called `BeautifulSoup` which parses markup-language files and strings and gives you ability to query it using DOM selection methods like CSS Selectors or XPATH. 163 | 164 | You can read up [CSS Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) and [XPATHs](https://developer.mozilla.org/en-US/docs/Web/XPath) from the Mozilla web docs. 165 | 166 | We'll work mainly on CSS Selectors and in short. 167 | 168 | - To select elements IDs, you use `#` 169 | - To select elements by classes, you use `.` 170 | - And to select elements by attributes, you use `[ ]` 171 | 172 | Using the inspect element, you can notice that my alternative name is described as following: 173 | 174 | ![Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%203.png](Lecture%20Notes%2013778c78ac534a93aa4b6a6c8104467c/Untitled%203.png) 175 | 176 | We can refer to this element by saying: 177 | 178 | "Get me the span tag with class of **alternate_name**" 179 | 180 | And in CSS selectors, that'll be translated to: 181 | 182 | `span.alternate_name` 183 | 184 | or 185 | 186 | `span[class="alternate_name"]` 187 | 188 | Let's do this using BeautifulSoup. 189 | 190 | 1. Install BeautifulSoup library using `pip install beautifulsoup4` 191 | 2. Import the BeautifulSoup parser in your script using `from bs4 import BeautifulSoup` 192 | 3. Send BeautifulSoup class your HTML code and define a parser to use 193 | 4. Voila, you can now use CSS selectors using `select()` and `select_one()` 194 | 195 | ```bash 196 | >>> from bs4 import BeautifulSoup 197 | >>> soup = BeautifulSoup(response.content,'html.parser') 198 | >>> soup.select("span.alternate_name") 199 | [(Sinny)] 200 | >>> soup.select_one("span.alternate_name") 201 | (Sinny) 202 | >>> soup.select_one("span.alternate_name").text 203 | '(Sinny)' 204 | >>> soup.select_one("span.alternate_name").text[1:-1] 205 | 'Sinny' 206 | ``` 207 | 208 | Live example: 209 | 210 | [https://quotes.toscrape.com/](https://quotes.toscrape.com/) 211 | 212 | Note: script also attached 213 | 214 | Some sample projects i did(explained in live session): 215 | 216 | - Allison's project for different sites 217 | 218 | Some caveats to remember: 219 | 220 | - What you see in `***Right Click → View Page Source`\*\*\* is what you'll actually see when you download and parse using Requests and BS4 221 | - BeautifulSoup isn't capable of querying JavaScript code or CSS styles and hence only works on static sites without dynamically loaded content 222 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/quotes.csv: -------------------------------------------------------------------------------- 1 | author,text 2 | Albert Einstein,“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” 3 | J.K. Rowling,"“It is our choices, Harry, that show what we truly are, far more than our abilities.”" 4 | Albert Einstein,“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.” 5 | Jane Austen,"“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”" 6 | Marilyn Monroe,"“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”" 7 | Albert Einstein,“Try not to become a man of success. Rather become a man of value.” 8 | André Gide,“It is better to be hated for what you are than to be loved for what you are not.” 9 | Thomas A. Edison,"“I have not failed. I've just found 10,000 ways that won't work.”" 10 | Eleanor Roosevelt,“A woman is like a tea bag; you never know how strong it is until it's in hot water.” 11 | Steve Martin,"“A day without sunshine is like, you know, night.”" 12 | Marilyn Monroe,"“This life is what you make it. No matter what, you're going to mess up sometimes, it's a universal truth. But the good part is you get to decide how you're going to mess it up. Girls will be your friends - they'll act like it anyway. But just remember, some come, some go. The ones that stay with you through everything - they're your true best friends. Don't let go of them. Also remember, sisters make the best friends in the world. As for lovers, well, they'll come and go too. And baby, I hate to say it, most of them - actually pretty much all of them are going to break your heart, but you can't give up because if you give up, you'll never find your soulmate. You'll never find that half who makes you whole and that goes for everything. Just because you fail once, doesn't mean you're gonna fail at everything. Keep trying, hold on, and always, always, always believe in yourself, because if you don't, then who will, sweetie? So keep your head high, keep your chin up, and most importantly, keep smiling, because life's a beautiful thing and there's so much to smile about.”" 13 | J.K. Rowling,"“It takes a great deal of bravery to stand up to our enemies, but just as much to stand up to our friends.”" 14 | Albert Einstein,"“If you can't explain it to a six year old, you don't understand it yourself.”" 15 | Bob Marley,"“You may not be her first, her last, or her only. She loved before she may love again. But if she loves you now, what else matters? She's not perfect—you aren't either, and the two of you may never be perfect together but if she can make you laugh, cause you to think twice, and admit to being human and making mistakes, hold onto her and give her the most you can. She may not be thinking about you every second of the day, but she will give you a part of her that she knows you can break—her heart. So don't hurt her, don't change her, don't analyze and don't expect more than she can give. Smile when she makes you happy, let her know when she makes you mad, and miss her when she's not there.”" 16 | Dr. Seuss,"“I like nonsense, it wakes up the brain cells. Fantasy is a necessary ingredient in living.”" 17 | Douglas Adams,"“I may not have gone where I intended to go, but I think I have ended up where I needed to be.”" 18 | Elie Wiesel,"“The opposite of love is not hate, it's indifference. The opposite of art is not ugliness, it's indifference. The opposite of faith is not heresy, it's indifference. And the opposite of life is not death, it's indifference.”" 19 | Friedrich Nietzsche,"“It is not a lack of love, but a lack of friendship that makes unhappy marriages.”" 20 | Mark Twain,"“Good friends, good books, and a sleepy conscience: this is the ideal life.”" 21 | Allen Saunders,“Life is what happens to us while we are making other plans.” 22 | Pablo Neruda,"“I love you without knowing how, or when, or from where. I love you simply, without problems or pride: I love you in this way because I do not know any other way of loving but this, in which there is no I or you, so intimate that your hand upon my chest is my hand, so intimate that when I fall asleep your eyes close.”" 23 | Ralph Waldo Emerson,“For every minute you are angry you lose sixty seconds of happiness.” 24 | Mother Teresa,"“If you judge people, you have no time to love them.”" 25 | Garrison Keillor,“Anyone who thinks sitting in church can make you a Christian must also think that sitting in a garage can make you a car.” 26 | Jim Henson,“Beauty is in the eye of the beholder and it may be necessary from time to time to give a stupid or misinformed beholder a black eye.” 27 | Dr. Seuss,"“Today you are You, that is truer than true. There is no one alive who is Youer than You.”" 28 | Albert Einstein,"“If you want your children to be intelligent, read them fairy tales. If you want them to be more intelligent, read them more fairy tales.”" 29 | J.K. Rowling,"“It is impossible to live without failing at something, unless you live so cautiously that you might as well not have lived at all - in which case, you fail by default.”" 30 | Albert Einstein,“Logic will get you from A to Z; imagination will get you everywhere.” 31 | Bob Marley,"“One good thing about music, when it hits you, you feel no pain.”" 32 | Dr. Seuss,"“The more that you read, the more things you will know. The more that you learn, the more places you'll go.”" 33 | J.K. Rowling,"“Of course it is happening inside your head, Harry, but why on earth should that mean that it is not real?”" 34 | Bob Marley,"“The truth is, everyone is going to hurt you. You just got to find the ones worth suffering for.”" 35 | Mother Teresa,“Not all of us can do great things. But we can do small things with great love.” 36 | J.K. Rowling,"“To the well-organized mind, death is but the next great adventure.”" 37 | Charles M. Schulz,“All you need is love. But a little chocolate now and then doesn't hurt.” 38 | William Nicholson,“We read to know we're not alone.” 39 | Albert Einstein,“Any fool can know. The point is to understand.” 40 | Jorge Luis Borges,“I have always imagined that Paradise will be a kind of library.” 41 | George Eliot,“It is never too late to be what you might have been.” 42 | George R.R. Martin,"“A reader lives a thousand lives before he dies, said Jojen. The man who never reads lives only one.”" 43 | C.S. Lewis,“You can never get a cup of tea large enough or a book long enough to suit me.” 44 | Marilyn Monroe,“You believe lies so you eventually learn to trust no one but yourself.” 45 | Marilyn Monroe,"“If you can make a woman laugh, you can make her do anything.”" 46 | Albert Einstein,"“Life is like riding a bicycle. To keep your balance, you must keep moving.”" 47 | Marilyn Monroe,“The real lover is the man who can thrill you by kissing your forehead or smiling into your eyes or just staring into space.” 48 | Marilyn Monroe,"“A wise girl kisses but doesn't love, listens but doesn't believe, and leaves before she is left.”" 49 | Martin Luther King Jr.,“Only in the darkness can you see the stars.” 50 | J.K. Rowling,"“It matters not what someone is born, but what they grow to be.”" 51 | James Baldwin,"“Love does not begin and end the way we seem to think it does. Love is a battle, love is a war; love is a growing up.”" 52 | Jane Austen,"“There is nothing I would not do for those who are really my friends. I have no notion of loving people by halves, it is not my nature.”" 53 | Eleanor Roosevelt,“Do one thing every day that scares you.” 54 | Marilyn Monroe,"“I am good, but not an angel. I do sin, but I am not the devil. I am just a small girl in a big world trying to find someone to love.”" 55 | Albert Einstein,"“If I were not a physicist, I would probably be a musician. I often think in music. I live my daydreams in music. I see my life in terms of music.”" 56 | Haruki Murakami,"“If you only read the books that everyone else is reading, you can only think what everyone else is thinking.”" 57 | Alexandre Dumas fils,“The difference between genius and stupidity is: genius has its limits.” 58 | Stephenie Meyer,"“He's like a drug for you, Bella.”" 59 | Ernest Hemingway,“There is no friend as loyal as a book.” 60 | Helen Keller,"“When one door of happiness closes, another opens; but often we look so long at the closed door that we do not see the one which has been opened for us.”" 61 | George Bernard Shaw,“Life isn't about finding yourself. Life is about creating yourself.” 62 | Charles Bukowski,"“That's the problem with drinking, I thought, as I poured myself a drink. If something bad happens you drink in an attempt to forget; if something good happens you drink in order to celebrate; and if nothing happens you drink to make something happen.”" 63 | Suzanne Collins,“You don’t forget the face of the person who was your last hope.” 64 | Suzanne Collins,"“Remember, we're madly in love, so it's all right to kiss me anytime you feel like it.”" 65 | C.S. Lewis,"“To love at all is to be vulnerable. Love anything and your heart will be wrung and possibly broken. If you want to make sure of keeping it intact you must give it to no one, not even an animal. Wrap it carefully round with hobbies and little luxuries; avoid all entanglements. Lock it up safe in the casket or coffin of your selfishness. But in that casket, safe, dark, motionless, airless, it will change. It will not be broken; it will become unbreakable, impenetrable, irredeemable. To love is to be vulnerable.”" 66 | J.R.R. Tolkien,“Not all those who wander are lost.” 67 | J.K. Rowling,"“Do not pity the dead, Harry. Pity the living, and, above all those who live without love.”" 68 | Ernest Hemingway,“There is nothing to writing. All you do is sit down at a typewriter and bleed.” 69 | Ralph Waldo Emerson,“Finish each day and be done with it. You have done what you could. Some blunders and absurdities no doubt crept in; forget them as soon as you can. Tomorrow is a new day. You shall begin it serenely and with too high a spirit to be encumbered with your old nonsense.” 70 | Mark Twain,“I have never let my schooling interfere with my education.” 71 | Dr. Seuss,“I have heard there are troubles of more than one kind. Some come from ahead and some come from behind. But I've bought a big bat. I'm all ready you see. Now my troubles are going to have troubles with me!” 72 | Alfred Tennyson,“If I had a flower for every time I thought of you...I could walk through my garden forever.” 73 | Charles Bukowski,“Some people never go crazy. What truly horrible lives they must lead.” 74 | Terry Pratchett,"“The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.”" 75 | Dr. Seuss,"“Think left and think right and think low and think high. Oh, the thinks you can think up if only you try!”" 76 | J.D. Salinger,"“What really knocks me out is a book that, when you're all done reading it, you wish the author that wrote it was a terrific friend of yours and you could call him up on the phone whenever you felt like it. That doesn't happen much, though.”" 77 | George Carlin,“The reason I talk to myself is because I’m the only one whose answers I accept.” 78 | John Lennon,"“You may say I'm a dreamer, but I'm not the only one. I hope someday you'll join us. And the world will live as one.”" 79 | W.C. Fields,“I am free of all prejudice. I hate everyone equally. ” 80 | Ayn Rand,“The question isn't who is going to let me; it's who is going to stop me.” 81 | Mark Twain,“′Classic′ - a book which people praise and don't read.” 82 | Albert Einstein,“Anyone who has never made a mistake has never tried anything new.” 83 | Jane Austen,"“A lady's imagination is very rapid; it jumps from admiration to love, from love to matrimony in a moment.”" 84 | J.K. Rowling,"“Remember, if the time should come when you have to make a choice between what is right and what is easy, remember what happened to a boy who was good, and kind, and brave, because he strayed across the path of Lord Voldemort. Remember Cedric Diggory.”" 85 | Jane Austen,"“I declare after all there is no enjoyment like reading! How much sooner one tires of any thing than of a book! -- When I have a house of my own, I shall be miserable if I have not an excellent library.”" 86 | Jane Austen,"“There are few people whom I really love, and still fewer of whom I think well. The more I see of the world, the more am I dissatisfied with it; and every day confirms my belief of the inconsistency of all human characters, and of the little dependence that can be placed on the appearance of merit or sense.”" 87 | C.S. Lewis,“Some day you will be old enough to start reading fairy tales again.” 88 | C.S. Lewis,“We are not necessarily doubting that God will do the best for us; we are wondering how painful the best will turn out to be.” 89 | Mark Twain,“The fear of death follows from the fear of life. A man who lives fully is prepared to die at any time.” 90 | Mark Twain,“A lie can travel half way around the world while the truth is putting on its shoes.” 91 | C.S. Lewis,"“I believe in Christianity as I believe that the sun has risen: not only because I see it, but because by it I see everything else.”" 92 | J.K. Rowling,"“The truth."" Dumbledore sighed. ""It is a beautiful and terrible thing, and should therefore be treated with great caution.”" 93 | Jimi Hendrix,"“I'm the one that's got to die when it's time for me to die, so let me live my life the way I want to.”" 94 | J.M. Barrie,“To die will be an awfully big adventure.” 95 | E.E. Cummings,“It takes courage to grow up and become who you really are.” 96 | Khaled Hosseini,“But better to get hurt by the truth than comforted with a lie.” 97 | Harper Lee,“You never really understand a person until you consider things from his point of view... Until you climb inside of his skin and walk around in it.” 98 | Madeleine L'Engle,"“You have to write the book that wants to be written. And if the book will be too difficult for grown-ups, then you write it for children.”" 99 | Mark Twain,“Never tell the truth to people who are not worthy of it.” 100 | Dr. Seuss,"“A person's a person, no matter how small.”" 101 | George R.R. Martin,"“... a mind needs books as a sword needs a whetstone, if it is to keep its edge.”" 102 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_2/quotes_scraper.py: -------------------------------------------------------------------------------- 1 | from typing import Dict 2 | import requests 3 | from bs4 import BeautifulSoup 4 | from csv import DictWriter 5 | 6 | def return_raw_html(url): 7 | res = requests.get(url) 8 | return res.content.decode() 9 | 10 | def scrap_quotes(): 11 | base_url = "https://quotes.toscrape.com/" 12 | page_url = "" 13 | output = [] 14 | while(True): 15 | print(page_url) 16 | content = return_raw_html(base_url+page_url) 17 | soup = BeautifulSoup(content,'html.parser') 18 | quotes = soup.select(".quote") 19 | for quote in quotes: 20 | quote_text= quote.select_one('span[itemprop="text"]').text 21 | quote_author= quote.select_one('small[itemprop="author"]').text 22 | output.append({"text":quote_text, "author":quote_author}) 23 | page_url_raw = soup.select_one("li.next a") 24 | if page_url_raw == None: 25 | break 26 | page_url = page_url_raw['href'] 27 | return output 28 | 29 | def main(): 30 | quotes = scrap_quotes() 31 | with open("quotes.csv","w") as file: 32 | writer = DictWriter(file, fieldnames=["author","text"]) 33 | writer.writeheader() 34 | for q in quotes: 35 | writer.writerow(q) 36 | if __name__ == "__main__": 37 | main() -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/AJAX.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import requests as r\n" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "link = 'https://directory.ntschools.net/api/System/GetAllSchools'" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "data = r.get(link)\n", 28 | "data.content" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "import json " 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "parsed_data = json.loads(data.content)" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "parsed_data" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "link = 'https://directory.ntschools.net/api/System/GetSchool?itSchoolCode='" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": null, 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "school_codes = []\n", 74 | "for i in parsed_data:\n", 75 | " school_codes.append(i['itSchoolCode'])" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": null, 81 | "metadata": {}, 82 | "outputs": [], 83 | "source": [ 84 | "school_codes" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "school_data = r.get(link+'acacisch')" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "acacisch = json.loads(school_data.content)\n", 103 | "print(acacisch['name'])\n", 104 | "print(acacisch['physicalAddress']['description'])\n", 105 | "print(acacisch['physicalAddress']['displayAddress'])\n", 106 | "print(acacisch['telephoneNumber'])\n", 107 | "print(acacisch['schoolManagement'][0]['firstName'])\n", 108 | "print(acacisch['schoolManagement'][0]['position'])\n", 109 | "print(acacisch['schoolManagement'][1]['firstName'])\n", 110 | "print(acacisch['schoolManagement'][1]['position'])" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "acacisch" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": null, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "\"\"\" Complete in assignment \"\"\"" 129 | ] 130 | } 131 | ], 132 | "metadata": { 133 | "kernelspec": { 134 | "display_name": "Python 3", 135 | "language": "python", 136 | "name": "python3" 137 | }, 138 | "language_info": { 139 | "codemirror_mode": { 140 | "name": "ipython", 141 | "version": 3 142 | }, 143 | "file_extension": ".py", 144 | "mimetype": "text/x-python", 145 | "name": "python", 146 | "nbconvert_exporter": "python", 147 | "pygments_lexer": "ipython3", 148 | "version": "3.7.6" 149 | } 150 | }, 151 | "nbformat": 4, 152 | "nbformat_minor": 4 153 | } 154 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | # Python Bootcamp | DSC@DSU 4 | 5 |

6 | 7 | ## Week 2, Day 3 8 | 9 | By _Bahawal Baloch_, **Python@DSU Co-Lead** and Core Team **DSC@DSU** 10 | 11 | why is it hard to scrap dynamic web content ? 12 | 13 | Dynamic web content is such content that is added to the web page on run time. for example load more option adds more items to the webpage on the run time. This creates a problem for scrapers since when we use requests to get a page we get a static page which do not contain the complete information we need to scrap. 14 | 15 | How does dynamic web content works ? 16 | 17 | data is requested to the server then the HTML page is modify based on the response using jquery. 18 | 19 | Most commonly web devs use AJAX for this job. AJAX stands for asynchronous javascript XML. 20 | 21 | XML stands for Extensible Markup Language. 22 | 23 | How do we know if a website is using AJAX ? 24 | 25 | we can check it by simply checking the Network tab in inspect 26 | 27 | [https://directory.ntschools.net/#/schools](https://directory.ntschools.net/#/schools) 28 | 29 | ![lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/inspect.png](lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/inspect.png) 30 | 31 | As you can we the Data coming from an AJAX request can be found in the XHR tab 32 | 33 | XHR stands for XML Http request. 34 | 35 | let's see how can we find the address where we need to make AJAX request to get the required data. 36 | 37 | ![lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/header.png](lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/header.png) 38 | 39 | after finding the address we need to find the parameter which is used to call the data. 40 | 41 | if we scroll down we'll find 42 | 43 | ![lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/parameters.png](lecture%20notes%20-%202%203%20b111b8ff3373478abf32962786aa1994/parameters.png) 44 | 45 | this section which tell us how to pass parameters to our request. 46 | 47 | now we have both things now we need to iterate over the school code to 48 | 49 | ### the code. 50 | 51 | What is the return type of the request ? 52 | 53 | JSON ? 54 | 55 | JavaScript Object Notation is most similar to the python dictionary. 56 | 57 | how to read JSON ? 58 | 59 | using the JSON package in python. 60 | 61 | `parsed_data = json.loads(data.content)` 62 | 63 | ## Web Sessions 64 | 65 | What are web sessions ? 66 | 67 | whenever we login into a website or whenever you go to an eCommerce website and you are able to create a cart and maintain your data in cookies. This process of saving the user's data in local storage is called a web session. The session terminates by either leaving the website or deleting the cookies. 68 | 69 | Why do we need web sessions ? 70 | 71 | Some of the website's data might not be available directly we might need to login into a website to retrieve the data. We use the `requests` library to handle such scenarios. 72 | 73 | Example of a website. 74 | 75 | Explain Headers 76 | 77 | HTTP headers let the client and the server pass additional information with an HTTP request or response. Header cards or headers are extra parameters which go with our requests with tells the receiver certain things about us. Like what web browsers are we using etc 78 | 79 | Explain web tokens 80 | 81 | ```python 82 | import requests as r 83 | from bs4 import BeautifulSoup as soup 84 | link = 'https://www.codechef.com/' 85 | payload = { 86 | 'name': 'bahawal32', 87 | 'pass': 'Python321!', 88 | 'form_id': 'new_login_form', 89 | 'op': 'Login' 90 | } 91 | 92 | headers = {'User-Agent': 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36'} 93 | with r.Session() as s: 94 | p = s.get(link,headers=headers) 95 | soup_page = soup(p.content) 96 | payload['csrfToken']=soup_page.select('#edit-csrfToken')[0]['value'] 97 | payload['form_build_id']=soup_page.find('input',{'name':'form_build_id'})['id'] 98 | log_in = s.post(link,headers=headers,data=payload) 99 | soup_in = soup(log_in.content) 100 | print(soup_in) 101 | ``` 102 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/header.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/header.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/inspect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/inspect.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/parameters.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_2/day_3/lecture notes - 2 3 b111b8ff3373478abf32962786aa1994/parameters.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_2/day_3/sessions.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "link ='https://www.codechef.com/'" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import requests as r\n", 19 | "from bs4 import BeautifulSoup as soup" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "page = r.get(link).content" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "soup_page = soup(page)" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "soup_page.find('input',{'name':'form_build_id'})" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "form_id = soup_page.select_one('input[name=\"form_build_id\"]')['id']" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "token = soup_page.select_one('input[name=\"csrfToken\"]')['value']" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": null, 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "token" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "payload = {\n", 83 | "'name': 'bahawal32',\n", 84 | "'pass': 'Python321!',\n", 85 | "'form_id': 'new_login_form',\n", 86 | "'op': 'Login'\n", 87 | "}" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "with r.session() as s:\n", 97 | " page = s.get(link).content\n", 98 | " soup_page = soup(page)\n", 99 | " form_id = soup_page.select_one('input[name=\"form_build_id\"]')['id']\n", 100 | " token = soup_page.select_one('input[name=\"csrfToken\"]')['value']\n", 101 | " payload['csrfToken'] = token\n", 102 | " payload['form_build_id'] = form_id\n", 103 | " reponse = s.post(link,data=payload)" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": null, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | " reponse.content" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "'''look into request headers yourself :D '''\n", 122 | "headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'}" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "headers" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": null, 137 | "metadata": {}, 138 | "outputs": [], 139 | "source": [] 140 | } 141 | ], 142 | "metadata": { 143 | "kernelspec": { 144 | "display_name": "Python 3", 145 | "language": "python", 146 | "name": "python3" 147 | }, 148 | "language_info": { 149 | "codemirror_mode": { 150 | "name": "ipython", 151 | "version": 3 152 | }, 153 | "file_extension": ".py", 154 | "mimetype": "text/x-python", 155 | "name": "python", 156 | "nbconvert_exporter": "python", 157 | "pygments_lexer": "ipython3", 158 | "version": "3.7.6" 159 | } 160 | }, 161 | "nbformat": 4, 162 | "nbformat_minor": 4 163 | } 164 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | 5 | - ## Week 3 | Web automation using Selenium 6 | 7 | - ### [Day 1](day_1/) 8 | 9 | - Introduction to WebDrivers 10 | - Installation of Chrome/Firefox driver binary 11 | - Interacting with websites like clicks 12 | - Extracting DOM elements 13 | 14 | - ### [Day 2](day_2/) 15 | 16 | - Logging into websites 17 | - Sending keyboard input to websites 18 | - Soundcloud bot 19 | - WebDriver options 20 | 21 | - ### [Day 3](day_3/) 22 | - Handling exceptions 23 | - Explicit wait conditions 24 | - Facebook bot which comments and likes a given post 25 | 26 | ## Assignment 27 | 28 | ### 1. Take as user input a Facebook post link and have your bot like the post and share it on your timeline 29 | 30 | Notes: 31 | 32 | - You'll need to use m.facebook.com for ease 33 | - For submission, like and share this post: https://www.facebook.com/DeveloperStudentClubDHASuffaUniversity/posts/2838706263116774 34 | - The shared post should have the following caption(also sent through Selenium bot): 35 | - "This post was shared using a bot that I learnt to create from Python Bootcamp 2020 held by DSC@DSU. #DSCDSU #DeveloperStudentClubs #DSCPakistan #Python #Bot" 36 | - The caption is not strict, but should contain the hashtags even if you modify your message according to your flavor. 37 | - You'll need to submit a video recording of you running the script and showing the bot performing the liking and sharing of above mentioned post. 38 | 39 | **Optional, extra marks** : Make the bot comment on the above mentioned post as well. 40 | 41 | ### 2 **Optional, extra marks**: make the bot created above make at least 50 comments in succession on the page. These 50 comments should be lines split from a paragraph you may write on: 42 | 43 | - Your aims and goals after this bootcamp? 44 | - How did you like the instructors and the management? 45 | - What did you learn from the bootcamp? 46 | - Would you be willing to carry the legacy forward for a free, open source driven tech bootcamp like this one? 47 | 48 | - You can use main function to predefine anything but make sure you divide everything you can in functions. 49 | - Submit your github folder link, your output(in form of video this time) on Google Classroom 50 | - Make sure you fork [https://github.com/sinnytk/Python-Bootcamp-DSC-DSU](https://github.com/sinnytk/Python-Bootcamp-DSC-DSU), clone the repo locally and create your scripts in the respective week's folder and push it. 51 | - Make sure in each folder there's a [readme.md](http://readme.md) file with code output and brief explanation of what you did and why you did it (not necessary, but good practice) 52 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/image_(2).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_1/Lecture Notes 0c08aa6d2c274766a967cdc4fd129b8a/image_(2).png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_1/README.md: -------------------------------------------------------------------------------- 1 | # Python Bootcamp | DSC@DSU 2 | 3 |

4 | 5 | ## Week 3, Day 1 6 | 7 | By _Tarun Kumar_, Python@DSU Lead and Core Team member 8 | 9 | We have tried requesting HTML pages and retrieving content from APIs using the `BeautifulSoup4` and `Requests` packages. 10 | 11 | Now we move towards full-fledged web browser automation where you can replicate opening a complete browser and performing actions a human would, like: 12 | 13 | - Entering text into fields 14 | - Clicking on buttons 15 | - Submitting forms 16 | - Scrolling 17 | - Copying content 18 | - Hovering and navigating all around websites 19 | 20 | Interested yet? 21 | 22 | The powerful framework that we'll be using is called Selenium, specifically it's WebDriver module. 23 | 24 | ### Installing Selenium WebDriver 25 | 26 | You need two things to start working with Selenium. 27 | 28 | 1. Installing the Python library using `pip` 29 | 30 | ```bash 31 | pip install selenium 32 | ``` 33 | 34 | 2. Downloading the binary for your respective browser and put it either in your script's folder or your Python default path 35 | 36 | ``` 37 | Download the driver binary for your respective browser. I'll be using Chrome so I'll put in the guide for that. 38 | 39 | Chrome: https://chromedriver.chromium.org/downloads 40 | Edge: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/ 41 | Firefox: https://github.com/mozilla/geckodriver/releases 42 | 43 | 1. Check your chrome version by going to chrome://version 44 | 2. Download the respective binary from the link above 45 | 3. Extract the .zip containing the .exe (or for Unix systems normal object file) to your Python path, that we discussed in the first lecture. 46 | My path was: /usr/bin 47 | For windows, if we put the chromedriver.exe under C:/Windows, then there is no need to specify executable_path since Python will search under C:/Windows. 48 | ``` 49 | 50 | ### Opening Chrome and accessing sites 51 | 52 | To start working with Selenium, you can simply import the WebDriver and open a Chrome instance like: 53 | 54 | ```bash 55 | >>> from selenium import webdriver 56 | >>> driver = webdriver.Chrome() 57 | >>> driver.get("www.github.com/sinnytk") 58 | ``` 59 | 60 | ![Lecture%20Notes%200c08aa6d2c274766a967cdc4fd129b8a/Untitled.png](Lecture%20Notes%200c08aa6d2c274766a967cdc4fd129b8a/Untitled.png) 61 | 62 | ### Finding and selecting DOM elements 63 | 64 | Similar to the `select` function in a BeautifulSoup parsed object, you can find elements using CSS selector, but Selenium being much more robust has options to find elements by: 65 | 66 | - xpath 67 | - id 68 | - class names 69 | 70 | You can find more on the different selector methods [here](https://selenium-python.readthedocs.io/locating-elements.html#locating-elements) 71 | 72 | Suppose, I want to find all the pinned repositories on my Github. 73 | 74 | Using `inspect element` I could find that the clickable links are enclosed in a `div` with class `pinned-item-list-item-content` and then an `a` tag with class `text-bold` 75 | 76 | ![Lecture%20Notes%200c08aa6d2c274766a967cdc4fd129b8a/image_(2).png]() 77 | 78 | ```bash 79 | >>> driver.find_elements_by_css_selector("**div.pinned-item-list-item-content a.text-bold**") 80 | 81 | [, , , , , ] 82 | 83 | ``` 84 | 85 | ### `WebElement` properties 86 | 87 | Now if you may notice, we're not getting a list of `Tag` objects like with `BeautifulSoup4`, but instead we're getting a list of `WebElement` objects. These are native to Selenium and provide numerous functions and actions that can be performed on them. 88 | 89 | Let's try some of them: 90 | 91 | ```bash 92 | >>> repo_links_elems = driver.find_elements_by_css_selector("div.pinned-item-list-item-content a.text-bold") 93 | ``` 94 | 95 | - Raw text of the element: 96 | 97 | ```bash 98 | >>> repo_links_elems[0] 99 | 100 | >>> repo_links_elems[0].get_attribute("text") 101 | '\n ration-app\n' 102 | >>> repo_links_elems[0].text 103 | 'ration-app' 104 | >>> 105 | ``` 106 | 107 | - Attributes of the element: 108 | 109 | ```bash 110 | >>> repo_links_elems[0].get_attribute('href') 111 | 'https://github.com/sinnytk/ration-app' 112 | >>> repo_links_elems[0].get_attribute('class') 113 | 'text-bold flex-auto min-width-0 ' 114 | ``` 115 | 116 | - Accessing child elements 117 | 118 | ```bash 119 | >>> repo_links_elems[0].get_attribute("outerHTML") 120 | '\n ration-app\n' 121 | # the outerHTML tag returns the HTML of the element. 122 | # we can see a tag enclosed in the anchor tag. 123 | 124 | >>> repo_links_elems[0].find_elements_by_css_selector("span") 125 | [] 126 | >>> repo_links_elems[0].find_elements_by_css_selector("span")[0].text 127 | 'ration-app' 128 | ``` 129 | 130 | ### Interacting using clicks and other human behavior 131 | 132 | Now I showed you some properties of the `WebElement` object above, but that's not even half of what Selenium can do. 133 | 134 | Selenium was originally produced as a testing framework where you can run automated tests on your websites to ensure every nook and cranny is working as expected without have a human manually laboring through the different test cases. 135 | 136 | And as is often in websites, they require interactions like clicks, scrolling, hovering and sending key input for typing and shortcuts etc. 137 | 138 | You can do all such actions using a Selenium WebDriver. 139 | 140 | If you remember the [React based Calculator site](https://ahfarmer.github.io/calculator/) that I showed requiring JavaScript in the web scraping sessions, let's try having Selenium perform actions for us on that site? 141 | 142 | ![Lecture%20Notes%200c08aa6d2c274766a967cdc4fd129b8a/Untitled%201.png](Lecture%20Notes%200c08aa6d2c274766a967cdc4fd129b8a/Untitled%201.png) 143 | 144 | 1. Go to the site using `driver.get()` 145 | 146 | ```bash 147 | >>> from selenium import webdriver 148 | >>> driver = webdriver.Chrome() 149 | >>> driver = webdriver.get("https://ahfarmer.github.io/calculator/") 150 | ``` 151 | 152 | 2. From inspection, we can find that all the buttons are enclosed in `button` which is enclosed in a `div` with class `component-button` 153 | 154 | ```bash 155 | >>> btns = driver.find_elements_by_css_selector('div.component-button button') 156 | >>> for btn in btns: 157 | ... print(btn.text) 158 | ... 159 | AC 160 | +/- 161 | % 162 | ÷ 163 | 7 164 | 8 165 | 9 166 | x 167 | 4 168 | 5 169 | 6 170 | - 171 | 1 172 | 2 173 | 3 174 | + 175 | 0 176 | . 177 | = 178 | 179 | ``` 180 | 181 | Now that you've queried the buttons, let's try to click them? 182 | 183 | A dumb approach would be to iterate over the buttons and search for values we want to press. 184 | 185 | But as we do not have a specific ID or way to correctly find a specific button, but what we do have a list of elements which can be mapped to what they represent in value. 186 | 187 | And what's the best structure to map values? A **dictionary** 188 | 189 | ```bash 190 | SYMBOL_TO_INDEX = {'AC': 0, '%': 2, '/': 3, 191 | '7': 4, '8': 5, '9': 6, 192 | '*': 7, '4': 8, '5': 9, 193 | '6': 10, '-': 11, '1': 12, 194 | '2': 13, '3': 14, '+': 15, 195 | '0': 16, '.': 17, '=': 18} 196 | # You may notice I haven't assigned the mapping **+/-** to 1. 197 | # This is just to avoid wasting time on supporting negative numbers for now 198 | ``` 199 | 200 | Now if I wanted to evaluate 2+2, I would do: 201 | 202 | ```bash 203 | >>> btns[SYMBOL_TO_INDEX['2']].click() # retrieves index of 2 and presses it 204 | >>> btns[SYMBOL_TO_INDEX['+']].click() # retrieves index of + and presses it 205 | >>> btns[SYMBOL_TO_INDEX['2']].click() # retrieves index of 2 and presses it 206 | >>> btns[SYMBOL_TO_INDEX['=']].click() # retrieves index of = and presses it 207 | 208 | # result is evaluated in a div called component-display 209 | >>> driver.find_element_by_css_selector('div.component-display div').text 210 | '4' 211 | ``` 212 | 213 | Live session: 214 | 215 | - Build quotes scraper using Selenium 216 | 217 | And that's how you make your browser do stuff for you. 218 | 219 | Look into: 220 | 221 | - Selenium options 222 | - Start headless processes 223 | - Define custom browsers like Brave 224 | 225 | Some tasks you can do on your own: 226 | 227 | - Try going over the BS4 examples using Selenium now. 228 | - Scrap the school listings by clicking instead of API interaction. 229 | - Create a bot that plays a song using terminal on Patari.pk 230 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_1/react_calc.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | 3 | DRIVER = None 4 | CALC_BUTTONS = None 5 | SYMBOL_TO_INDEX = {'AC': 0, '%': 2, '/': 3, 6 | '7': 4, '8': 5, '9': 6, 7 | '*': 7, '4': 8, '5': 9, 8 | '6': 10, '-': 11, '1': 12, 9 | '2': 13, '3': 14, '+': 15, 10 | '0': 16, '.': 17, '=': 18} 11 | 12 | 13 | def initiate_browser(url): 14 | local_driver = webdriver.Chrome() 15 | local_driver.get(url) 16 | return local_driver 17 | 18 | 19 | def validate_and_execute(expr): 20 | if expr == 'exit': 21 | return None 22 | CALC_BUTTONS[SYMBOL_TO_INDEX['AC']].click() 23 | 24 | # break down the expression into characters/symbols 25 | for symbol in expr: 26 | # if symbol is a space, skip 27 | if symbol == ' ': 28 | continue 29 | 30 | # get the button element index for a symbol in the expression 31 | btn_index = SYMBOL_TO_INDEX.get(symbol, None) 32 | 33 | # if there's no index available for entered symbol, return 34 | if not btn_index: 35 | print('\tInvalid expression') 36 | return None 37 | 38 | # click on the mapped element 39 | CALC_BUTTONS[btn_index].click() 40 | 41 | # evaluate the expression 42 | CALC_BUTTONS[SYMBOL_TO_INDEX['=']].click() 43 | 44 | # return the evaluated answer 45 | answer = DRIVER.find_element_by_css_selector( 46 | 'div.component-display div').text 47 | return answer 48 | 49 | 50 | def program_loop(): 51 | print("\n\n\n\n\n") 52 | print("Keep entering mathematical expressions to solve or type 'exit' to terminate.") 53 | print("Some examples of expressions:") 54 | print("\t12+15") 55 | print("\t42-5") 56 | print("\t5*5") 57 | print("\t25/5") 58 | print("\n\n") 59 | expr = "" 60 | while(expr != 'exit'): 61 | expr = input("Enter expression or type 'exit':") 62 | answer = validate_and_execute(expr) 63 | if answer: 64 | print(f"{expr}={answer}") 65 | 66 | 67 | def main(): 68 | global DRIVER 69 | global CALC_BUTTONS 70 | 71 | DRIVER = initiate_browser("https://ahfarmer.github.io/calculator/") 72 | CALC_BUTTONS = DRIVER.find_elements_by_css_selector( 73 | "div.component-button button") 74 | 75 | program_loop() 76 | DRIVER.quit() 77 | 78 | 79 | if __name__ == "__main__": 80 | main() 81 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 2.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 3.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled 4.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_2/Lecture Notes 8078ffb6e5f84399b2d64c93984199b4/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | # Python Bootcamp | DSC@DSU 4 | 5 |

6 | 7 | ## Week 3, Day 2 8 | 9 | By _Tarun Kumar_, **Python@DSU Lead** and Core Team **DSC@DSU** 10 | 11 | Let's get our hands dirtied by going one step further into automating website access. 12 | 13 | I discussed in the previous session that we can automate keyboard input on websites, so let's get started with that first? 14 | 15 | ### Searching and playing songs on Soundcloud 16 | 17 | Let's a build a bot that'll: 18 | 19 | 1. Go to [Soundcloud](http://soundcloud.com) 20 | 2. Search a song in the search field 21 | 3. Press enter to execute search 22 | 4. Play the first song from search results. 23 | 24 | ```bash 25 | # opens chrome driver 26 | # goes to soundcloud 27 | # selects search input element 28 | >>> driver = webdriver.Chrome() 29 | >>> driver.get("https://soundcloud.com") 30 | >>> input_el =driver.find_element_by_css_selector("input[type='search']") 31 | >>> input_el.get_attribute("placeholder") 32 | 'Search for artists, bands, tracks, podcasts' 33 | ``` 34 | 35 | However when we click on the search bar to put it into focus, like a normal human user would. I get an `ElementNotInteractableException` 36 | 37 | ```bash 38 | >>> input_el.click() 39 | Traceback (most recent call last): 40 | File "", line 1, in 41 | File "/home/tarun/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click 42 | self._execute(Command.CLICK_ELEMENT) 43 | File "/home/tarun/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute 44 | return self._parent.execute(command, params) 45 | File "/home/tarun/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute 46 | self.error_handler.check_response(response) 47 | File "/home/tarun/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response 48 | raise exception_class(message, screen, stacktrace) 49 | selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable 50 | (Session info: chrome=85.0.4183.121) 51 | ``` 52 | 53 | What can cause the above issue? 54 | 55 | - Element is not loaded yet (not possible, we can see the element) 56 | - Element is hidden inside other elements (check the inspect element to make sure) 57 | 58 | Now there are multiple ways to fix this and usually the best way to do is the simplest one. 59 | 60 | _Query using the XPath_ 61 | 62 | `Right click on element -> Inspect element -> Copy XPath` 63 | 64 | ![Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled.png](Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled.png) 65 | 66 | ```bash 67 | >>> input_el = driver.find_element_by_xpath('//*[@id="content"]/div/div/div[2]/div/div[1]/span/span/form/input') 68 | >>> input_el.click() 69 | ``` 70 | 71 | ### Keyboard input 72 | 73 | Selenium `WebElement` provides a convenient function called `send_keys` that you can use to emulate user input into fields, like so: 74 | 75 | ```bash 76 | >>> input_el.send_keys("Amir Zaki The Day She Left") 77 | ``` 78 | 79 | ![Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%201.png](Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%201.png) 80 | 81 | Now to initiate the search we need to hit `enter`, we have two options to do so: 82 | 83 | - Sending in raw `\n` which is basically a enter in most search fields. 84 | 85 | ```bash 86 | >>> input_el.send_keys("\n") 87 | ``` 88 | 89 | - Using the Keys object from Selenium common keys. 90 | 91 | ```bash 92 | >>> from selenium.webdriver.common.keys import Keys 93 | >>> Keys.ENTER 94 | '\ue007' 95 | >>> input_el.send_keys(Keys.ENTER) 96 | ``` 97 | 98 | Finally playing the song: 99 | 100 | ```bash 101 | driver.find_element_by_css_selector("a.snippetUXPlayButton").click() 102 | ``` 103 | 104 | ![Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%202.png](Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%202.png) 105 | 106 | ### Logging into Facebook 107 | 108 | Let's get to logging into your Facebook accounts using Selenium. 109 | 110 | ```bash 111 | >>> driver.get("https://facebook.com") 112 | >>> email_input = driver.find_element_by_id("email") 113 | >>> password_input = driver.find_element_by_id("pass") 114 | ``` 115 | 116 | ![Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%203.png](Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%203.png) 117 | 118 | ```bash 119 | # **getpass** is a tool which you can use to hide user input from terminal stdout 120 | 121 | >>> from getpass import getpass 122 | >>> email = "rovekoy922@94jo.com" 123 | >>> password = getpass() 124 | Password: 125 | >>> login_btn = driver.find_element_by_css_selector("button[data-testid='royal_login_button']") 126 | >>> login_btn.click() 127 | ``` 128 | 129 | ![Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%204.png](Lecture%20Notes%208078ffb6e5f84399b2d64c93984199b4/Untitled%204.png) 130 | 131 | Questions to ponder: 132 | 133 | - Look into `Action Chains` 134 | - Sending a series of inputs 135 | - Key_up, key down 136 | - Drag and drop 137 | - Action Chains for form filling 138 | - How would you select a specific value in a `select` field? 139 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_2/fb_script.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | from getpass import getpass 3 | from selenium.webdriver.chrome.options import Options 4 | 5 | option = Options() 6 | 7 | option.add_argument("--disable-infobars") 8 | 9 | # Pass the argument 1 to allow and 2 to block 10 | option.add_experimental_option("prefs", { 11 | "profile.default_content_setting_values.notifications": 1 12 | }) 13 | driver = None 14 | 15 | 16 | def go_to_home(): 17 | home_btn = driver.find_element_by_css_selector('a[aria-label="Home"]') 18 | home_btn.click() 19 | 20 | 21 | def login(url): 22 | driver.get(url) 23 | email_field = driver.find_element_by_id("email") 24 | password_field = driver.find_element_by_id("pass") 25 | login_btn = driver.find_element_by_css_selector( 26 | 'button[data-testid="royal_login_button"]') 27 | 28 | email = "rovekoy922@94jo.com" 29 | password = getpass("Enter password: ") 30 | email_field.send_keys(email) 31 | password_field.send_keys(password) 32 | login_btn.click() 33 | 34 | 35 | def initiate_browser(): 36 | driver = webdriver.Chrome(chrome_options=option) 37 | return driver 38 | 39 | 40 | def main(): 41 | global driver 42 | driver = initiate_browser() 43 | login("https://facebook.com") 44 | go_to_home() 45 | driver.quit() 46 | 47 | 48 | if __name__ == "__main__": 49 | main() 50 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/Lecture Notes 121d9a943593432e8769f200aa9621cf/Untitled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_3/day_3/Lecture Notes 121d9a943593432e8769f200aa9621cf/Untitled.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | # Python Bootcamp | DSC@DSU 4 | 5 |

6 | 7 | ## Week 3, Day 3 8 | 9 | By _Tarun Kumar_, **Python@DSU Lead** and Core Team **DSC@DSU** 10 | 11 | This will be the last session on Selenium and I plan to cover the different functionalities, caveats and cool features that the library provides. 12 | 13 | ### Pausing on SoundCloud 14 | 15 | But first let's discuss the Soundcloud example which played a song you queried for. As it happens in a professional environment, you sometimes come upon examples or implementations that should work but for some reason don't, the Soundcloud bot that we were trying to create in the last session is the prime example of that. 16 | 17 | I found out that (and built up on what we found in the live session) that the classes of the play button change and as they do, the previous triggered action is overridden in the chain of actions. 18 | 19 | ![Lecture%20Notes%20121d9a943593432e8769f200aa9621cf/Untitled.png](Lecture%20Notes%20121d9a943593432e8769f200aa9621cf/Untitled.png) 20 | 21 | I found out that `XPath` does fix this if you just copy the Xpath of the element, which comes out to be: 22 | 23 | ```bash 24 | >>> elem_xpath = '//*[@id="content"]/div/div/div[3]/div/div/div/ul/li[1]/div/div/div/div[2]/div[1]/div/div/div[1]/a' 25 | >>> driver.find_element_by_xpath(elem_xpath).click() 26 | # song plays 27 | >>> driver.find_element_by_xpath(elem_xpath).click() 28 | # song pauses 29 | 30 | ``` 31 | 32 | As I have mentioned a few times, I personally prefer using CSS Selectors so I got a similar selector working for pausing as well! 33 | 34 | ```bash 35 | >>> play_pause_btn = driver.find_element_by_css_selector("div.sound__header div div div") 36 | >>> play_pause_btn.click() 37 | # song plays 38 | >>> play_pause_btn.click() 39 | # song paused 40 | ``` 41 | 42 | ### Expected Wait Conditions 43 | 44 | All good right? The bot works now.. _only in theory_, not in script. 45 | 46 | When I converted the bot to a full-fledged script (available in the repository as `soundcloud_script.py`), I found out that the bot errors out when trying to play a song after searching it: 47 | 48 | ```bash 49 | selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.sound__header div div div"} 50 | (Session info: chrome=85.0.4183.121) 51 | ``` 52 | 53 | Looking into the error, we can deduce that we're trying to search for an element that doesn't exist in on the page yet. 54 | 55 | This is because we're not waiting for the page to load before accessing that element. 56 | 57 | What would be a simple fix? A way to make the script wait before querying that element (i.e letting the page load?) 58 | 59 | The simplest fix is to use `time.sleep()` to make the script stop execution for a while. And that does indeed fix the script and makes the script execute smoothly by adding a constant execution halt of `2` seconds. (see `soundcloud_script_sleep.py` in the repo) 60 | 61 | Some problems with using `time.sleep` 62 | 63 | - It's constant, therefore: 64 | - If the page load takes more time than explicitly defined, it'll fail 65 | - Or if the page load takes much less time than before, time is wasted 66 | - It's not dynamic enough to account for complex conditions where you want specific content to load instead of an element 67 | 68 | One way to use sleep dynamically would be to handle exceptions like so: 69 | 70 | ```bash 71 | >>>from selenium.common.exceptions import NoSuchElementException 72 | >>> while True: 73 | ... try: 74 | ... play_pause_btn = driver.find_element_by_css_selector("div.sound__header div div div") 75 | ... play_pause_btn.click() 76 | ... break 77 | ... except NoSuchElementException: 78 | ... print("play button not found, waiting 0.5 seconds") 79 | ... time.sleep(0.5) 80 | ... 81 | play button not found, waiting 0.5 seconds 82 | play button not found, waiting 0.5 seconds 83 | ``` 84 | 85 | Selenium provides a really good wait condition framework to accommodate page loading. These wait conditions are called `Expected Conditions` and can make a `WebDriver` check for existence of an element and then do something or throw a timeout exception. 86 | 87 | You can read up on [Explicit Waits over here](https://selenium-python.readthedocs.io/waits.html#explicit-waits). 88 | Let's try to use a `presence_of_element_located` check before clicking on the button for our Soundcloud example. 89 | 90 | We'll need to import 3 things to use expected conditions. 91 | 92 | - `selenium.webdriver.support.expected_conditions` 93 | - `selenium.webdriver.support.ui.WebDriverWait` 94 | - `selenium.webdriver.common.by.By` 95 | 96 | These 3 classes will help us simplify the dynamic sleep we defined above. 97 | 98 | ```python 99 | from selenium.webdriver.common.by import By 100 | from selenium.webdriver.support.ui import WebDriverWait 101 | from selenium.webdriver.support import expected_conditions as EC 102 | from selenium.common.exceptions import TimeoutException 103 | 104 | try: 105 | element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "myDynamicElement")) 106 | except TimeoutException: 107 | print("Element not found, exiting driver") 108 | driver.quit() 109 | 110 | # WebDriverWait(driver, 10) | make the webdriver keep checking for 10 seconds 111 | # .until | until the following condition returns true 112 | # EC.presence_of_element_located | use an element's presence as check 113 | # (By.ID, "myDynamicElement") | find element by ID of myDynamicElement 114 | ``` 115 | 116 | Let's convert the above for our play button element. 117 | 118 | ```python 119 | play_btn = WebDriverWait(driver, 10).until(EC.presence_of_element_located( 120 | (By.CSS_SELECTOR, "div.sound__header div div div"))) 121 | play_btn.click() 122 | ``` 123 | 124 | ### Building a Facebook bot 125 | 126 | Covered in live session. 127 | 128 | The bot will: 129 | 130 | - use m.facebook.com 131 | - go to a given post link 132 | - like and comment on it 133 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/fb_bot.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "from selenium import webdriver\n", 10 | "from getpass import getpass\n", 11 | "from selenium.common.exceptions import NoSuchElementException, TimeoutException\n", 12 | "from selenium.webdriver.support.ui import WebDriverWait\n", 13 | "from selenium.webdriver.support import expected_conditions as EC\n", 14 | "from selenium.webdriver.common.by import By\n", 15 | "from selenium.webdriver.chrome.options import Options" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": 9, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "driver = webdriver.Chrome()" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 10, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "driver.get(\"https://www.facebook.com\")" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 11, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "def login(url):\n", 43 | " driver.get(url)\n", 44 | " email_field = driver.find_element_by_id(\"email\")\n", 45 | " password_field = driver.find_element_by_id(\"pass\")\n", 46 | " login_btn = driver.find_element_by_css_selector(\n", 47 | " 'button[data-testid=\"royal_login_button\"]')\n", 48 | "\n", 49 | " email = \"sinnytk@gmail.com\"\n", 50 | " password = getpass(\"Enter password: \")\n", 51 | " email_field.send_keys(email)\n", 52 | " password_field.send_keys(password)\n", 53 | " login_btn.click()" 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": 12, 59 | "metadata": {}, 60 | "outputs": [ 61 | { 62 | "name": "stdout", 63 | "output_type": "stream", 64 | "text": [ 65 | "Enter password: ········\n" 66 | ] 67 | } 68 | ], 69 | "source": [ 70 | "login(\"https://www.facebook.com\")" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 13, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "post_url = \"https://m.facebook.com/DeveloperStudentClubDHASuffaUniversity/posts/2838471249806942\"" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": 15, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "driver.get(\"https://m.facebook.com/DeveloperStudentClubDHASuffaUniversity/posts/2838471249806942\")" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": 21, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "like_btn = driver.find_element_by_css_selector(\"div[data-sigil='ufi-inline-actions'] div\")" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": 22, 103 | "metadata": {}, 104 | "outputs": [ 105 | { 106 | "data": { 107 | "text/plain": [ 108 | "''" 109 | ] 110 | }, 111 | "execution_count": 22, 112 | "metadata": {}, 113 | "output_type": "execute_result" 114 | } 115 | ], 116 | "source": [ 117 | "like_btn.get_attribute(\"outerHTML\")" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": 27, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "like_btn.click()" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 28, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "comment_box = driver.find_element_by_css_selector(\"div[data-sigil='m-composer'] textarea#composerInput\")" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": 29, 141 | "metadata": {}, 142 | "outputs": [ 143 | { 144 | "data": { 145 | "text/plain": [ 146 | "''" 147 | ] 148 | }, 149 | "execution_count": 29, 150 | "metadata": {}, 151 | "output_type": "execute_result" 152 | } 153 | ], 154 | "source": [ 155 | "comment_box.get_attribute(\"outerHTML\")" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 30, 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "comment_box.send_keys(\"Hello from Python Bootcamp Week 3 Day 3 Live!\")" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": 31, 170 | "metadata": {}, 171 | "outputs": [], 172 | "source": [ 173 | "post_button = driver.find_element_by_css_selector(\"div[data-sigil='m-composer'] button[data-sigil='touchable composer-submit']\")" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": 32, 179 | "metadata": {}, 180 | "outputs": [ 181 | { 182 | "data": { 183 | "text/plain": [ 184 | "''" 185 | ] 186 | }, 187 | "execution_count": 32, 188 | "metadata": {}, 189 | "output_type": "execute_result" 190 | } 191 | ], 192 | "source": [ 193 | "post_button.get_attribute(\"outerHTML\")" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 33, 199 | "metadata": {}, 200 | "outputs": [], 201 | "source": [ 202 | "post_button.click()" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 38, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 37, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [] 225 | } 226 | ], 227 | "metadata": { 228 | "kernelspec": { 229 | "display_name": "Python 3", 230 | "language": "python", 231 | "name": "python3" 232 | }, 233 | "language_info": { 234 | "codemirror_mode": { 235 | "name": "ipython", 236 | "version": 3 237 | }, 238 | "file_extension": ".py", 239 | "mimetype": "text/x-python", 240 | "name": "python", 241 | "nbconvert_exporter": "python", 242 | "pygments_lexer": "ipython3", 243 | "version": "3.8.5" 244 | } 245 | }, 246 | "nbformat": 4, 247 | "nbformat_minor": 4 248 | } 249 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/soundcloud_pause.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "from selenium import webdriver" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 2, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "driver = webdriver.Chrome()" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 4, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "driver.get(\"https://www.soundcloud.com\")" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 5, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "search_bar = driver.find_element_by_xpath(\n", 37 | " '//*[@id=\"content\"]/div/div/div[2]/div/div[1]/span/span/form/input')\n", 38 | "search_bar.click()" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 6, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "search_bar.send_keys(\"amir zaki the day she left\")" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 8, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "from selenium.webdriver.common.keys import Keys" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 10, 62 | "metadata": {}, 63 | "outputs": [], 64 | "source": [ 65 | "search_bar.send_keys(Keys.ENTER)" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": 17, 71 | "metadata": { 72 | "scrolled": true 73 | }, 74 | "outputs": [], 75 | "source": [ 76 | "play_button = driver.find_element_by_xpath('''//*[@id=\"content\"]/div/div/div[3]/div/div/div/ul/li[1]/div/div/div/div[2]/div[1]/div/div/div[1]/a''')" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": 19, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "play_button.click()" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": 20, 91 | "metadata": {}, 92 | "outputs": [], 93 | "source": [ 94 | "play_button = driver.find_element_by_css_selector('div.sound__content div.sound__header div div div')" 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 23, 100 | "metadata": {}, 101 | "outputs": [], 102 | "source": [ 103 | "play_button.click()" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": null, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [] 112 | } 113 | ], 114 | "metadata": { 115 | "kernelspec": { 116 | "display_name": "Python 3", 117 | "language": "python", 118 | "name": "python3" 119 | }, 120 | "language_info": { 121 | "codemirror_mode": { 122 | "name": "ipython", 123 | "version": 3 124 | }, 125 | "file_extension": ".py", 126 | "mimetype": "text/x-python", 127 | "name": "python", 128 | "nbconvert_exporter": "python", 129 | "pygments_lexer": "ipython3", 130 | "version": "3.8.5" 131 | } 132 | }, 133 | "nbformat": 4, 134 | "nbformat_minor": 4 135 | } 136 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/soundcloud_script_EC.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | from selenium.common.exceptions import TimeoutException, NoSuchElementException 3 | from selenium.webdriver.support.ui import WebDriverWait 4 | from selenium.webdriver.support import expected_conditions as EC 5 | from selenium.webdriver.common.by import By 6 | from selenium.webdriver.chrome.options import Options 7 | from os import system as sys 8 | option = Options() 9 | driver = None 10 | 11 | 12 | header = ''' 13 | / ___| ___ _ _ _ __ __| | ___| | ___ _ _ __| | 14 | \___ \ / _ \| | | | '_ \ / _` |/ __| |/ _ \| | | |/ _` | 15 | ___) | (_) | |_| | | | | (_| | (__| | (_) | |_| | (_| | 16 | |____/ \___/ \__,_|_| |_|\__,_|\___|_|\___/ \__,_|\__,_| 17 | 18 | ____ _ 19 | | _ \| | __ _ _ _ ___ _ __ 20 | | |_) | |/ _` | | | |/ _ \ '__| 21 | | __/| | (_| | |_| | __/ | 22 | |_| |_|\__,_|\__, |\___|_| 23 | |___/ 24 | 25 | Tarun Kumar | Python Bootcamp 2020 26 | 27 | ''' 28 | 29 | 30 | def print_header(): 31 | sys("clear") 32 | print(header) 33 | 34 | 35 | def play_pause(): 36 | play_btn = WebDriverWait(driver, 10).until(EC.presence_of_element_located( 37 | (By.CSS_SELECTOR, "div.sound__header div div div"))) 38 | play_btn.click() 39 | 40 | 41 | def search(q): 42 | search_bar = driver.find_element_by_xpath( 43 | '//*[@id="content"]/div/div/div[2]/div/div[1]/span/span/form/input') 44 | search_bar.click() 45 | search_bar.send_keys(q.strip() + '\n') 46 | 47 | 48 | def search_and_play(): 49 | song_to_play = input("\n\nEnter the song you want to play: ") 50 | song_to_play = song_to_play.title() 51 | search(song_to_play) 52 | play_pause() 53 | play_pause_exit = 0 54 | while(play_pause_exit != 2): 55 | print( 56 | f"{'Playing' if play_pause_exit == 0 else 'Paused'} {song_to_play}, press P to {'pause' if play_pause_exit == 0 else 'play'} it") 57 | print('or enter X to exit the program') 58 | 59 | input_value = input() 60 | if input_value.lower() == 'p': 61 | play_pause() 62 | print_header() 63 | if play_pause_exit == 0: 64 | play_pause_exit = 1 65 | else: 66 | play_pause_exit = 0 67 | elif input_value.lower() == 'x': 68 | play_pause_exit = 2 69 | else: 70 | print_header() 71 | print("Exiting the program") 72 | 73 | 74 | def initiate_browser(): 75 | driver = webdriver.Chrome(options=option) 76 | driver.set_network_conditions( 77 | offline=False, 78 | latency=500, # additional latency (ms) 79 | download_throughput=500 * 1024, # maximal throughput 80 | upload_throughput=500 * 1024) # maximal throughput 81 | return driver 82 | 83 | 84 | def main(): 85 | global driver 86 | print_header() 87 | driver = initiate_browser() 88 | driver.get("https://www.soundcloud.com") 89 | search_and_play() 90 | driver.quit() 91 | 92 | 93 | if __name__ == "__main__": 94 | main() 95 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_3/day_3/soundcloud_script_sleep.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | from time import sleep 3 | from selenium.webdriver.chrome.options import Options 4 | from os import system as sys 5 | option = Options() 6 | driver = None 7 | 8 | 9 | header = ''' 10 | / ___| ___ _ _ _ __ __| | ___| | ___ _ _ __| | 11 | \___ \ / _ \| | | | '_ \ / _` |/ __| |/ _ \| | | |/ _` | 12 | ___) | (_) | |_| | | | | (_| | (__| | (_) | |_| | (_| | 13 | |____/ \___/ \__,_|_| |_|\__,_|\___|_|\___/ \__,_|\__,_| 14 | 15 | ____ _ 16 | | _ \| | __ _ _ _ ___ _ __ 17 | | |_) | |/ _` | | | |/ _ \ '__| 18 | | __/| | (_| | |_| | __/ | 19 | |_| |_|\__,_|\__, |\___|_| 20 | |___/ 21 | 22 | Tarun Kumar | Python Bootcamp 2020 23 | 24 | ''' 25 | 26 | 27 | def print_header(): 28 | sys("clear") 29 | print(header) 30 | 31 | 32 | def play_pause(): 33 | play_btn = driver.find_element_by_css_selector( 34 | "div.sound__header div div div") 35 | play_btn.click() 36 | 37 | 38 | def search(q): 39 | search_bar = driver.find_element_by_xpath( 40 | '//*[@id="content"]/div/div/div[2]/div/div[1]/span/span/form/input') 41 | search_bar.click() 42 | search_bar.send_keys(q.strip() + '\n') 43 | 44 | 45 | def search_and_play(): 46 | song_to_play = input("\n\nEnter the song you want to play: ") 47 | song_to_play = song_to_play.title() 48 | search(song_to_play) 49 | sleep(2) 50 | play_pause() 51 | play_pause_exit = 0 52 | while(play_pause_exit != 2): 53 | print( 54 | f"{'Playing' if play_pause_exit == 0 else 'Paused'} {song_to_play}, press P to {'pause' if play_pause_exit == 0 else 'play'} it") 55 | print('or enter X to exit the program') 56 | 57 | input_value = input() 58 | if input_value.lower() == 'p': 59 | play_pause() 60 | print_header() 61 | if play_pause_exit == 0: 62 | play_pause_exit = 1 63 | else: 64 | play_pause_exit = 0 65 | elif input_value.lower() == 'x': 66 | play_pause_exit = 2 67 | else: 68 | print_header() 69 | print("Exiting the program") 70 | 71 | 72 | def initiate_browser(): 73 | driver = webdriver.Chrome(options=option) 74 | return driver 75 | 76 | 77 | def main(): 78 | global driver 79 | print_header() 80 | driver = initiate_browser() 81 | driver.get("https://www.soundcloud.com") 82 | search_and_play() 83 | driver.quit() 84 | 85 | 86 | if __name__ == "__main__": 87 | main() 88 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/README.md: -------------------------------------------------------------------------------- 1 | # DSC-DSU | Python Bootcamp 2020 2 | 3 |

4 | 5 | - ## Week 4 | PyQt5 , Flask and the way forward 6 | 7 | - ### [Day 1](day_1/) 8 | 9 | - Introduction to PyQt5 10 | - Installation of PyQt5 11 | - Layouts and widgets 12 | - Event handling 13 | 14 | - ### [Day 2](day_2/) 15 | 16 | - Introduction to Flask 17 | - installation of Flask 18 | - web apps 19 | - Jinja2 templeting 20 | 21 | - ### [Day 3](day_3/) 22 | - The way forward -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/README.md: -------------------------------------------------------------------------------- 1 | ![Screenshots/1.png](Screenshots/1.png) 2 | 3 | ![Screenshots/2.png](Screenshots/1.png) 4 | 5 | ![Screenshots/3.png](Screenshots/1.png) 6 | 7 | ![Screenshots/4.png](Screenshots/1.png) 8 | 9 | ![Screenshots/5.png](Screenshots/1.png) 10 | 11 | ![Screenshots/6.png](Screenshots/1.png) 12 | 13 | ![Screenshots/7.png](Screenshots/1.png) 14 | 15 | ![Screenshots/8.png](Screenshots/1.png) 16 | 17 | ![Screenshots/9.png](Screenshots/1.png) 18 | 19 | ![Screenshots/10.png](Screenshots/1.png) 20 | 21 | ![Screenshots/11.png](Screenshots/1.png) 22 | 23 | ![Screenshots/12.png](Screenshots/1.png) 24 | 25 | ![Screenshots/13.png](Screenshots/1.png) 26 | 27 | ![Screenshots/14.png](Screenshots/1.png) 28 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/1.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/10.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/11.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/12.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/13.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/14.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/2.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/3.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/4.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/5.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/6.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/7.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/8.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_2/MongoDB Atlas Configuration/Screenshots/9.png -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | This is the session covering python web framework Flask basics, templating with Jinja and Simple social media web app using flask 4 | ### What is Flask? 5 | Flask is a web application framework written in Python. It is developed by Armin Ronacher, who leads an international group of Python enthusiasts named Pocco. Flask is based on the Werkzeug WSGI toolkit and Jinja2 template engine. Both are Pocco projects. 6 | Flask documentation: https://flask.palletsprojects.com/en/1.1.x/ 7 | 8 | ### WSGI 9 | Web Server Gateway Interface (WSGI) has been adopted as a standard for Python web application development. WSGI is a specification for a universal interface between the web server and the web applications. 10 | 11 | ### Jinja2 12 | Jinja2 is a popular templating engine for Python. A web templating system combines a template with a certain data source to render dynamic web pages. 13 | Flask is often referred to as a micro framework. It aims to keep the core of an application simple yet extensible. Flask does not have a built-in abstraction layer for database handling, nor does it have form validation support. Instead, Flask supports the extensions to add such functionality to the application. 14 | Jinja documentation: https://jinja.palletsprojects.com/en/2.11.x/ 15 | 16 | ### Installing Flask 17 | ```bash 18 | pip install Flask 19 | ``` 20 | 21 | ### Hello World in Flask 22 | ```bash 23 | from flask import Flask 24 | app = Flask(__name__) 25 | 26 | @app.route('/') 27 | def hello_world(): 28 | return 'Hello World’ 29 | 30 | if __name__ == '__main__': 31 | app.run() 32 | ``` 33 | Importing the flask module in the project is mandatory. An object of the Flask class is our WSGI application. 34 | The Flask constructor takes the name of the current module (__name__) as an argument. 35 | The route() function of the Flask class is a decorator, which tells the application which URL should call the associated function. 36 | 37 | ### Routing in Flask 38 | ```bash 39 | from flask import Flask 40 | app = Flask(__name__) 41 | 42 | @app.route('/home/') 43 | def hello_name(name): 44 | return ("Hello" + name) 45 | 46 | @app.route('/home/') 47 | def hello_int(number): 48 | return ("Hello" + number) 49 | 50 | @app.route('/home/') 51 | def hello_float(number): 52 | return ("Hello" + number) 53 | 54 | if __name__ == '__main__': 55 | app.run() 56 | ``` 57 | Run the above code from Python Shell. Visit the URL http://localhost:5000/home/John in the browser. 58 | The given string is used as argument to the hello_name() function. The browser displays the following output − 59 | ``` 60 | Hello John 61 | ``` 62 | Same as for other variable types url the following result should appear: 63 | For http://localhost:5000/home/11: 64 | ``` 65 | Hello 11 66 | ``` 67 | For http://localhost:5000/home/1.1: 68 | ``` 69 | Hello 1.1 70 | ``` 71 | 72 | ### Templating in Flask 73 | Consider the following HTML code we will save this code as home.html: 74 | ```html 75 | 76 | 77 |

FLASK

78 | 79 | 80 | ``` 81 | Now write python script: 82 | ```bash 83 | from flask import Flask,render_template 84 | app = Flask(__name__) 85 | 86 | @app.route('/') 87 | def index(): 88 | return render_template(‘home.html’) 89 | 90 | if __name__ == '__main__': 91 | app.run(debug = True) 92 | ``` 93 | This will render template of home.html and "FLASK" will be displayed in browser. 94 | 95 | Similary for displaying name html code below: 96 | ```html 97 | 98 | 99 |

Hello {{ name }}!

100 | 101 | 102 | ``` 103 | Python code below: 104 | ```bash 105 | from flask import Flask, render_template 106 | app = Flask(__name__) 107 | 108 | @app.route('/hello/') 109 | def hello_name(user): 110 | return render_template('hello.html', name = user) 111 | 112 | if __name__ == '__main__': 113 | app.run(debug = True) 114 | ``` 115 | 116 | ### Handling HTML Form Data with Flask 117 | To get the form data from html forms we will be using POST method in app.route method and request function from Flask library to get form element. 118 | HTML code: 119 | ```html 120 |
121 | 122 | 123 |
124 | ``` 125 | 126 | Python code: 127 | ```bash 128 | from flask import Flask,render_template,request 129 | app = Flask(__name__) 130 | 131 | @app.route('/') 132 | def index(): 133 | return render_template(‘form.html’) 134 | 135 | @app.route('/handle_form', methods=['POST']) 136 | def handle_data(): 137 | text = request.form['text_data'] 138 | return render_template("name.html") 139 | 140 | if __name__ == '__main__': 141 | app.run(debug = True) 142 | ``` 143 | 144 | ### Building a Simple Social Media Web App 145 | 146 | Covered in live session. 147 | 148 | The social media web-app will: 149 | 150 | - Register User 151 | - Login User 152 | - Post and display user tweets 153 | 154 | Some necessary libraries to import and install: 155 | -WTForms 156 | -Pymongo 157 | Installation: 158 | ``` 159 | pip install WTForms 160 | ``` 161 | ``` 162 | pip install pymongo 163 | ``` 164 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/main.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, render_template, flash, redirect, url_for, session, request 2 | from wtforms import Form, StringField, PasswordField, validators 3 | import pymongo as pym 4 | import datetime 5 | 6 | app = Flask(__name__) 7 | app.secret_key='secret123' 8 | 9 | connection = pym.MongoClient("mongodb+srv://mtk:1234@cluster0.pepx2.mongodb.net/test?retryWrites=true&w=majority") 10 | db = connection.get_database('test') 11 | 12 | user_detail = pym.collection.Collection(db, 'User_details') 13 | tweets = pym.collection.Collection(db, 'tweets') 14 | 15 | @app.route('/index',methods=['GET','POST']) 16 | def index(): 17 | if request.method == 'POST': 18 | tweet = request.form['tweet'] 19 | ct = datetime.datetime.now() 20 | 21 | user_id = tweets.insert_one( 22 | { "email" : session["username"], 23 | "tweet" : tweet, 24 | "name" : session['name'], 25 | "time" : str(ct.date()) 26 | }) 27 | 28 | #flash('Tweet added', 'success') 29 | 30 | answer = get_tweets(session['username']) 31 | return render_template('Home.html',dictionary=answer) 32 | 33 | answer = get_tweets(session['username']) 34 | return render_template('Home.html',dictionary=answer) 35 | 36 | # Register Form Class 37 | class RegisterForm(Form): 38 | email = StringField('Email', [validators.Email()]) 39 | name = StringField('Name', [validators.Length(min=1, max=50)]) 40 | password = PasswordField('Password', [ 41 | validators.DataRequired(), 42 | validators.EqualTo('confirm', message='Passwords do not match') 43 | ]) 44 | confirm = PasswordField('Confirm Password') 45 | 46 | 47 | # User Register 48 | @app.route('/', methods=['GET', 'POST']) 49 | def register(): 50 | 51 | users = user_detail 52 | 53 | form = RegisterForm(request.form) 54 | if request.method == 'POST' and form.validate(): 55 | name = form.name.data 56 | email = form.email.data 57 | password = form.password.data 58 | 59 | myid = users.insert_one( 60 | { "name" : name, 61 | "email" : email, 62 | "password" : password 63 | }) 64 | 65 | flash('You are now registered and can log in', 'success') 66 | 67 | redirect(url_for('login')) 68 | #return render_template('Login.html') 69 | return render_template('Register.html', form=form) 70 | 71 | # User login 72 | @app.route('/login', methods=['GET', 'POST']) 73 | def login(): 74 | user = user_detail 75 | if request.method == 'POST': 76 | # Get Form Fields 77 | email = request.form['username'] 78 | password_candidate = request.form['password'] 79 | 80 | response = user.find_one({'email': email}) 81 | 82 | 83 | if response: 84 | password = response['password'] 85 | 86 | # Compare Passwords 87 | if password_candidate == password: 88 | # Passed 89 | session['logged_in'] = True 90 | session['username'] = email 91 | session['name'] = response['name'] 92 | 93 | flash('You are now logged in', 'success') 94 | return redirect(url_for('index')) 95 | else: 96 | error = 'Invalid login' 97 | return render_template('Login.html', error=error) 98 | else: 99 | error = 'Email not found' 100 | return render_template('Login.html', error=error) 101 | 102 | return render_template('Login.html') 103 | 104 | # Logout 105 | @app.route('/logout') 106 | def logout(): 107 | session.clear() 108 | flash('You are now logged out', 'success') 109 | return redirect(url_for('login')) 110 | 111 | 112 | def get_tweets(email): 113 | pid = tweets.find({"email":session['username']}) 114 | 115 | user_tweets = {} 116 | y = {} 117 | for docs in pid: 118 | y = docs 119 | key = y["_id"] 120 | user_tweets.setdefault(key,[]) 121 | user_tweets[key].append(y["name"]) 122 | user_tweets[key].append(y["tweet"]) 123 | user_tweets[key].append(y["time"]) 124 | 125 | return user_tweets 126 | 127 | 128 | if __name__ == '__main__': 129 | app.run() 130 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/templates/Home.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | W3.CSS Template 5 | 6 | 7 | 8 | 9 | 10 | 11 | 14 | 15 | 16 | 17 |
18 |
19 | 20 | Logo 21 | 22 | 23 | 24 |
25 | 26 |
27 | 28 | Avatar 29 | 30 | Logout 31 |
32 |
33 | 34 | 35 |
36 | 37 |
38 | 39 |
40 | 41 |
42 |
43 |

My Profile

44 |

Avatar

45 |
46 |

Designer, UI

47 |

London, UK

48 |

April 1, 1988

49 |
50 |
51 |
52 | 53 | 54 |
55 |
56 | 57 | 58 | 59 |
60 |
61 |
62 |
63 | 64 |
65 | 66 | 67 |
68 | 69 |
70 |
71 |
72 | 78 |
79 |
80 |
81 | 82 | {% for key, value in dictionary.items() %} 83 |

84 | Avatar 85 | {{value[2]}} 86 |

{{value[0]}}


87 |
88 |

{{value[1]}}

89 | 90 | 91 |
92 | {% endfor %} 93 | 94 |
95 | 96 |
97 | 98 | 99 |
100 |
101 | 102 | 103 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/templates/Login.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | Login 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 26 | 27 | 28 |
29 |
30 |
31 |

Login

32 |
33 |
34 |
35 | 36 | 37 |
38 |
39 | 40 | 41 |
42 | 43 |
44 |
45 |
46 |
47 | 48 | 49 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/templates/Register.html: -------------------------------------------------------------------------------- 1 | 2 | Register 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 25 | 26 |
27 |
28 |
29 |

Register

30 |
31 | {% from "includes/_formhelpers.html" import render_field %} 32 |
33 |
34 | {{render_field(form.name, class_="form-control")}} 35 |
36 |
37 | {{render_field(form.email, class_="form-control")}} 38 |
39 |
40 | {{render_field(form.password, class_="form-control")}} 41 |
42 |
43 | {{render_field(form.confirm, class_="form-control")}} 44 |
45 |

46 |
47 |
48 | 49 |
50 |
51 | 52 | 53 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/templates/includes/_formhelpers.html: -------------------------------------------------------------------------------- 1 | 2 | {% macro render_field(field) %} 3 | {{ field.label }} 4 | {{ field(**kwargs)|safe }} 5 | {% if field.errors %} 6 | {% for error in field.errors %} 7 | {{ error }} 8 | {% endfor %} 9 | {% endif %} 10 | {% endmacro %} -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_2/templates/includes/_messages.html: -------------------------------------------------------------------------------- 1 | {% with messages = get_flashed_messages(with_categories=true) %} 2 | {% if messages %} 3 | {% for category, message in messages %} 4 |
{{ message }}
5 | {% endfor %} 6 | {% endif %} 7 | {% endwith %} 8 | 9 | {% if error %} 10 |
{{error}}
11 | {% endif %} 12 | 13 | {% if msg %} 14 |
{{msg}}
15 | {% endif %} -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_3/README.md: -------------------------------------------------------------------------------- 1 | # Lecture Notes 2 | 3 | # Python Bootcamp | DSC@DSU 4 | 5 |

6 | 7 | ## Week 4, Day 3 8 | 9 | By _Tarun Kumar_, **Python@DSU Lead** and Core Team **DSC@DSU** 10 | _Bahawal Baloch_, **Python@DSU Co-Lead** and Core Team **DSC@DSU** 11 | 12 | # [Lecture Notes](presentation.pdf) 13 | -------------------------------------------------------------------------------- /python_bootcamp_2020/week_4/day_3/presentation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinnytk/DSC-DSU/b8f4653c0617e968b2a73a8165986ddf87317e3e/python_bootcamp_2020/week_4/day_3/presentation.pdf -------------------------------------------------------------------------------- /version_control_with_git/lecture_notes.md: -------------------------------------------------------------------------------- 1 | # Version Control with Git 2 | 3 | Tarun Kumar, *Core Member DSC@DSU* 4 | 5 | # Some questions to derive version control systems 6 | 7 | - How do you share your code with your project partner? 8 | - Upload it somewhere? (Google Drive, email etc) 9 | - Share using USB or other hardware media? 10 | - Can the above discussed methods allow the following: 11 | - Make changes to the code in parallel to you and simultaneously synchronize (without sharing the code back and forth)? 12 | - Keep track of whatever changes s(he) are making? (Code added/deleted/modified) 13 | - Know why those changes were made without them explicitly going over all the changes they made? 14 | - Move back to an older version of code after noticing the new changes are unwanted (causing bugs or are otherwise, we're talking hundreds of code lines added) 15 | 16 | ## What is a Version Control system? 17 | 18 | - Tools to keep track of changes (history) of a directory and it's containing files 19 | 20 | (usually in our cases, ***Source Codes***) 21 | 22 | - Keep information(metadata) about those changes 23 | - So in short, something we need to properly collaborate and contribute. 24 | - Features of a version control system 25 | - Code collaboration 26 | - Keep a history of versions 27 | - Keeping metadata about versions (who, when, why, which/what) 28 | - Moving back and forth versions 29 | - and much more.. 30 | - Examples of all commit VCS: 31 | - Google Docs 32 | - Notion 33 | 34 | ## What is Git? 35 | 36 | Most commonly used Version Control system and the one we'll deep dive today. 37 | 38 | Created by Linus Torvalds, the same guy behind ***Linux*** operating system. 39 | 40 | Subversion, Mercurial are also famous version control systems. 41 | 42 | ## *And how does Git work?* 43 | 44 | - **Git Data Model** 45 | - History as **Directed Acyclic Graphs (DAG)** 46 | - **Snapshots** (trees) 47 | 48 | or, more explicitly: 49 | 50 | - **Commits** (snapshots + metadata) 51 | - Parent (commit before) 52 | - Author 53 | - Message 54 | - Snapshot 55 | - **Repository** 56 | - A directory injected with Git version control 57 | - **Staging Area** 58 | - The place where you explicitly changes go before becoming a snapshot (a commit) 59 | 60 | # Installing and integrating Git 61 | 62 | Command-line interface 63 | 64 | - Windows (CMD, Powershell) 65 | - Linux/MacOS (Bash - zsh) 66 | 67 | GUI based: 68 | 69 | - Editor or IDE integration (Sublime, VSCode, Android Studio etc) 70 | - [https://git-scm.com/downloads/guis](https://git-scm.com/downloads/guis) 71 | 72 | # Working with git interface 73 | 74 | - **Basic** 75 | 76 | - A description of command and it's usage: 77 | 78 | ```bash 79 | git help 80 | ``` 81 | 82 | - Initializing a git repository(local) 83 | 84 | ```bash 85 | git init 86 | ``` 87 | 88 | - Adding files to staging area (to track their changes): 89 | 90 | ```bash 91 | git add #track a single file 92 | git add --all #track all files in the repository 93 | ``` 94 | 95 | - Creating snapshots (commits) 96 | 97 | ```bash 98 | git commit 99 | git commit -m "message here" #write message without opening a editor 100 | ``` 101 | 102 | - Checking the history of commits and changes we've made: 103 | 104 | ```bash 105 | git log 106 | git log -a --graph --decorate # A more beautiful way to visualize the tree 107 | git log -a --graph --decorate --oneline #Above but concise 108 | ``` 109 | 110 | - Checking specific differences we've made to tracked files (in ref with staging area): 111 | 112 | ```bash 113 | git diff 114 | git diff # difference between commits, not staging area 115 | ``` 116 | 117 | - Creating a gitignore file 118 | - **Collaborating and branching** 119 | 120 | - Creating an alternative branch in our current tree 121 | 122 | ```bash 123 | git branch # Shows all branches 124 | git branch # creates branch with name "branch_name" 125 | ``` 126 | 127 | - Working on branches 128 | 129 | ```bash 130 | git checkout # Change working branch to "branch_name" 131 | git checkout -b # Single command to create and checkout 132 | ``` 133 | 134 | - Merging 135 | 136 | ```bash 137 | git merge 138 | 139 | # If you get conflicts during merging 140 | # and once you're done with fixing them: 141 | git merge --continue 142 | ``` 143 | 144 | - **Remotes** 145 | 146 | - Show all the remotes where this repository is deployed 147 | 148 | ```bash 149 | git remote 150 | ``` 151 | 152 | - Adding new remotes 153 | 154 | ```bash 155 | git remote add 156 | ``` 157 | 158 | - Pushing local repository changes to remote 159 | 160 | ```bash 161 | git push : 162 | 163 | # in case upstream is not set up 164 | git branch --set-upstream-to=/ 165 | # or setting upstream directly in pus 166 | ``` 167 | 168 | - Syncing changes with remote 169 | 170 | ```bash 171 | git fetch # only retrieve changes 172 | git pull # git fetch and then merge 173 | ``` 174 | 175 | - Starting from existing remote repository 176 | 177 | ```bash 178 | git clone 179 | ``` 180 | 181 | # Working with Github interface 182 | 183 | - Public vs private repositories 184 | - Pull requests 185 | - Difference between cloning and forking 186 | - readme.md 187 | 188 | # Things not covered? 189 | 190 | - Open Source Licenses (what people can and cannot do with your code) 191 | - Continuous Integration (auto deploy a piece of software once you push to a Github repo) 192 | - Setting automated testing for Github repositories 193 | - Some advanced commands worth looking into: 194 | 195 | ```bash 196 | git rebase # change merge tip from ancestor commit to something else 197 | git bisect # automate searching commit history in a binary fashion 198 | git stash # VERY useful to store changes 199 | git add -p # interactive adding, selecting which changes to add to staging 200 | ``` 201 | 202 | # Extra resources: 203 | 204 | 205 | 206 | - Primary review source and motivation for this video, [Lecture 6: Version Control of Missing Semester](https://www.youtube.com/watch?v=2sjqTHE0zok) 207 | - [Pro Git book](https://git-scm.com/book/en/v2) is the ultimate resource for learning and mastering Git, so it's suggested you read at least the first few chapters 208 | - [Learn Git Branching](https://learngitbranching.js.org/) is for the ones who want to master the Git interface using interactive tutorials and tests. 209 | - [Oh Shit, Git!?!](https://ohshitgit.com/) is exactly what you sound like when you make a mistake in Git (11 out of 10 times). A great resource as well to recover from those common pitfalls. 210 | - [How to write good commit messages by FreeCodeCamp](https://www.freecodecamp.org/news/writing-good-commit-messages-a-practical-guide/) must watch guide on common commit message conventions to make the most out of your code contributions. --------------------------------------------------------------------------------