├── .github └── workflows │ └── build_deploy.yml ├── .gitignore ├── Jupyter_to_Software.pdf ├── LICENSE ├── Makefile ├── README.md ├── _static ├── academis.css ├── academis_header.png ├── academis_logo.png └── favicon.ico ├── challenges ├── backpack_problem.rst ├── binary-search.png ├── binary_search.py ├── binary_search.rst ├── binary_tree.png ├── blockchain.rst ├── burglar.png ├── burglar.svg ├── chained_list.py ├── chained_list.rst ├── dice │ ├── dice.rst │ ├── double_dice.png │ └── pillow_dice.png ├── factorial.rst ├── josephus.rst ├── magic_square.rst ├── maze.rst ├── memory │ ├── 10images.zip │ ├── memory.jpg │ ├── memory.png │ └── memory.rst ├── metaclass.rst ├── monks.jpg ├── sorting.rst ├── tennis.rst ├── tree_array.png ├── tree_traversal.rst └── tsp.rst ├── classes ├── abc.rst ├── class_diagram.md ├── class_playing_field.png ├── class_playing_field_alt.png ├── class_snake.png ├── classes.rst ├── composition.rst ├── decorator_class.rst ├── inheritance.rst ├── metaclasses.rst ├── oop_simple.png └── operator_overloading.rst ├── concurrency ├── README.rst ├── async_factorial.py ├── factorial.py ├── gauss_elim.py ├── subprocess_factorial.py ├── test_gauss_elim.py └── thread_factorial.py ├── conf.py ├── error_handling ├── debugging.rst ├── exceptions.rst ├── generate_maze_buggy.py ├── get_traceback.py ├── interactive_debugger.rst ├── loggers.py ├── logging.rst ├── logging_example.py └── warnings.rst ├── examples ├── breakout.png ├── breakout.py ├── breakout.rst ├── snake.py ├── tetris.png ├── tetris.py └── tetris.rst ├── exercises ├── class_inheritance.py ├── collection_exercise.py ├── comprehension_exercise.py ├── exercise_fibonacci.py ├── exercise_logging.py ├── exercises.pdf └── exercises_and_solutions.pdf ├── functions ├── decorators.rst ├── function_parameters.rst ├── functools.rst ├── generators.rst ├── levels.rst └── scope.rst ├── getting_started ├── create_repo.png ├── git_dialog.png ├── git_repo.rst ├── git_url.png ├── structure.rst └── virtualenv.rst ├── hall_of_fame.rst ├── index.rst ├── introspection.rst ├── links.rst ├── performance ├── mandelbrot.py └── profiling.rst ├── pip.rst ├── project.rst ├── quality ├── continuous_integration.rst └── packaging.rst ├── requirements.txt ├── shortcuts ├── collections.rst ├── comprehensions.rst └── enums.rst ├── software_engineering ├── README.rst ├── code_review.rst ├── flowchart.png ├── flowchart.svg ├── grid_model.png ├── grid_model.svg ├── prototype.png ├── prototype.rst ├── prototype_curses.py ├── prototype_opencv.py ├── software_engineering.png ├── software_engineering.svg ├── tiles.zip └── title.png ├── structure ├── commandline_args.rst ├── main_block.rst ├── modules.rst ├── namespaces.png ├── namespaces.rst ├── structuring_python_programs.png └── structuring_python_programs.svg └── testing ├── facade.rst └── unit_test.rst /.github/workflows/build_deploy.yml: -------------------------------------------------------------------------------- 1 | 2 | name: deploy advanced python 3 | 4 | on: 5 | push: 6 | branches: [ master ] 7 | 8 | jobs: 9 | 10 | build: 11 | name: Build 12 | runs-on: ubuntu-latest 13 | steps: 14 | 15 | - name: checkout repo 16 | uses: actions/checkout@v1 17 | 18 | - name: build static html 19 | run: | 20 | python -m pip install --upgrade pip 21 | pip install -r requirements.txt 22 | make html 23 | 24 | - name: copy to academis server 25 | uses: appleboy/scp-action@master 26 | with: 27 | host: ${{ secrets.ACADEMIS_HOST }} 28 | username: ${{ secrets.ACADEMIS_USERNAME }} 29 | port: 22 30 | key: ${{ secrets.SSH_PRIVATE_KEY }} 31 | source: build/html/* 32 | target: /www/academis/advanced_python 33 | rm: true 34 | strip_components: 2 35 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | 103 | software_engineering/tiles/* 104 | -------------------------------------------------------------------------------- /Jupyter_to_Software.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/Jupyter_to_Software.pdf -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Kristian Rother 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line, and also 5 | # from the environment for the first two. 6 | SPHINXOPTS ?= 7 | SPHINXBUILD ?= sphinx-build 8 | SOURCEDIR = . 9 | BUILDDIR = build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # Advanced Python 3 | 4 | Examples and Exercises for advanced Python programming techniques 5 | 6 | The Live Website is available at [http://www.academis.eu/advanced_python](http://www.academis.eu/advanced_python) 7 | 8 | ## Source 9 | 10 | [github.com/krother/advanced_python](https://github.com/krother/advanced_python) 11 | 12 | ## License 13 | 14 | © 2018 Dr. Kristian Rother (krother@academis.eu) 15 | 16 | The code is distributed under the conditions of the MIT License. 17 | -------------------------------------------------------------------------------- /_static/academis.css: -------------------------------------------------------------------------------- 1 | 2 | body { 3 | margin-top: 250px; 4 | margin-bottom: 250px; 5 | background-image: url("academis_header.png"); 6 | background-repeat: no-repeat; 7 | } 8 | -------------------------------------------------------------------------------- /_static/academis_header.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/_static/academis_header.png -------------------------------------------------------------------------------- /_static/academis_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/_static/academis_logo.png -------------------------------------------------------------------------------- /_static/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/_static/favicon.ico -------------------------------------------------------------------------------- /challenges/backpack_problem.rst: -------------------------------------------------------------------------------- 1 | Backpack Problem 2 | ================ 3 | 4 | **🎯 Optimize the value of a heist.** 5 | 6 | .. figure:: burglar.png 7 | :alt: Burglar 8 | 9 | 10 | A burglar broke into a villa. There he finds so many valuables that he 11 | can’t put them all in his backpack. Write a program that makes an 12 | optimal selection. 13 | 14 | The burglar is an experienced professional who can estimate the market 15 | value and size of each item in no time: 16 | 17 | ================ ==== ====== 18 | item size value 19 | ================ ==== ====== 20 | laptop 2 600,- 21 | cutlery 2 400,- 22 | spotify speakers 3 300,- 23 | jewels 2 1100,- 24 | vase 5 700,- 25 | camera 2 500,- 26 | painting 4 900,- 27 | cash 1 800,- 28 | ================ ==== ====== 29 | 30 | The backpack has a capacity of ``8``. 31 | 32 | When your program manages to pack items worth ``3000``, it can be used 33 | as an app for amateur burglars. 34 | 35 | Hints 36 | ----- 37 | 38 | **The optimal solution uses dynamic programming.** 39 | 40 | Use the following pseudocode: 41 | 42 | 1. create an empty data structure for the best combinations for each backpack size 43 | 2. insert an empty combination for a size 0 backpack 44 | 3. start with a size 1 backpack 45 | 4. store the best combination for the current size minus one as ``current best`` 46 | 5. go through all items 47 | 6. create a new combination usign an item plus the best combination for 48 | the space remaining 49 | 7. if the combination is more valuable than the ``current best``, 50 | replace ``current best`` by the new combination 51 | 8. if the combination is worth the same amount, save both 52 | 9. increase the size of the backpack by 1 53 | 10. repeat step 4 until you reach the desired size 54 | 55 | *Translated with* `www.DeepL.com `__ 56 | -------------------------------------------------------------------------------- /challenges/binary-search.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/binary-search.png -------------------------------------------------------------------------------- /challenges/binary_search.py: -------------------------------------------------------------------------------- 1 | 2 | # create random numbers 3 | from random import randint, seed 4 | 5 | seed(42) 6 | N = 1_000_000 7 | ids = [randint(1_000_000, 9_000_000) for i in range(N)] 8 | 9 | 10 | def search(query, ids): 11 | """brute-force search""" 12 | for i in ids: 13 | if i == query: 14 | return i 15 | return -1 16 | 17 | def binary_search(query, ids, start, stop): 18 | """recursive binary search""" 19 | if ids[start] == query: 20 | return start 21 | elif stop - start <= 1: 22 | return -1 23 | split = start + (stop - start) // 2 24 | if ids[split] > query: 25 | return binary_search(query, ids, start, split) 26 | else: 27 | return binary_search(query, ids, split, stop) 28 | 29 | ids.sort() 30 | 31 | #%timeit search(8997173, ids) 32 | 33 | #%timeit binary_search(8997173, ids, 0, len(ids)) 34 | -------------------------------------------------------------------------------- /challenges/binary_search.rst: -------------------------------------------------------------------------------- 1 | Binary Search 2 | ============= 3 | 4 | **🎯 Measure the performance of an algorithm.** 5 | 6 | A search problem 7 | ---------------- 8 | 9 | Suppose you have a list of 1 million numbers: 10 | 11 | .. code:: python3 12 | 13 | from random import randint, seed 14 | 15 | seed(42) 16 | N = 1_000_000 17 | ids = [randint(1_000_000, 9_000_000) for i in range(N)] 18 | 19 | We want to know at which position the number ``8997173`` occurs – or 20 | whether it occurs at all. 21 | 22 | 23 | Brute-force solution 24 | -------------------- 25 | 26 | Of course, you could loop through all the numbers: 27 | 28 | .. code:: python3 29 | 30 | def search(query, ids): 31 | """brute-force search""" 32 | for i in ids: 33 | if i == query: 34 | return i 35 | return -1 36 | 37 | search(8997173, ids) 38 | 39 | Because of the ``for`` loop, the time cost of this function grows 40 | **linearly**. If you search twice as much data, it will take twice as 41 | long. The **Big-O-Notation** for this would be O(N). 42 | 43 | 44 | Recursive solution 45 | ------------------ 46 | 47 | An alternative approach is **binary search**, one of the most 48 | fundamental recursive algorithms: 49 | 50 | .. code:: python3 51 | 52 | def binary_search(query, ids, start, stop): 53 | """recursive binary search""" 54 | if ids[start] == query: 55 | return start 56 | elif stop - start <= 1: 57 | return -1 58 | 59 | split = start + (stop - start) // 2 60 | if ids[split] > query: 61 | return binary_search(query, ids, start, split) 62 | else: 63 | return binary_search(query, ids, split, stop) 64 | 65 | ids.sort() 66 | binary_search(8997173, ids, 0, len(ids)) 67 | 68 | The function requires the data to be sorted. The search itself runs in 69 | **O(log N)** or **logarithmic** time. If you search twice as much data, 70 | only one extra recursion is necessary. 71 | 72 | Visual Explanation 73 | ------------------ 74 | 75 | .. figure:: binary-search.png 76 | :alt: BINÄRY SEARCH 77 | 78 | *source: idea-instructions.com, CC-BY-NC-SA 4.0* 79 | 80 | 81 | Challenge 82 | --------- 83 | 84 | Measure the time it takes both algorithms to run on 1 million numbers. 85 | In IPython/Jupyter, you can use the magic function ``%time``: 86 | 87 | .. code:: ipython3 88 | 89 | In [2]: %time search(8997173, ids) 90 | 91 | Compare the time required by the brute-force and binary implementations. 92 | Then increase the size of the data by factor 10 and repeat the 93 | measurement. 94 | 95 | Questions 96 | ~~~~~~~~~ 97 | 98 | - Which of the searches is faster? 99 | - How much time does the sorting take? 100 | - When does binary search pay off? 101 | - For what data would the brute-force algorithm be faster? 102 | -------------------------------------------------------------------------------- /challenges/binary_tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/binary_tree.png -------------------------------------------------------------------------------- /challenges/blockchain.rst: -------------------------------------------------------------------------------- 1 | Blockchain 2 | ========== 3 | 4 | **🎯 Implement your own blockchain algorithm.** 5 | 6 | Step 1 7 | ------ 8 | 9 | Write a function that generates random transactions in the format 10 | ``(name1, name2, amount)``. 11 | 12 | We want to save these transactions *forgery-proof*, so that they are as 13 | difficult to manipulate as possible afterwards. 14 | 15 | Step 2 16 | ------ 17 | 18 | Define a data type *"Block"* that contains the following: 19 | 20 | - The hash of a previous block 21 | - Some transactions 22 | - A checksum (any number or string) 23 | 24 | Step 3 25 | ------ 26 | 27 | Write a function that calculates a hash from all properties of a block. 28 | To do this, represent the entire block as a string. Use the hash 29 | function ``sha256``: 30 | 31 | :: 32 | 33 | import hashlib 34 | 35 | h = hashlib.sha256() 36 | h.update(text.encode()) 37 | print(h.hexdigest()) 38 | 39 | Step 4 40 | ------ 41 | 42 | Create the blockchain as an empty list. 43 | 44 | Create the first block, the "Genesis block". Use ‘genesis’ as previous 45 | hash. Place some random transactions in the block. 46 | 47 | Find a checksum so that the *sha256-hexdigest* ends with four zeros 48 | (``0000``). You may need to try many checksums. 49 | 50 | Add the finished block to the blockchain. 51 | 52 | Step 5 53 | ------ 54 | 55 | Create the second block: 56 | 57 | - The hash is the ``hexdigest`` of the previous block 58 | - Add more transactions. 59 | - Again find a checksum that generates a ``hexdigest`` with four zeros 60 | at the end. 61 | - Add the finished block to the blockchain. 62 | 63 | Step 6 64 | ------ 65 | 66 | Generate more blocks. 67 | 68 | Questions 69 | --------- 70 | 71 | - What happens if the number of necessary zeros in the hex digest is 72 | set to 2 or 6? 73 | - What happens if someone changes a transaction in the Genesis block? 74 | - What makes the blockchain forgery-proof? 75 | - How could a blockchain still be forged? 76 | - Why is finding the checksum also called *"proof of work"*? 77 | - Why are several computers involved in a blockchain? 78 | - What is a *"consensus algorithm"*? 79 | 80 | *Translated with* `www.DeepL.com `__ 81 | -------------------------------------------------------------------------------- /challenges/burglar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/burglar.png -------------------------------------------------------------------------------- /challenges/chained_list.py: -------------------------------------------------------------------------------- 1 | 2 | from timeit import timeit 3 | from random import random 4 | 5 | N = 100000 6 | 7 | def front_to_end(data): 8 | elem = data.pop(0) 9 | data.append(elem) 10 | 11 | 12 | class ChainElement: 13 | 14 | def __init__(self, val): 15 | self.value = val 16 | self.next = None 17 | 18 | 19 | class ChainedList: 20 | 21 | def __init__(self, n): 22 | self.first = ChainElement(random()) 23 | self.last = self.first 24 | 25 | for i in range(n-1): 26 | self.last.next = ChainElement(random()) 27 | self.last = self.last.next 28 | 29 | def front_to_end(self): 30 | self.last.next = self.first 31 | new_first = self.first.next 32 | self.last = self.first 33 | self.first.next = None 34 | self.first = new_first 35 | 36 | 37 | data = [random() for i in range(N)] 38 | chain = ChainedList(N) 39 | 40 | t1 = timeit('front_to_end(data)', globals=locals()) 41 | t2 = timeit('chain.front_to_end()', globals=locals()) 42 | 43 | print(f"time --- list: {t1:6.3} chained: {t2:6.3}") 44 | -------------------------------------------------------------------------------- /challenges/chained_list.rst: -------------------------------------------------------------------------------- 1 | Chained List 2 | ============ 3 | 4 | **🎯 Identify strengths and weaknesses of data structures.** 5 | 6 | Cycle Entries 7 | ------------- 8 | 9 | In this problem, we need to **remove the first element of a sequence** 10 | and **add it to the end**. 11 | 12 | An implementation using a Python list could look like this: 13 | 14 | .. code:: python3 15 | 16 | from random import random 17 | 18 | def front_to_end(data): 19 | elem = data.pop(0) 20 | data.append(elem) 21 | 22 | N = 100 23 | data = [random() for i in range(N)] 24 | front_to_end(data) 25 | 26 | 27 | Measure time behavior 28 | --------------------- 29 | 30 | This operation looks harmless. Is it? 31 | 32 | Use the ``timeit`` module to find out how long it runs. The following 33 | code sniplet prints out the time needed: 34 | 35 | .. code:: python3 36 | 37 | from timeit import timeit 38 | 39 | t = timeit('front_to_end(data)', globals=locals()) 40 | print(f"time used: {t:6.3}") 41 | 42 | Now scale up N (by adding one or more zeros) and see how the time 43 | changes. 44 | 45 | 46 | Using a Chained List 47 | -------------------- 48 | 49 | Let’s try an alternative data struture: the **chained list**. In a 50 | chained list, each element points to the next element in the chain (or 51 | ``None`` if it is the last element in the chain). 52 | 53 | It is common to keep the first and last element of a chain in extra 54 | variables. Here is a Python implementation using classes: 55 | 56 | .. code:: python3 57 | 58 | class ChainElement: 59 | 60 | def __init__(self, val): 61 | self.value = val 62 | self.next = None 63 | 64 | 65 | class ChainedList: 66 | """Helper class to manage the elements of a chain""" 67 | 68 | def __init__(self, n): 69 | self.first = ChainElement(random()) 70 | self.last = self.first 71 | 72 | for i in range(n-1): 73 | self.last.next = ChainElement(random()) 74 | self.last = self.last.next 75 | 76 | def front_to_end(self): 77 | """move the first element of the chain to the end""" 78 | ... 79 | 80 | # create a chained list 81 | chain = ChainedList(N) 82 | chain.front_to_end() 83 | 84 | 85 | The Challenge 86 | ------------- 87 | 88 | - implement the ``front_to_end()`` method so that it does an operation 89 | equivalent to the list function above 90 | - measure the time required for ``front_to_end()`` (without creating 91 | the chain) 92 | - try different values of N and compare the result to the list-based 93 | implementation 94 | - what are the pros and cons of either implementation? 95 | -------------------------------------------------------------------------------- /challenges/dice/dice.rst: -------------------------------------------------------------------------------- 1 | Dice 2 | ==== 3 | 4 | **🎯 Write a program that rolls two dice.** 5 | 6 | .. figure:: double_dice.png 7 | :alt: double dice 8 | 9 | Write a class ``Dice`` that: 10 | 11 | - displays the image of a dice 12 | - changes the dice image a few times (as animation effect) 13 | - shows a random dice result 14 | - stores the result as an attribute 15 | 16 | Write a program that creates two Dice objects. Re-roll both dice until 17 | they show the same number 18 | 19 | Hints 20 | ----- 21 | 22 | - for the graphics you can use ``arcade``. 23 | - the function ``time.sleep(x)`` waits ``x`` seconds 24 | - Use the following image for the dice: 25 | 26 | .. figure:: pillow_dice.png 27 | :alt: dice faces 28 | 29 | 30 | *Translated with* `www.DeepL.com `__ 31 | -------------------------------------------------------------------------------- /challenges/dice/double_dice.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/dice/double_dice.png -------------------------------------------------------------------------------- /challenges/dice/pillow_dice.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/dice/pillow_dice.png -------------------------------------------------------------------------------- /challenges/factorial.rst: -------------------------------------------------------------------------------- 1 | Factorials 2 | ========== 3 | 4 | **🎯 Calculate the factorial of a number (n!):** 5 | 6 | .. code:: python3 7 | 8 | 1 * 2 * 3 * 4 ... * 10 9 | 10 | .. code:: python3 11 | 12 | def factorial(n): 13 | """Returns a factorial of n.""" 14 | pass 15 | 16 | 17 | # Test code 18 | assert factorial(0) == 1 19 | assert factorial(1) == 1 20 | assert factorial(2) == 2 21 | assert factorial(3) == 6 22 | assert factorial(7) == 5040 23 | 24 | 25 | Extra Challenges 26 | ---------------- 27 | 28 | * implement the program as a *recursive function* 29 | 30 | 31 | *Translated with [www.DeepL.com](www.DeepL.com/Translator)* 32 | -------------------------------------------------------------------------------- /challenges/josephus.rst: -------------------------------------------------------------------------------- 1 | Josephus’ Problem 2 | ================= 3 | 4 | **🎯 Clear up murders in the monastery.** 5 | 6 | .. figure:: monks.jpg 7 | :alt: Sean Connery hunts the monk-killer 8 | 9 | Sean Connery hunts the monk-killer 10 | 11 | In a monastery there are 1 assassin and the following 9 monks: 12 | 13 | .. code:: python3 14 | 15 | monks = [ 16 | "Adalbertus", "Bonifacius", "Commodus", 17 | "Dominicus", "Emarius", "Franziskus", 18 | "Gustavus", "Henrik", "Iohannes" 19 | ] 20 | 21 | Every night the 5th monk is murdered (counting from the 1st monk or the 22 | last victim). The last monk catches the assassin. 23 | 24 | Which monk will survive? 25 | 26 | *Translated with*\ `www.DeepL.com `__ 27 | -------------------------------------------------------------------------------- /challenges/magic_square.rst: -------------------------------------------------------------------------------- 1 | Magic Square 2 | ============ 3 | 4 | **🎯 Solve a magic square.** 5 | 6 | - create a magic square with 3 \* 3 fields. 7 | - fill the square with the numbers from 1-9 8 | - the sum of the numbers in each row, column and diagonal shall be 15 9 | - use each number only once 10 | - print the finished square 11 | 12 | Hints 13 | ----- 14 | 15 | - Write a function that calculates all sums 16 | - A *brute-force* approach is to try out all permutations\* 17 | - See the ``itertools`` module 18 | 19 | Extra Challenge 20 | --------------- 21 | 22 | Fill a magic square with 4 \* 4 fields with the numbers 1-16 (sum 34). 23 | 24 | If you don’t want to try all the possibilities (9!), you can describe 25 | the magic square as a linear system of equations. The Python package 26 | **PuLP** allows you to express the necessary equations in a very compact 27 | way. 28 | 29 | *Translated with* `www.DeepL.com `__ 30 | -------------------------------------------------------------------------------- /challenges/maze.rst: -------------------------------------------------------------------------------- 1 | Graph Traversal 2 | =============== 3 | 4 | **🎯 Find your way out of the maze.** 5 | 6 | .. code:: python3 7 | 8 | maze = """ 9 | ############ 10 | # # ##S# 11 | ### # # 12 | ### ###### # 13 | ### # ## # 14 | # ## ## ## # 15 | # # # 16 | #X##########""".strip().split('\n') 17 | 18 | x, y = (10, 1) 19 | target = (1, 7) 20 | 21 | Write a function that will walk the maze (the graph) until the exit 22 | (``X``) is reached. 23 | 24 | Hints 25 | ----- 26 | 27 | You can proceed according to the **graph traversal algorithm**: 28 | 29 | 1. create a stack of the nodes to visit 30 | 2. create a stack of already visited nodes 31 | 3. take the next node from the stack 32 | 4. check whether the node is the exit, if yes, finish 33 | 5. if the node is a wall, continue to 3. 34 | 6. add the neighbours of the node to the nodes to visit 35 | 36 | Try out what changes when you replace the stack by a queue. 37 | 38 | *Translated with* `www.DeepL.com `__ 39 | -------------------------------------------------------------------------------- /challenges/memory/10images.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/memory/10images.zip -------------------------------------------------------------------------------- /challenges/memory/memory.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/memory/memory.jpg -------------------------------------------------------------------------------- /challenges/memory/memory.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/memory/memory.png -------------------------------------------------------------------------------- /challenges/memory/memory.rst: -------------------------------------------------------------------------------- 1 | Memory 2 | ====== 3 | 4 | **🎯 Write a memory game in which a player uncovers pairs of cards using 5 | the mouse or keyboard.** 6 | 7 | |image0| 8 | 9 | You can use the images in :download:`10images.zip` 10 | 11 | The images have a 256 x 256 pixel size and are placed 300 pixels apart. 12 | 13 | Hints 14 | ----- 15 | 16 | - the ``arcade`` library allows you to include graphics and act on user 17 | input. 18 | - you find more images on `unsplash.com `__ and 19 | `opengameart.org `__. 20 | 21 | *Translated with*\ `www.DeepL.com `__ 22 | 23 | Image sources 24 | ------------- 25 | 26 | All images have been taken from `unsplash.com `__: 27 | 28 | - Bubbles by Marko Blažević on Unsplash 29 | - Coffee by Nathan Dumlao on Unsplash 30 | - Ivy by asoggetti on Unsplash 31 | - Orange by Vino Li on Unsplash 32 | - Rainbow paint by Daniel Olah on Unsplash 33 | - Pebbles by John Salzarulo on Unsplash 34 | - Waterfall by Ben Guerin on Unsplash 35 | - Clouds by Zbynek Burival on Unsplash 36 | - White flower by Annie Spratt on Unsplash 37 | - Puddle by Erik Mclean on Unsplash 38 | 39 | .. |image0| image:: memory.jpg 40 | 41 | -------------------------------------------------------------------------------- /challenges/metaclass.rst: -------------------------------------------------------------------------------- 1 | Decorators with a Metaclass 2 | =========================== 3 | 4 | **🎯 Implement the decorator `add_two`, so that the following code works:** 5 | 6 | .. code:: python3 7 | 8 | @add_two 9 | def double(a): 10 | return a * 2 11 | 12 | print(double(20)) 13 | print(add_two(40)) 14 | 15 | **you should get the output:** 16 | 17 | :: 18 | 19 | 42 20 | 42 21 | 22 | Do not modify the above code! 23 | 24 | ---- 25 | 26 | Hints 27 | ----- 28 | 29 | In this task you can practice the following concepts: 30 | 31 | - Decorator functions 32 | - Decorator Classes 33 | - Metaclasses 34 | 35 | A **decorator class** and a **function** are different things, aren’t 36 | they? In Python this is not necessarily the case! Any Python object can 37 | behave like any other. 38 | 39 | Fans of strongly typed languages will cringe during this exercise. 40 | However, implementing a decorator that behaves like a function is a good 41 | way to understand the deeper machinery of namespaces and dynamic typing 42 | in Python. 43 | 44 | Optional Goal 1 45 | --------------- 46 | 47 | If the exercise was too easy for you, implement the decorator as a 48 | class. For this you have to deal with the term `metaclass`. 49 | 50 | Optional Goal 2 51 | --------------- 52 | 53 | The decorator should still work if it is stacked several times on top of 54 | each other: 55 | 56 | .. code:: python3 57 | 58 | @add_two 59 | @add_two 60 | def double(a): 61 | return a * 2 62 | 63 | double(19) 64 | 65 | shall result in `42`. 66 | 67 | *Translated with* `www.DeepL.com `__ 68 | -------------------------------------------------------------------------------- /challenges/monks.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/monks.jpg -------------------------------------------------------------------------------- /challenges/sorting.rst: -------------------------------------------------------------------------------- 1 | Sorting Algorithms 2 | ================== 3 | 4 | **🎯 Try several sorting algorithms.** 5 | 6 | Sorting algorithms are among the most fundamental algorithms of all. In 7 | this exercise, you will gain a basic understanding of some of these 8 | algorithms. 9 | 10 | **This is an offline exercise.** 11 | 12 | Material 13 | -------- 14 | 15 | - a deck of 16 playing cards per group (2-3 participants) 16 | - the pseudocode of the 4 algorithms 17 | - a stopwatch 18 | - paper to write down the time 19 | 20 | Instructions 21 | ------------ 22 | 23 | - read the algorithm description carefully 24 | - shuffle the deck of cards 25 | - start the stopwatch 26 | - execute one of the algorithms 27 | - stop the clock as soon as all the cards are sorted 28 | 29 | Selection Sort 30 | ~~~~~~~~~~~~~~ 31 | 32 | 1. lay the cards in a row in front of you 33 | 2. go through all the cards 34 | 3. take the card with the smallest number and put it on the target pile 35 | 4. repeat step 2 until all cards are sorted 36 | 37 | Insertion Sort 38 | ~~~~~~~~~~~~~~ 39 | 40 | 1. take a new card from the pile 41 | 2. go through the already placed cards 42 | 3. place the new card in the the correct position 43 | 4. repeat steps 1 and 2 until all cards are sorted 44 | 45 | Bubblesort 46 | ~~~~~~~~~~ 47 | 48 | 1. lay out all cards in a row in front of you 49 | 2. go through the cards from left to right 50 | 3. if there is a smaller card to the right of a larger one, swap them 51 | 4. repeat steps 2 and 3 until nothing changes in one pass 52 | 53 | Mergesort 54 | ~~~~~~~~~ 55 | 56 | 1. divide the deck of cards into two equal piles 57 | 2. sort each pile according to the mergesort method 58 | 3. turn both piles upside down so that the smallest card is visible 59 | 4. place the smaller card from one of the piles on the target pile 60 | 5. Repeat step 4 until all cards are sorted. 61 | 62 | Optional Goals 63 | -------------- 64 | 65 | - repeat the exercise with a small (8) and a large (16) deck of cards 66 | - count **comparison** and **swap** operations 67 | - implement one of the algorithms 68 | - measure the runtime with ``%timeit`` 69 | 70 | *Translated with* `www.DeepL.com `__ 71 | -------------------------------------------------------------------------------- /challenges/tennis.rst: -------------------------------------------------------------------------------- 1 | Tennis 2 | ====== 3 | 4 | **🎯 Write a class `TennisGame` that determines the score of a round of tennis.** 5 | 6 | Use the following structure: 7 | 8 | .. code:: python3 9 | 10 | class TennisGame: 11 | 12 | def __init__(self): 13 | self.score = { 14 | 'player1': 0, 15 | 'player2': 0, 16 | } 17 | 18 | def point(self, player: str) -> None: 19 | """called with 'player1' or 'player2' to record points""" 20 | self.scores[player] += 1 21 | 22 | def get_score(self) -> str: 23 | """calculates the score""" 24 | ... 25 | 26 | 27 | .. topic:: Rules 28 | 29 | 1. The first player with at least four points and two points more than the opponent wins. 30 | 2. The score is then *"Win Player 1"* or *"Win Player 2"*. 31 | 3. The current score in tennis is somewhat peculiar: points from 0 to 3 are described with *"love", "fifteen", "thirty" and "forty"*. 32 | 4. If both players have at least three points, the score with equal points is *"deuce"*. 33 | 5. If both players have at least three points, the score will be *"Advantage Player 1"* or *"Advantage Player 2"* depending on which player has more points. 34 | 35 | *Translated with* `www.DeepL.com `__ 36 | -------------------------------------------------------------------------------- /challenges/tree_array.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/challenges/tree_array.png -------------------------------------------------------------------------------- /challenges/tree_traversal.rst: -------------------------------------------------------------------------------- 1 | Tree Traversal 2 | ============== 3 | 4 | **🎯 Output all nodes in a tree.** 5 | 6 | Collect all living beings from the following tree structure in a single 7 | list. 8 | 9 | .. code:: python3 10 | 11 | tree3 = [ 12 | ['Rose', 'banana tree'], 13 | [ 14 | [ 15 | ['hammerhead shark', 'stingray'], 16 | [ 17 | ['chimp', 'human'], 18 | 'rat' 19 | ] 20 | ], 21 | ['bee', 'ant'] 22 | ] 23 | ] 24 | 25 | .. hint:: 26 | 27 | **Hints:** 28 | 29 | - Draw the tree. In which order would you go through the beings? 30 | - How can you check whether you are dealing with a string or a list? 31 | - Is it basically possible to walk trees of any depth with the same program? 32 | - Research *“Tree Traversal”* 33 | 34 | *Translated with*\ `www.DeepL.com `__ 35 | -------------------------------------------------------------------------------- /challenges/tsp.rst: -------------------------------------------------------------------------------- 1 | Traveling Salesman 2 | ================== 3 | 4 | **🎯 Implement a Solution for the Traveling Salesman Problem** 5 | 6 | A traveling salesman would like to visit N cities and cover as short a 7 | distance as possible. 8 | 9 | Write a program that *visits* all cities with the following coordinates: 10 | 11 | .. code:: python3 12 | 13 | import random 14 | 15 | N = 10 16 | random.seed(42) 17 | x = [random.randint(1, 100) for i in range(N)] 18 | y = [random.randint(1, 100) for i in range(N)] 19 | 20 | A solution could look like this: 21 | 22 | :: 23 | 24 | 7 5 2 8 6 1 0 3 9 4 25 | 26 | total distance traveled: 123.45 27 | 28 | Tasks 29 | ----- 30 | 31 | - Implement a random solution first. 32 | - Try a *brute force* solution that tries out all the options. 33 | - Why isn’t this solution always the best? 34 | - Measure the runtime for different values of *N*. 35 | - Write a *heuristic solution*. 36 | - Research the traveling salesman problem. 37 | - Research what a **NP-complete problem** is. 38 | 39 | *Translated with* `www.DeepL.com `__ 40 | -------------------------------------------------------------------------------- /classes/abc.rst: -------------------------------------------------------------------------------- 1 | Abstract Base Classes 2 | ===================== 3 | 4 | If you want to use inheritance but not allow instances of the 5 | superclass, you can use the **ABC Metaclass**. With ``ABCMeta``, you 6 | need to create a subclass that overwrites all abstract methods and 7 | properties: 8 | 9 | .. code:: python3 10 | 11 | from abc import ABCMeta, abstractmethod, abstractproperty 12 | 13 | class AbstractAnimal(metaclass=ABCMeta): 14 | 15 | @abstractmethod 16 | def make_noise(self): 17 | pass 18 | 19 | # an abstract read-only-property 20 | @abstractproperty 21 | def species(self): 22 | pass 23 | 24 | # abstract read/write property 25 | def getname(self): 26 | pass 27 | 28 | def setname(self, value): 29 | pass 30 | 31 | name = abstractproperty(getname, setname) 32 | 33 | # non-abstract method 34 | def is_alive(self): 35 | return True 36 | 37 | -------------- 38 | 39 | Exercise 40 | -------- 41 | 42 | Implement the Dog class so that the code below runs. 43 | 44 | .. code:: python3 45 | 46 | class Dog(AbstractAnimal): 47 | 48 | ... 49 | 50 | rex = Dog() 51 | rex.name = 'Rex' 52 | print(rex.is_alive()) 53 | rex.make_noise() 54 | print(rex.species()) 55 | -------------------------------------------------------------------------------- /classes/class_diagram.md: -------------------------------------------------------------------------------- 1 | 2 | # Class Diagrams 3 | 4 | One of the first and most important things converting ideas and into code is to structure data. 5 | You want to start structuring your core business data. 6 | In the case of a snake game, this means how the playing field, the snake and the food items are represented. 7 | 8 | Class diagrams are a graphical tool to structure your data and check whether it is complete and non-redundant before writing code. 9 | 10 | ## What does a class diagram contain? 11 | 12 | Here is a class diagram for a `PlayingField` class, the box in which a snake will move: 13 | 14 | ![class diagram for the PlayingField](class_playing_field.png) 15 | 16 | On top, the class diagram contains a **title**, the name of the class in `SnakeCase` notation. 17 | 18 | The second section lists the **attributes** of the class and their **data types**: 19 | 20 | * the x/y size of the playing field, a tuple of two integers 21 | * the x/y position of the food item, also a tuple of two integers 22 | 23 | The third section lists the **methods** of the class with their **arguments** and **return types**: 24 | 25 | * the `add_food()` method takes two integer arguments and returns nothing 26 | * the `add_random_food()` method has no arguments and returns nothing 27 | * the `get_walls()` method takes no arguments and returns a list of x/y integer tuples 28 | 29 | ## What the PlayingFiled does not contain 30 | 31 | It is worth pointing out that the `PlayingField` class lacks two things on purpose: 32 | 33 | First, it does not contain an attribute `snake`. 34 | To be precise, it does not know that snakes even exist. 35 | It does not have to know about it. 36 | We want the `PlayingField` and the `Snake` to manage themselves as independently as possible. 37 | This will make debugging a lot easier. 38 | 39 | Second, there is no method `draw()`. 40 | Drawing things is usually not part of your core business. 41 | In the snake game, you may want to change the user interface later (e.g. by adding graphics and sound effects). 42 | The core logic of how the snake moves should not change because of that. 43 | 44 | ---- 45 | 46 | ## Write Skeleton Code 47 | 48 | A great thing about class diagrams is that you can create them to code easily. 49 | The Python `dataclasses` module saves you a lot of typing: 50 | 51 | from dataclasses import dataclass 52 | 53 | @dataclass 54 | class PlayingField: 55 | 56 | size: (int, int) 57 | food: (int, int) = None 58 | 59 | def add_food(self, x, y): 60 | ... 61 | 62 | def add_random_food(self): 63 | ... 64 | 65 | def get_walls(self): 66 | ... 67 | 68 | This code defines the `size` and `food` attributes and annotates their data types. 69 | The `food` attribute has a default value. 70 | The class also defines the methods from the class diagram (each with the obligatory `self`). 71 | But we leave the method bodies empty for now. 72 | 73 | The `@dataclass` automatically creates the `__init__()` and `__repr__()` methods for you, so that you can set and inspect the attribute values. 74 | The code is already executable: 75 | 76 | pf = PlayingField(size=(10, 10)) 77 | print(pf) 78 | print(pf.size) 79 | print(pf.get_walls()) 80 | 81 | Although our class does nothing yet, it helps to think about your desing and write other code that depends on it. 82 | 83 | ---- 84 | 85 | ## Alternative Designs 86 | 87 | Usually, there is more than one way to design a class. 88 | Consider this alternative design for `PlayingField`: 89 | 90 | ![alternative PlayingField class](class_playing_field_alt.png) 91 | 92 | There are a few differences: 93 | 94 | * size and food have separate x and y attributes instead of being tuples 95 | * the walls are represented by a list of `(int, int)` tuples 96 | * the `add_food()` method expects a tuple instead of two integers 97 | * there methods `is_wall()` and `get_walls()` are no longer there 98 | 99 | One could discuss a lot which design is better. 100 | You are better off postponing that discussion to a cleanup stage once the code is running. 101 | The differences are very small and easy to change. 102 | In Python, one could even state that the data structures are practically *identical*. 103 | 104 | Using the `@property` decorator, you can translate attributes into each other. 105 | The following code translates the `size` attribute into two new attributes `size_x` and `size_y`: 106 | 107 | @property 108 | def size_x(self): 109 | return self.size[0] 110 | 111 | @property 112 | def size_y(self): 113 | return self.size[1] 114 | 115 | Now you can use all three attributes without storing redundant data: 116 | 117 | pf = PlayingField(size=(5, 5)) 118 | print(pf.size) 119 | print(pf.size_x) 120 | print(pf.size_y) 121 | 122 | 123 | More complex and difficult questions arise when planning relationships between multiple classes. 124 | There will be multiple working alternatives, but some may fall on your feet in the long run. 125 | You may want to read more about **SOLID principles**, **Object Composition** and **Design Patterns**. 126 | 127 | ## Classes vs SQL 128 | 129 | If you have worked with SQL, there is a striking parallel between SQL tables and classes. 130 | Tables have columns, classes have attributes. 131 | Tables have rows, classes have instances. 132 | Both structure data. 133 | Class diagrams are conceptually very close to Entity-Relationship (ER) diagrams used in the database world. 134 | 135 | ---- 136 | 137 | ## Exercise 138 | 139 | Turn the class diagram of the Snake class into skeleton code. 140 | Leave all methods empty. 141 | 142 | ![Snake class diagram](class_snake.png) 143 | 144 | ---- 145 | 146 | ## Further Reading 147 | 148 | The class diagrams in this article were designed with the online tool [Creately](https://app.creately.com). 149 | -------------------------------------------------------------------------------- /classes/class_playing_field.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/classes/class_playing_field.png -------------------------------------------------------------------------------- /classes/class_playing_field_alt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/classes/class_playing_field_alt.png -------------------------------------------------------------------------------- /classes/class_snake.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/classes/class_snake.png -------------------------------------------------------------------------------- /classes/classes.rst: -------------------------------------------------------------------------------- 1 | Classes 2 | ======= 3 | 4 | What are classes? 5 | ----------------- 6 | 7 | Classes are a tool to manage complexity in a program. They group two 8 | things in a single structural unit: **attributes (data)** and **methods 9 | (behavior)**. 10 | 11 | In more simple terms, you can use a class to stuff complexity into it, 12 | so that your main program becomes simple. In my opinion, this way of 13 | structuring code should be the main motivation to using classes in 14 | Python. 15 | 16 | In this article, you find an example how to use a class to structure 17 | your code. 18 | 19 | -------------- 20 | 21 | Defining a class 22 | ---------------- 23 | 24 | To define a class, you need to define three things: 25 | 26 | - give the class a name (in ``SnakeCase``) 27 | - define attributes (variables that belong to the class) 28 | - define methods (functions that belong to the class) 29 | 30 | In the code below, a class for a bank account is defined: 31 | 32 | .. code:: python3 33 | 34 | class Account: 35 | """ 36 | Account of a bank client. 37 | """ 38 | def __init__(self, owner, start_balance=0): 39 | self.name = owner 40 | self.balance = start_balance 41 | 42 | def deposit(self, amt): 43 | self.balance += amt 44 | 45 | def withdraw(self, amt): 46 | self.balance -= amt 47 | 48 | The class ``Account`` contains two attributes (``name`` and ``balance``) 49 | and two methods (``deposit`` and ``withdraw``). 50 | 51 | Note that you need to add the word ``self`` every time you refer to an 52 | attribute. You also must use ``self`` as the first parameter in every 53 | method of a class. 54 | 55 | -------------- 56 | 57 | Creating Objects 58 | ---------------- 59 | 60 | To use a class, you need to create an object from it first. Objects are 61 | *“live versions”* of a class, the class being an idealized abstration 62 | (in the sense of `Platos Theory of Forms `__). 63 | If you think of **BankAccount** as a class, the actual accounts of **Ada Lovelace** and **Mahatma Gandi** 64 | would be the objects of that class. 65 | 66 | You can create multiple objects from a class, and each objects has its 67 | own, independent attributes (e.g. if **BankAccount** has an attribute **balance**, 68 | then **Ada** and **Mahatma** could have a different amount of money). 69 | 70 | Syntactically, you can think of a class as a function that returns 71 | objects. (This is a gross oversimplification to what textbooks on 72 | classes say, but in Python it is more or less what happens). 73 | 74 | To create ``Account`` objects, you need to call the class. Creating an 75 | object will automatically call the constructor ``__init__(self)`` with 76 | the parameters supplied. 77 | 78 | .. code:: python3 79 | 80 | a = Account('Ada Lovelace', 1234) 81 | m = Account('Mahatma Gandhi', 10) 82 | 83 | Then you can access the attributes like any variable using the dot (``.``) syntax: 84 | 85 | .. code:: python3 86 | 87 | print(a.name) 88 | print(m.balance) 89 | 90 | And you can call methods in a similar way: 91 | 92 | .. code:: python3 93 | 94 | a.deposit(100) 95 | a.withdraw(10) 96 | print(a.balance) 97 | 98 | Note that these methods modify the state of *Adas* account object, but 99 | not *Mahatmas*. 100 | 101 | -------------- 102 | 103 | Making classes printable 104 | ------------------------ 105 | 106 | One disadvantage of classes is that when you print an object, you will 107 | see something like this: 108 | 109 | :: 110 | 111 | <__main__.Account at 0x7f64519d8438> 112 | 113 | A good workaround is to add a special method, ``__repr__(self)`` to the 114 | class that returns a string. This method will be called every time a 115 | string representation is needed: when printing and object, when an 116 | object appears inside a list or in error messages. 117 | 118 | Typically, you would build a short string in ``__repr__(self)`` that 119 | describes the object: 120 | 121 | .. code:: python3 122 | 123 | def __repr__(self): 124 | return f"" 125 | 126 | With this method defined, the instruction 127 | 128 | .. code:: python3 129 | 130 | print(a) 131 | 132 | would result in the output 133 | 134 | :: 135 | 136 | " 137 | 138 | It is a good idea to implement ``__repr__(self)`` as the first method in 139 | a new class. 140 | 141 | -------------- 142 | 143 | Caveats 144 | ------- 145 | 146 | In other programming languages classes are often advertised for 147 | *“modeling real-world objects or logical entities”*. This is partially 148 | true in Python. Note that Python offers a lot of alternatives to using 149 | classes, e.g. dictionaries, named tuples or DataFrames may often serve 150 | the same purpose equally well. 151 | 152 | Another motivation for using classes you find in textbooks is 153 | **encapsulation**, isolating parts of your program from the rest. 154 | Encapsulation does not exist in Python (e.g. you cannot declare parts of 155 | a class as ``private`` in a way that cannot be circumvented). If you 156 | depend on your code being strictly isolated from other parts (e.g. in a 157 | security-critical application or when organizing a very large program), 158 | **consider other programming languages than Python.** 159 | 160 | -------------- 161 | 162 | .. topic:: Dirty Tricks 163 | 164 | Python allows using classes in multiple creative ways. 165 | I call them **dirty tricks**. Most of them have their uses in 166 | larger programming libraries. But if you are writing a smaller program, 167 | they probably do more harm than good. 168 | 169 | Especially if you are still learning about classes, consider yourself 170 | warned of the following tricks: 171 | 172 | - Multiple Inheritance 173 | - Operator Overloading 174 | - Metaclasses 175 | - Monkey Patching 176 | 177 | These dirty tricks are likely to mess up your program. Do not use any of 178 | them unless you really know what you are doing! 179 | -------------------------------------------------------------------------------- /classes/composition.rst: -------------------------------------------------------------------------------- 1 | Object Composition 2 | ================== 3 | 4 | Objects can be attributes of other objects. 5 | 6 | You can use this mechanism to represent relationships between objects. 7 | For instance you could model one-to-one, one-to-many or many-to-many 8 | relationships. This has some similarity to database design, but with 9 | classes you have more options. 10 | 11 | Object Composition is something you use in Python all the time, 12 | e.g. when you have a list of integers and refer to the first item of the 13 | list, you are accessing an ``int`` object inside a ``list`` object. 14 | 15 | Modeling classes using Object Composition is one of the most important 16 | techniques in Object-Oriented-Programming. Look up the terms 17 | **Object-Oriented-Design** and **Design Patterns** for further reading. 18 | 19 | Here are two examples: 20 | 21 | -------------- 22 | 23 | Example 1: Bank 24 | --------------- 25 | 26 | Imagine you want to have a ``Bank`` class that represents many accounts. 27 | 28 | A very bad design solution would be: 29 | 30 | :: 31 | 32 | Bank is a subclass of Account 33 | 34 | This violates **Liskovs Substitution Principle**. A bank **is not** a 35 | special type of account! 36 | 37 | A better design solution would be: 38 | 39 | :: 40 | 41 | A Bank contains many Accounts 42 | 43 | An implementation could look like this: 44 | 45 | .. code:: python3 46 | 47 | class Bank: 48 | 49 | def __init__(self): 50 | self.accounts = {} 51 | 52 | def add_account(self, number, accounts): 53 | self.accounts[number] = account 54 | 55 | 56 | barclays = Bank() 57 | ada = Account('Ada Lovelace', 100) 58 | barclays.add_account(1234, ada) 59 | barclays.add_account(5555, Account('Bob', 255)) 60 | 61 | -------------- 62 | 63 | Example 2: Composite Pattern 64 | ---------------------------- 65 | 66 | In this exaple, ``Node`` objects contain other ``Node`` objects and 67 | ``Leaf`` objects to construct a tree. The ``traverse()`` method walks 68 | across all nodes in the tree. 69 | 70 | .. code:: python3 71 | 72 | class Node: 73 | 74 | def __init__(self, *args): 75 | self.subnodes = args 76 | 77 | def traverse(self): 78 | result = [] 79 | for n in self.subnodes: 80 | result += n.traverse() 81 | return result 82 | 83 | 84 | class Leaf: 85 | 86 | def __init__(self, name): 87 | self.value = name 88 | 89 | def traverse(self): 90 | return [self.value] 91 | 92 | 93 | my_tree = Node( 94 | Node( 95 | Node( 96 | Leaf('gorilla'), 97 | Node( 98 | Leaf('chimp'), Leaf('human') 99 | ) 100 | ), 101 | Node( 102 | Leaf('cat'),Leaf('dog') 103 | ), 104 | ), 105 | Node( 106 | Node( 107 | Leaf('chicken'), 108 | Node( 109 | Leaf('lizard'),Leaf('turtle') 110 | ), 111 | ), 112 | Node( 113 | Leaf('shark') 114 | ), 115 | ) 116 | ) 117 | 118 | print(my_tree.traverse()) 119 | -------------------------------------------------------------------------------- /classes/decorator_class.rst: -------------------------------------------------------------------------------- 1 | Decorator Classes 2 | ================= 3 | 4 | Decorator Classes are a structure to write more sophisticated decorators 5 | that have an internal state. They allow you to pass arguments into the 6 | decorator as well. 7 | 8 | Key points: 9 | ----------- 10 | 11 | - the constructor ``__init__()`` receives the arguments given in the line starting with ``@`` 12 | - the ``__call__()`` method should return the decorator function 13 | - the ``safe_call()`` method gets called in the end (you can give it a different name) 14 | 15 | -------------- 16 | 17 | Code Example 18 | ------------ 19 | 20 | Here is a code example that counts how many errors have been produced. 21 | 22 | .. code:: python3 23 | 24 | class FailureCounter: 25 | 26 | def __init__(self, message): 27 | self.message = message 28 | self.function = None 29 | self.failcount = 0 30 | 31 | def __call__(self, func): 32 | self.function = func 33 | return self.safe_call 34 | 35 | def safe_call(self, *args): 36 | try: 37 | self.function(*args) 38 | except IOError: 39 | self.failcount += 1 40 | print(self.message) 41 | print(f'An I/O error was caught in {self.function.__name__}') 42 | print(f"with the file name '{args[0]}'") 43 | print(f'this is failure #{self.failcount}\n') 44 | 45 | @FailureCounter('--- FILE ERROR ---') 46 | def risky_fileopen(filename): 47 | open(filename) 48 | 49 | risky_fileopen('not_existing_file') 50 | risky_fileopen('doesnotexist_either') 51 | -------------------------------------------------------------------------------- /classes/inheritance.rst: -------------------------------------------------------------------------------- 1 | Inheritance 2 | =========== 3 | 4 | Classes can extend other classes. A **subclass** will have all the 5 | attributes and methods of a **superclass**. You can say the subclass 6 | inherits from the superclass. 7 | 8 | A subclass can define new methods and attributes. It can also replace 9 | methods of the superclass. 10 | 11 | The most important design principle when using inheritance is **Liskovs 12 | Substitution Principle**. It says that you should be able to use a 13 | subclass whereever the superclass is used. 14 | 15 | -------------- 16 | 17 | Example 18 | ------- 19 | 20 | Here you find an example using the ``Account`` class defined earlier. 21 | 22 | .. code:: python3 23 | 24 | class SavingsAccount(Account): 25 | 26 | def __init__(self, owner, start_balance=0, interest_rate=1.0): 27 | super().__init__(owner, start_balance) 28 | self.interest_rate = interest_rate 29 | 30 | def add_interest(self): 31 | self.balance *= self.interest_rate 32 | 33 | def withdraw(self, amount): # replaces Account.withdraw 34 | print('**extra identification approved**') 35 | super().withdraw(amount) 36 | 37 | The ``super()`` function returns an instance of the superclass, so that 38 | you can call the inherited methods, even if you are replacing them. 39 | 40 | Using the subclass is very similar to using the superclass: 41 | 42 | .. code:: python3 43 | 44 | b = SavingsAccount('Betty', 100, interest_rate=1.03) 45 | print(b) 46 | 47 | b.deposit(100) # calls Account.deposit() 48 | print(b) # calls Account.__repr__() 49 | 50 | b.add_interest() # calls SavingsAccount.add_interest() 51 | print(b) 52 | 53 | -------------- 54 | 55 | Caveats 56 | ------- 57 | 58 | Once you get the hang of object-oriented programming, inheritance is not 59 | that difficult. At some point, it is very tempting to define lots of 60 | subclasses, and class hierarchies with multiple levels. Most of the 61 | time, having one level of inheritance is enough. Many times you do not 62 | need inheritance at all, and you are better off using **object composition**. 63 | 64 | Python allows to use **multiple inheritance** (i.e. a subclass with more 65 | than one superclass). Avoid writing your own multiple inheritance 66 | hierarchy unless you know exactly what you are doing. 67 | -------------------------------------------------------------------------------- /classes/metaclasses.rst: -------------------------------------------------------------------------------- 1 | Metaclasses 2 | =========== 3 | 4 | Metaclasses change the objects creation mechanics in Python. 5 | 6 | The when Python creates an object, it calls the ``__new__()`` method of 7 | the metaclass. The default behavior of ``__new__()`` is that it creates 8 | an instance and calls its ``__init__()``. Now you could write a new 9 | metaclass that instead of calling ``__init__()`` does something else. 10 | 11 | Valid Use Cases for changing a metaclass are: 12 | 13 | - writing an ORM like **Django models** or **SQLAlchemy** 14 | - hijacking internal Python logic (e.g. like **pytest** does) 15 | - emulating JavaScript-like objects (the Prototype pattern) 16 | 17 | Throughout 20 years of Python programming, I have not come across a 18 | single situation where writing a metaclass was necessary. But it helps 19 | to understand Python on a deeper level. 20 | 21 | -------------- 22 | 23 | Example 24 | ------- 25 | 26 | **WARNING: This code is a complex illustrative example that might drive 27 | you nuts!** 28 | 29 | 1. run the code 30 | 2. admire what is happening 31 | 3. try to understand what is happening 32 | 4. return to 1 33 | 34 | Here is the code 35 | 36 | .. code:: python3 37 | 38 | class CrazyMonkeyPack(type): 39 | 40 | def __new__(mcs, name, bases, dict): 41 | cls = type.__new__(mcs, name, bases, dict) 42 | 43 | def wrapper(*args): 44 | instance = [] 45 | for i in range(1, args[0]+1): 46 | monkey = cls(f'monkey #{i}') # calls __init__ 47 | monkey.state = 'crazy' # monkey-patches the state attribute 48 | instance.append(monkey) 49 | return instance 50 | 51 | return wrapper 52 | 53 | 54 | class CrazyMonkeys(metaclass=CrazyMonkeyPack): 55 | """A self-expanding horde of monkeys""" 56 | def __init__(self, name): 57 | self.name = name 58 | 59 | def __repr__(self): 60 | return f"<{self.name} ({self.state})>" 61 | 62 | 63 | monkeys = CrazyMonkeys(3) # calls __new__ 64 | print(monkeys) # see what happens! 65 | 66 | -------------- 67 | 68 | Final Warning 69 | ------------- 70 | 71 | Don't try using metaclasses at work, unless 72 | 73 | - you have excluded all alternatives 74 | - you really know what you are doing 75 | - you have talked to a developer more experienced than you 76 | -------------------------------------------------------------------------------- /classes/oop_simple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/classes/oop_simple.png -------------------------------------------------------------------------------- /classes/operator_overloading.rst: -------------------------------------------------------------------------------- 1 | Operator Overloading 2 | ==================== 3 | 4 | In Python, classes can have **many** ``__magic_methods__()`` – those 5 | that start and end with a double underscore. All ``__magic_methods__()`` 6 | have a special function and should never be called directly. Most of 7 | them directly map to Python operators. Here are some examples: 8 | 9 | ============== ================================================= 10 | method description 11 | ============== ================================================= 12 | ``__init__()`` called when creating an object from a class 13 | ``__repr__()`` called when converting object to a string 14 | ``__add__()`` called when using the ``+`` operator on an object 15 | ``__mul__()`` called when using the ``*`` operator on an object 16 | ``__gt__()`` called when comparing the object to another 17 | ``__len__()`` called when using ``len()`` 18 | ``__hash__()`` called when using the object as a dictionary key 19 | ============== ================================================= 20 | 21 | It is often questionable, whether overloading operators is a good idea. 22 | Many times it is not, because it obscures what is happening behind the 23 | scenes. 24 | 25 | Compare the clarity of: 26 | 27 | .. code:: python3 28 | 29 | a = vec1 * vec2 30 | b = vec1 @ vec2 31 | 32 | versus 33 | 34 | .. code:: python3 35 | 36 | a = vec1.dot_product(vec2) 37 | b = vec1.scalar_product(vec2) 38 | 39 | Below you find two examples that I have found useful: 40 | 41 | -------------- 42 | 43 | Sortable Objects 44 | ---------------- 45 | 46 | The method ``__gt__(self, other)`` is implicitly called when comparing 47 | two objects, as done by any sorting algorithm. 48 | 49 | .. code:: python3 50 | 51 | class Elephant: 52 | """Elephants that sort themselves by trunk size""" 53 | def __init__(self, name, trunk_size): 54 | self.name = name 55 | self.trunk_size = trunk_size 56 | 57 | def __repr__(self): 58 | return f"<{self.name} [trunk {self.trunk_size}]>" 59 | 60 | def __gt__(self, other): 61 | return self.trunk_size < other.trunk_size 62 | 63 | elephants = [ 64 | Elephant('mama', 5), 65 | Elephant('baby', 1), 66 | Elephant('grandma', 7), 67 | Elephant('daddy', 6), 68 | Elephant('son', 3), 69 | ] 70 | 71 | from pprint import pprint 72 | 73 | pprint(elephants) 74 | 75 | print("\nAnd now the biggest elephants go first:") 76 | elephants.sort() 77 | 78 | pprint(elephants) 79 | 80 | -------------- 81 | 82 | Hashable Vectors 83 | ---------------- 84 | 85 | This is a 2D vector class I used for screen positions in a game. I 86 | wanted the vectors to be capable of simple arithmetics. They also needed 87 | to be hashable (which NumPy arrays are not). 88 | 89 | .. code:: python3 90 | 91 | class Vector: 92 | 93 | def __init__(self, x, y): 94 | self.x = x 95 | self.y = y 96 | 97 | def __add__(self, other): 98 | x = self.x + other.x 99 | y = self.y + other.y 100 | return Vector(x, y) 101 | 102 | def __sub__(self, other): 103 | x = self.x - other.x 104 | y = self.y - other.y 105 | return Vector(x, y) 106 | 107 | def __mul__(self, n): 108 | '''scalar multiplication''' 109 | # considered unclean - for illustration only 110 | x = self.x * n 111 | y = self.y * n 112 | return Vector(x, y) 113 | 114 | def __hash__(self): 115 | return str(self).__hash__() 116 | 117 | def __repr__(self): 118 | return f'[{self.x};{self.y}]' 119 | 120 | 121 | UP = Vector(0, -1) 122 | LEFT = Vector(-1, 0) 123 | UPLEFT = UP + LEFT 124 | FAST_UP = UP * 3 125 | 126 | messages = { 127 | UP: 'moving up', 128 | LEFT: 'moving left', 129 | } 130 | -------------------------------------------------------------------------------- /concurrency/README.rst: -------------------------------------------------------------------------------- 1 | Concurrency 2 | =========== 3 | 4 | *Concurrent programming* means that you have two or more sub-programs 5 | running simultaneously. This potentially allows you to use all your 6 | processors at once. This sounds like an enticing idea, but there are 7 | good reasons to be very cautious with it. 8 | 9 | ---- 10 | 11 | Pros and Cons of Concurrency 12 | ---------------------------- 13 | 14 | Often concurrency is a bad idea. The devil is lurking in the details: 15 | 16 | - Coordinating parallel sub-programs is very difficult to debug (look up the words *“race condition”* and *“heisenbug”*). 17 | - Python has a strange thing called the **GIL (Global Interpreter Lock)**. That means, Python can really only execute one command at a time. 18 | - There are great existing solutions for many typical applications (web scraping, web servers). 19 | 20 | On the other hand, concurrency can be a good idea: 21 | 22 | - if your tasks are waiting for some I/O anyway, the speed of Python does not matter. 23 | - starting multiple separate Python processes is rather easy (with the `multiprocessing` module). 24 | - if you are looking for a challenge. 25 | 26 | There are three noteworthy approaches to concurrency in Python: 27 | **threads**, **coroutines** and **multiple processes**. 28 | 29 | ---- 30 | 31 | Multithreading 32 | -------------- 33 | 34 | This is the old way to implement parallel execution. 35 | It has its flaws but you can grasp the basic idea: 36 | 37 | .. literalinclude:: thread_factorial.py 38 | 39 | ---- 40 | 41 | Async Coroutines 42 | ---------------- 43 | 44 | The `async` interface has been added to Python more recently. 45 | It fixes many problems of threads. 46 | 47 | .. literalinclude:: async_factorial.py 48 | 49 | ---- 50 | 51 | Subprocesses 52 | ------------ 53 | 54 | The `subprocess` module allows you to launch extra processes through the operating system. 55 | Subprocesses are not restricted to Python programs. 56 | This is the most flexible approach, but also has the highest overhead. 57 | 58 | .. literalinclude:: subprocess_factorial.py 59 | 60 | .. literalinclude:: factorial.py 61 | 62 | ---- 63 | 64 | Challenge: Gaussian Elimination 65 | ------------------------------- 66 | 67 | In :download:`gauss_elim.py` you find an implementation of the `Gauss Elimination Algorithm `__ to solve linear equation systems. 68 | The algorithm has a **cubic time complexity**. 69 | 70 | Parallelize the execution of the algorithm and check whether it gets any faster. 71 | 72 | In :download:`test_gauss_elim.py` you find unit tests for the module. 73 | 74 | .. note:: 75 | 76 | The linear equation solver is written in plain Python. 77 | Of course, Numpy would also speed up the execution considerably. 78 | -------------------------------------------------------------------------------- /concurrency/async_factorial.py: -------------------------------------------------------------------------------- 1 | """ 2 | Example of parallel execution with asyncio 3 | 4 | see: 5 | https://docs.python.org/3/library/asyncio-task.html 6 | """ 7 | import asyncio 8 | import random 9 | from functools import reduce 10 | 11 | 12 | def multiply(a, b): 13 | return a * b 14 | 15 | 16 | async def factorial(number): 17 | """delayed calculation of factorial numbers""" 18 | result = reduce(multiply, range(1, number + 1), 1) 19 | delay = random.randint(5, 20) 20 | await asyncio.sleep(delay) 21 | print(f"after {delay:2} seconds: {number}! = {result}") 22 | 23 | 24 | async def main(): 25 | # create concurrent tasks 26 | tasks = [] 27 | for i in range(10): 28 | tasks.append(asyncio.create_task(factorial(i))) 29 | 30 | # wait for tasks to finish 31 | # (all have to be awaited) 32 | for t in tasks: 33 | await t 34 | 35 | 36 | # run the toplevel async function 37 | asyncio.run(main()) 38 | -------------------------------------------------------------------------------- /concurrency/factorial.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import time 3 | import random 4 | 5 | n = int(sys.argv[1]) 6 | result = 1 7 | 8 | while n > 0: 9 | result *= n 10 | n -= 1 11 | 12 | delay = random.randint(5, 15) 13 | time.sleep(delay) 14 | print(f"factorial of {sys.argv[1]} = {result} after {delay} sec") 15 | -------------------------------------------------------------------------------- /concurrency/gauss_elim.py: -------------------------------------------------------------------------------- 1 | 2 | from copy import deepcopy 3 | import numpy as np 4 | 5 | def solve_linear(data: list[float]) -> list[float]: 6 | """solves linear equations with the Gauss Elimination method""" 7 | nrows = len(data) 8 | ncols = len(data[0]) 9 | M = deepcopy(data) 10 | for i in range(ncols - 1): 11 | # normalize each row by position i 12 | for row in M[i:]: 13 | first = row[i] 14 | for j in range(i, ncols): 15 | row[j] = row[j] / first 16 | # subtract equation i from all 17 | for j in range(i + 1, nrows): 18 | for k in range(i, ncols): 19 | M[j][k] -= M[i][k] 20 | 21 | # retrieve coefficients from triangular form 22 | coef = [0.0] * nrows 23 | for i in range(nrows - 1, -1, -1): # process rows in reverse order 24 | temp = 0.0 25 | for j in range(i + 1, ncols - 1): 26 | temp -= M[i][j] * coef[j] 27 | temp += M[i][-1] 28 | coef[i] = temp 29 | 30 | return coef 31 | 32 | 33 | def create_linear_equation_sys(size): 34 | """creates a solvable linear equation system""" 35 | A = np.random.rand(size, size) 36 | # Ensure matrix A is invertible by checking the determinant 37 | while np.linalg.det(A) == 0: 38 | A = np.random.rand(size, size) 39 | 40 | solution = np.random.randint(-100, 100, size=size) # random solution vector 41 | 42 | b = np.dot(A, solution) # right-hand-side vector 43 | 44 | mtx = np.hstack([A, b.reshape(-1, 1)]) 45 | return mtx 46 | 47 | def round_output(v, digits=3): 48 | return [round(x, digits) for x in v] 49 | 50 | if __name__ == '__main__': 51 | # solve subsequent tasks (slow) 52 | tasks = [create_linear_equation_sys(200) for _ in range(10)] 53 | for i, M in enumerate(tasks, 1): 54 | print(f"\ntask {i}") 55 | result = round_output(solve_linear(M)) 56 | print("done") 57 | -------------------------------------------------------------------------------- /concurrency/subprocess_factorial.py: -------------------------------------------------------------------------------- 1 | """ 2 | Launch processes with the subprocess module 3 | 4 | https://docs.python.org/3/library/subprocess.html 5 | """ 6 | 7 | import subprocess 8 | 9 | # launch a single external process 10 | # r = subprocess.run(["python", "factorial.py", str(5)]) 11 | # try some other command than Python 12 | 13 | 14 | procs = [] 15 | for i in range(10): 16 | cmd = ["python", "factorial.py", str(i)] 17 | p = subprocess.Popen(cmd) # , stdout=subprocess.PIPE) 18 | # add stdout argument to see results immediately 19 | procs.append(p) 20 | 21 | for p in procs: 22 | p.wait() 23 | # read output from pipe 24 | #print(p.stdout.read().encode()) 25 | -------------------------------------------------------------------------------- /concurrency/test_gauss_elim.py: -------------------------------------------------------------------------------- 1 | 2 | import pytest 3 | 4 | from gauss_elim import solve_linear 5 | 6 | 7 | EXAMPLES = [ 8 | ( 9 | [ 10 | [2, 3, 3, 3], 11 | [3, -2, -9, 4], 12 | [5, -1, -18, -1], 13 | ], 14 | [3, -2, 1]), 15 | ( 16 | [ 17 | [1, 2, 3], 18 | [4, 1, 5], 19 | ], 20 | [1, 1], 21 | ), 22 | ( 23 | [ 24 | [1, 2, 3], 25 | [1, 2, 3], 26 | ], 27 | [1, 1], 28 | ), 29 | ] 30 | 31 | @pytest.mark.parametrize("matrix,expected", EXAMPLES) 32 | def test_solve(matrix, expected): 33 | result = solve_linear(matrix) 34 | result = [round(x, 3) for x in result] 35 | assert result == expected 36 | -------------------------------------------------------------------------------- /concurrency/thread_factorial.py: -------------------------------------------------------------------------------- 1 | """ 2 | Factorial with threads 3 | # adopted from 4 | http://www.devshed.com/c/a/Python/Basic-Threading-in-Python/1/ 5 | """ 6 | 7 | import threading 8 | import time 9 | import random 10 | 11 | 12 | class FactorialThread(threading.Thread): 13 | def __init__(self, number): 14 | super().__init__() 15 | self.number = number 16 | 17 | @staticmethod 18 | def factorial(n): 19 | return ( 20 | 1 if n == 0 21 | else n * FactorialThread.factorial(n - 1) 22 | ) 23 | 24 | def run(self): 25 | result = self.factorial(self.number) 26 | time.sleep(random.randint(5, 20)) 27 | print(f"{self.number}! = {result}") 28 | 29 | 30 | for number in range(10): 31 | FactorialThread(number).start() 32 | -------------------------------------------------------------------------------- /conf.py: -------------------------------------------------------------------------------- 1 | # Configuration file for the Sphinx documentation builder. 2 | # 3 | # For the full list of built-in configuration values, see the documentation: 4 | # https://www.sphinx-doc.org/en/master/usage/configuration.html 5 | 6 | # -- Project information ----------------------------------------------------- 7 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information 8 | 9 | project = 'Advanced Python' 10 | copyright = '2024, Kristian Rother' 11 | author = 'Kristian Rother' 12 | release = '1.0' 13 | html_title = f"{project}" 14 | 15 | # -- General configuration --------------------------------------------------- 16 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration 17 | 18 | extensions = [ 19 | 'sphinx_design', 20 | 'sphinx_copybutton', 21 | 'sphinx.ext.todo', 22 | 'myst_parser', 23 | ] 24 | 25 | templates_path = ['_templates'] 26 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'README.md'] 27 | 28 | language = 'en' 29 | 30 | # -- Options for HTML output ------------------------------------------------- 31 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output 32 | 33 | html_theme = 'furo' 34 | html_static_path = ['_static'] 35 | html_logo = "_static/academis_logo.png" 36 | html_favicon = "_static/favicon.ico" 37 | 38 | html_css_files = [ 39 | "academis.css", 40 | ] 41 | html_theme_options = { 42 | "source_repository": "https://github.com/krother/advanced_python", 43 | "source_branch": "master", 44 | "source_directory": "", 45 | } 46 | -------------------------------------------------------------------------------- /error_handling/debugging.rst: -------------------------------------------------------------------------------- 1 | 2 | Debugging 3 | ========= 4 | 5 | Debugging your code is a skill of its own. 6 | In this chapter, you find a list of debugging tools and techniques that you might want to try. 7 | 8 | Challenge: Maze Generator 9 | ------------------------- 10 | 11 | There should be more levels in the dungeon levels. 12 | We could use a random algorithm to generate levels. 13 | They might look like this: 14 | 15 | :: 16 | 17 | ########## 18 | #.#...#..# 19 | #........# 20 | #..##.##.# 21 | #.#..#...# 22 | #..#.....# 23 | ##.##.##.# 24 | #........# 25 | #...#....# 26 | ########## 27 | 28 | In :download:`generate_maze_buggy.py` you find a basic implementation of the algorithm. 29 | However, it is **very buggy**. There are about 20 bugs in the program. 30 | 31 | Types of Bugs 32 | ------------- 33 | 34 | In Python, you can distinguish multiple types of bugs: 35 | 36 | 1. SyntaxErrors 37 | +++++++++++++++ 38 | 39 | When there is a bug in the Python Syntax or indentation, Python will refuse to execute any code. 40 | From the error message you know that there is a problem and roughly where it is. 41 | 42 | Most of the time, syntax issues are fairly easy to fix. Your editor should highlight them right away. 43 | 44 | 2. Runtime Exceptions 45 | +++++++++++++++++++++ 46 | 47 | When Python executes a part of the code but then crashes, you have an Exception at runtime. 48 | The ``NameError``, ``TypeError``, ``ValueError`` and many others fall in this category. 49 | The error message will give you some hints what to look for, but the source of the error might be somewhere else. 50 | In any case you know there is a problem, and Python knows it too. 51 | 52 | 3. Semantic errors 53 | ++++++++++++++++++ 54 | 55 | If Python executes the code without error, but does not deliver the expected result, 56 | you can call this a **Semantic Error**. 57 | You still know that there is a problem, but Python doesn't. 58 | Semantic errors are harder to debug. 59 | 60 | 4. Complex issues 61 | +++++++++++++++++ 62 | 63 | More complex bugs are: race conditions (timing issues with multiple threads), 64 | border cases (bugs that only occur with exotic input), and Heisenbugs (bugs that disappear when you start debugging them). 65 | These are tough, and I won't cover them specifically here. 66 | 67 | 5. Unknown Bugs 68 | +++++++++++++++ 69 | 70 | Finally, there might be bugs in the program that nobody knows about. 71 | This is of course bad, and we need to keep that possibility in the back of our heads. 72 | 73 | Debugging Techniques 74 | -------------------- 75 | 76 | * read the code 77 | * read the error message (message on bottom, line numbers, type of error on top) 78 | * inspect variables with `print(x)` 79 | * inspect the type of variables with `print(type(x))` 80 | * reproduce the bug 81 | * use minimal input data 82 | * use minimal number of iterations 83 | * isolate the bug by commenting parts of the program 84 | * drop assertions in your code 85 | * write more tests 86 | * explain the problem to someone else 87 | * step through the code in an interactive debugger 88 | * clean up your code 89 | * run a code cleanup tool (``black``) 90 | * run a type checker (``mypy``) 91 | * run a linter (``pylint``) 92 | * take a break 93 | * sleep over it 94 | * ask for a code review 95 | * write down what the problem is 96 | * draw a formal description of your program logic (flowchart, state diagram 97 | * draw a formal description of your data structure (class diagram, ER-diagram) 98 | * background reading on the library / algorithm you are implementing 99 | * google the error message 100 | 101 | 102 | .. seealso:: 103 | 104 | - `Debugging Tutorial `__ 105 | - `Kristians Debugging Tutorial Video `__ 106 | -------------------------------------------------------------------------------- /error_handling/exceptions.rst: -------------------------------------------------------------------------------- 1 | Handling Exceptions 2 | =================== 3 | 4 | Raising Exceptions 5 | ------------------ 6 | 7 | You can create errors with your own messages that stop the program: 8 | 9 | .. code:: python3 10 | 11 | raise ValueError("expected a valid Pokemon name") 12 | 13 | ---- 14 | 15 | Catching Exceptions 16 | ------------------- 17 | 18 | The ``try.. except`` clause allows your program to catch errors and act 19 | on them instead of terminating the program. 20 | 21 | .. code:: python3 22 | 23 | cards = "234567890JQKA" 24 | 25 | for card in cards: 26 | try: 27 | number = int(card) 28 | print(f"dealt {card}") 29 | except ValueError: 30 | print(f"{card} could not be dealt") 31 | 32 | ---- 33 | 34 | How not to catch Exceptions 35 | --------------------------- 36 | 37 | Exceptions should always be caught with an explicit error type. 38 | Using a generic ``except`` makes debugging difficult: 39 | 40 | .. code:: python3 41 | 42 | cards = "234567890JQKA" 43 | 44 | for card in cards: 45 | try: 46 | number = int(card) 47 | print(f"dealt {car}") 48 | except: 49 | print(f"{card} could not be dealt") 50 | 51 | In this example, ``car`` causes a ``NameError`` for every card. You 52 | don’t get any clues what exactly went wrong. 53 | 54 | This is called the *“diaper pattern”* and considered a very bad habit. 55 | 56 | What Exceptions to catch 57 | ------------------------ 58 | 59 | - File operations 60 | - web operations 61 | - big function calls 62 | - database operations 63 | - NEVER CATCH everything 64 | 65 | ---- 66 | 67 | Creating your own Exceptions 68 | ---------------------------- 69 | 70 | You can define your own types of Exceptions: 71 | 72 | .. code:: python3 73 | 74 | class WrongInputError(Exception): pass 75 | 76 | and raise them: 77 | 78 | .. code:: python3 79 | 80 | text = input("please enter a number between 1-4: ") 81 | if not text in "1234": 82 | raise WrongInputError("{} is not a number between 1-4.".format(text)) 83 | 84 | The ``try..except`` works for your own Exception types as well. 85 | 86 | ---- 87 | 88 | Detailed traceback 89 | ------------------ 90 | 91 | Sometimes you may want to output very detailed information in an ``except`` block: 92 | 93 | .. literalinclude:: get_traceback.py 94 | -------------------------------------------------------------------------------- /error_handling/generate_maze_buggy.py: -------------------------------------------------------------------------------- 1 | # Maze Generator 2 | # 3 | # generates a random maze as a string 4 | # with '#' being walls and '.' being floors 5 | # 6 | # BUGGY CODE! 7 | # This code is full of bugs. 8 | # Try to fix them all. 9 | 10 | import random 11 | 12 | XMAX, YMAX = 12, 7 13 | 14 | 15 | def create_grid_string(floors: set[tuple[int, int]], xsize: int, ysize: int) -> str: 16 | """Creates a grid of size (xsize, ysize) from the given positions of floors.""" 17 | for y in range(ysize): 18 | grid = "" 19 | for x in range(xsize): 20 | grid = "#" if (xsize, ysize) in floors else "." 21 | grid == "\n" 22 | return grid 23 | 24 | 25 | def get_all_floor_positions(xsize: int, ysize: int) 26 | """Returns a list of (x, y) tuples covering all positions in a grid.""" 27 | return [(x, y) for x in range(0, xsize) for y in range(1, ysize - 1)] 28 | 29 | 30 | def get_neighbors(x: int, y: int) -> list[tuple(int, int)]: 31 | """Returns a list with the 8 neighbor positions of (x, y).""" 32 | return [ 33 | (x, - 1), (y, x + 1), (x - (1), y), (x + 1), y, 34 | (x, (-1, y)), (x + 1, y, 1), (x - 1, y + 1, x + 1, y + 1) 35 | ] 36 | 37 | 38 | def generate_floor_positions(xsize: int, ysize:int) -> set[tuple[int, int]]: 39 | """ 40 | Creates positions of floors for a random maze. 41 | 42 | 1. Pick a random location in the maze. 43 | 2. Count how many of its neighbors are already floors. 44 | 3. If there are 4 or less, make the position a floor otherwise it is a wall. 45 | 4. Continue with step 1 until every location has been visited once. 46 | """ 47 | positions = get_all_floor_positions(xsize, ysize) 48 | floors = set() 49 | while positions != []: 50 | x, y = random.choice(positions) 51 | neighbors = get_neighbors(x, y) 52 | free = [for nb in neighbors if nb in floors] 53 | if len(free) > 5: 54 | floors.add((x, y)) 55 | positions.remove((x, y)) 56 | return floors 57 | 58 | 59 | def create_maze(xsize: int, ysize: int): 60 | """Returns a xsize x ysize maze as a string.""" 61 | floors = generate_floor_position(xsize, ysize) 62 | maze = create_grid_string(floors, xsize, ysize) 63 | 64 | 65 | if __name__ == '__main__': 66 | maze = create_maze 67 | print(maze) -------------------------------------------------------------------------------- /error_handling/get_traceback.py: -------------------------------------------------------------------------------- 1 | """ 2 | Example: 3 | Print the state of all variables when an Exception occurs. 4 | """ 5 | 6 | import sys, traceback 7 | 8 | def print_exc(): 9 | tb = sys.exc_info()[2] 10 | while tb.tb_next: 11 | # jump through execution frames 12 | tb = tb.tb_next 13 | frame = tb.tb_frame 14 | code = frame.f_code 15 | print("Frame %s in %s at line %s"%\ 16 | (code.co_name, 17 | code.co_filename, 18 | frame.f_lineno)) 19 | # print local variables 20 | for k, v in frame.f_locals.items(): 21 | print("%20s = %s"%(k,str(v))) 22 | 23 | 24 | try: 25 | a = 3 26 | b = "Carrots" 27 | c = a / 0 28 | except ZeroDivisionError: 29 | print_exc() 30 | -------------------------------------------------------------------------------- /error_handling/interactive_debugger.rst: -------------------------------------------------------------------------------- 1 | 2 | Interactive Python Debugger 3 | =========================== 4 | 5 | The ``ipdb`` program is an *interactive debugger* that allows you to execute a program in slow motion. 6 | 7 | Install it with: 8 | 9 | :: 10 | 11 | pip install ipdb 12 | 13 | Then you can start your program in debugging mode from the terminal: 14 | 15 | :: 16 | 17 | python -m ipdb dungeon_explorer.py 18 | 19 | In ``ipdb`` you have a couple of keyboard shortcuts that allow you to navigate the programs execution: 20 | 21 | =================== ======================================================================== 22 | command description 23 | =================== ======================================================================== 24 | ``n`` execute the next line 25 | ``s`` execute the next line. If it contains a function, step inside it 26 | ``b 57`` set a breakpoint at line 57 27 | ``c`` continue execution until the next breakpoint 28 | ``ll`` list lots of lines around the current one 29 | ``myvar`` print contents of the variable ``myvar`` 30 | ``myvar = 1`` modify a variable 31 | =================== ======================================================================== 32 | -------------------------------------------------------------------------------- /error_handling/loggers.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import sys 3 | 4 | # write log messages to text file and standard output 5 | log = logging.getLogger('example logger') 6 | log.setLevel(logging.INFO) 7 | 8 | fmt='%(asctime)s | %(message)s' 9 | format = logging.Formatter(fmt, datefmt='%m/%d/%Y %I:%M:%S %p') 10 | 11 | handler = logging.StreamHandler(sys.stderr) 12 | handler.setFormatter(format) 13 | log.addHandler(handler) 14 | 15 | handler2 = logging.FileHandler('logfile.log', mode='w') 16 | handler2.setFormatter(format) 17 | log.addHandler(handler2) 18 | 19 | log.info('message from logger ') 20 | log.warning('message from logger 2') 21 | log.error('an error has occured') 22 | -------------------------------------------------------------------------------- /error_handling/logging.rst: -------------------------------------------------------------------------------- 1 | 2 | Logging 3 | ======= 4 | 5 | **Logging** is used collect diagnostic information from a program in case you need to debug or analyze its execution later. 6 | Logging is conceptually similar to adding ``print`` statements to your code, but with more fine-grained control. 7 | In Python, the ``logging`` module provides an entry point to generate log messages: 8 | 9 | .. literalinclude:: logging_example.py 10 | 11 | 12 | .. topic:: Exercise 13 | 14 | Change the logging level through ``DEBUG, ERROR, WARNING, INFO and CRITICAL`` 15 | and observe what is the priority order of the display levels. 16 | 17 | ---- 18 | 19 | Multiple Loggers 20 | ---------------- 21 | 22 | It is possible to create multiple loggers with different output channels, like screen, file or HTTP services. 23 | Each of them can be configured individually. 24 | Some of the options of the logging module include: 25 | 26 | =============================== ==================================================== 27 | option description 28 | =============================== ==================================================== 29 | level order of precedence 30 | filename where to save the log-messages 31 | format string that specifies what the message looks like 32 | filter more sophisticated filtering than with levels only 33 | =============================== ==================================================== 34 | 35 | Whether you need multiple loggers or not, the code example below contains a few ways to customize your log messages: 36 | 37 | .. literalinclude:: loggers.py 38 | -------------------------------------------------------------------------------- /error_handling/logging_example.py: -------------------------------------------------------------------------------- 1 | 2 | import logging 3 | 4 | 5 | logging.basicConfig(filename='debug.log', level=logging.DEBUG) 6 | 7 | 8 | def factorial(n=10): 9 | """Calculates factorials with log messages.""" 10 | i = 1 11 | factorial = 1 12 | while i < n: 13 | logging.info('starting iteration {}'.format(i)) 14 | factorial *= i 15 | logging.debug('new factorial: {}'.format(factorial)) 16 | i += 1 17 | logging.warning('Final result: {}'.format(factorial)) 18 | 19 | 20 | if __name__ == '__main__': 21 | logging.warning('this is a warning message') 22 | factorial(10) 23 | logging.critical('Factorial calculation ended') 24 | -------------------------------------------------------------------------------- /error_handling/warnings.rst: -------------------------------------------------------------------------------- 1 | Warnings 2 | ======== 3 | 4 | The ``warnings`` module writes warning messages to the ``sys.stderr`` 5 | stream: 6 | 7 | .. code:: python3 8 | 9 | import warnings 10 | 11 | warnings.warn("This is a drill!") 12 | 13 | The output warning also contains a line number: 14 | 15 | :: 16 | 17 | warn.py:3: UserWarning: This is a drill! 18 | 19 | Python has an option to stop with an error as soon as a warning occurs: 20 | 21 | :: 22 | 23 | python -W error warnme.py 24 | 25 | You could also mute all warnings: 26 | 27 | :: 28 | 29 | python -W ignore warnme.py 30 | -------------------------------------------------------------------------------- /examples/breakout.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/examples/breakout.png -------------------------------------------------------------------------------- /examples/breakout.py: -------------------------------------------------------------------------------- 1 | """ 2 | Breakout 3 | 4 | Install dependencies: 5 | 6 | pip install opencv-python pydantic 7 | 8 | (c) 2024 Dr. Kristian Rother 9 | """ 10 | import numpy as np 11 | import cv2 12 | import random 13 | from pydantic import BaseModel 14 | 15 | BR_ROWS, ROWS, COLS = 5, 20, 12 16 | TILE_SIZE_Y, TILE_SIZE_X = 32, 64 17 | SCREEN_SIZE_X, SCREEN_SIZE_Y = COLS * TILE_SIZE_X, ROWS * TILE_SIZE_Y 18 | 19 | FRAME_SPEED = 10 20 | MOVE_SPEED = 3 21 | 22 | 23 | def ticker(speed: int): 24 | counter = speed 25 | while True: 26 | for _ in range(counter): 27 | yield False 28 | yield True 29 | 30 | 31 | class Ball(BaseModel): 32 | x: int 33 | y: int 34 | dx: int = 0 35 | dy: int = 0 36 | size: int = 16 37 | docked: bool = True 38 | 39 | def start(self): 40 | if self.docked: 41 | self.dx = -2 42 | self.dy = -2 43 | self.docked = False 44 | 45 | def move(self, paddle, bricks, background): 46 | self.x += self.dx 47 | self.y += self.dy 48 | self.check_border() 49 | self.hit_paddle(paddle) 50 | self.hit_brick(bricks, background) 51 | 52 | def check_border(self): 53 | if self.y <= 0: 54 | self.y = 0 55 | self.dy = -self.dy 56 | if self.x <= 0: 57 | self.x = 0 58 | self.dx = -self.dx 59 | if self.x >= SCREEN_SIZE_X - 16: 60 | self.x = SCREEN_SIZE_X - 16 61 | self.dx = -self.dx 62 | 63 | def hit_paddle(self, paddle): 64 | if ( 65 | self.y == paddle.y - 16 66 | and self.x >= paddle.x - 16 67 | and self.x < paddle.x + paddle.width 68 | ): 69 | self.dy = random.choice([-1, -2]) 70 | self.dx = self.dx // abs(self.dx) * random.choice([1, 2, 3]) 71 | 72 | def hit_brick(self, bricks, background): 73 | xx, yy = self.x // TILE_SIZE_X, self.y // TILE_SIZE_Y 74 | if xx < COLS and yy < BR_ROWS and bricks[yy, xx] == 1: 75 | self.dx *= random.choice([-1, 1]) 76 | self.dy *= -1 77 | bricks[yy, xx] = 0 78 | background[ 79 | yy * TILE_SIZE_Y : (yy + 1) * TILE_SIZE_Y, 80 | xx * TILE_SIZE_X : (xx + 1) * TILE_SIZE_X, 81 | :, 82 | ] = 0 83 | 84 | 85 | class Paddle(BaseModel): 86 | x: int 87 | y: int 88 | width: int 89 | height: int = 16 90 | speed: int 91 | 92 | def move(self, ball, dx): 93 | self.x += dx 94 | if ball.docked: 95 | ball.x += dx 96 | 97 | def right(self, ball): 98 | if self.x < SCREEN_SIZE_X - self.width - self.speed: 99 | self.move(ball, self.speed) 100 | 101 | def left(self, ball): 102 | if self.x >= self.speed: 103 | self.move(ball, -self.speed) 104 | 105 | 106 | def draw(background, paddle, ball): 107 | frame = background.copy() 108 | frame[ 109 | paddle.y : paddle.y + paddle.height, paddle.x : paddle.x + paddle.width, : 110 | ] = 128 111 | frame[ball.y : ball.y + ball.size, ball.x : ball.x + ball.size, :] = 255 112 | cv2.imshow("Breakout", frame) 113 | 114 | 115 | bricks = np.ones((BR_ROWS, COLS), dtype=np.uint8) 116 | background = np.zeros((SCREEN_SIZE_Y, SCREEN_SIZE_X, 3), np.uint8) 117 | 118 | # draw the bricks 119 | background[TILE_SIZE_Y * 0 : TILE_SIZE_Y * 1, :] = (0, 0, 255) 120 | background[TILE_SIZE_Y * 1 : TILE_SIZE_Y * 2, :] = (0, 128, 255) 121 | background[TILE_SIZE_Y * 2 : TILE_SIZE_Y * 3, :] = (0, 255, 255) 122 | background[TILE_SIZE_Y * 3 : TILE_SIZE_Y * 4, :] = (0, 255, 0) 123 | background[TILE_SIZE_Y * 4 : TILE_SIZE_Y * 5, :] = (255, 0, 0) 124 | 125 | ball = Ball(x=425, y=600 - 16) 126 | paddle = Paddle(x=400, y=600, width=100, speed=16) 127 | 128 | frame_tick = ticker(FRAME_SPEED) 129 | move_tick = ticker(MOVE_SPEED) 130 | 131 | while bricks.sum() > 0 and ball.y < SCREEN_SIZE_Y: 132 | # draw everything 133 | if next(frame_tick): 134 | draw(background, paddle, ball) 135 | 136 | # handle keyboard input 137 | key = chr(cv2.waitKey(1) & 0xFF) 138 | if key == "a": 139 | paddle.left(ball) 140 | elif key == "d": 141 | paddle.right(ball) 142 | elif key == " ": 143 | ball.start() 144 | elif key == "q": 145 | break 146 | 147 | if next(move_tick) and not ball.docked: 148 | ball.move(paddle, bricks, background) 149 | 150 | cv2.destroyAllWindows() 151 | -------------------------------------------------------------------------------- /examples/breakout.rst: -------------------------------------------------------------------------------- 1 | 2 | Breakout 3 | ======== 4 | 5 | An OpenCV implementation of the oldschool game **Breakout**: 6 | 7 | .. image:: breakout.png 8 | 9 | The OpenCV keyboard handling is not super smooth yet. 10 | This might work better with one of the Python gaming libraries like `arcade`. 11 | 12 | .. literalinclude:: breakout.py 13 | 14 | 15 | .. seealso:: 16 | 17 | `The 1976 Breakout game `__ 18 | 19 | See who is among the authors! -------------------------------------------------------------------------------- /examples/snake.py: -------------------------------------------------------------------------------- 1 | """ 2 | Snake 3 | 4 | Install dependencies: 5 | 6 | pip install opencv-python pydantic 7 | 8 | (c) 2024 Dr. Kristian Rother 9 | """ 10 | import numpy as np 11 | import cv2 12 | from itertools import cycle 13 | from random import randint 14 | from pydantic import BaseModel 15 | 16 | ROWS, COLS = 14, 20 17 | TILE_SIZE = 32 18 | SCREEN_SIZE_X = COLS * TILE_SIZE 19 | SCREEN_SIZE_Y = ROWS * TILE_SIZE 20 | 21 | FRAME_SPEED = 10 22 | MOVE_SPEED = 100 23 | 24 | MOVES = { 25 | "a": (-1, 0), # left 26 | "d": (1, 0), # right 27 | "w": (0, -1), # up 28 | "s": (0, 1), # down 29 | } 30 | 31 | RAINBOW = [ 32 | (0, 0, 255), # BGR - blue, green red 33 | (0, 128, 255), 34 | (0, 255, 255), 35 | (0, 255, 0), 36 | (255, 0, 0), 37 | (255, 0, 255), 38 | ] 39 | 40 | 41 | Color = tuple[int, int, int] 42 | Position = tuple[int, int] 43 | 44 | 45 | class Food(BaseModel): 46 | position: Position 47 | color: Color 48 | 49 | 50 | class Snake(BaseModel): 51 | tail: list[Position] = [] # the tail is a queue 52 | colors: list[Color] = [] 53 | direction: Position = (1, 0) 54 | 55 | @property # decorator: allows us to use .head like an attribute 56 | def head(self): 57 | return self.tail[0] 58 | 59 | def move(self, food): 60 | x, y = self.head # calls the function automatically 61 | dx, dy = self.direction # unpack a tuple into two variables 62 | new_head = x + dx, y + dy # create a new position tuple 63 | self.tail.insert(0, new_head) 64 | if new_head != food.position: 65 | self.tail.pop() 66 | else: 67 | self.colors.insert(0, food.color) 68 | 69 | def collides(self): 70 | x, y = self.head 71 | return ( 72 | self.head in self.tail[1:] 73 | or x < 0 74 | or y < 0 75 | or x == COLS 76 | or y == ROWS 77 | ) 78 | 79 | 80 | def draw(snake, food): 81 | frame = np.zeros((SCREEN_SIZE_Y, SCREEN_SIZE_X, 3), np.uint8) 82 | for (fx, fy), col in zip(snake.tail, snake.colors): 83 | frame[ 84 | fy * TILE_SIZE : (fy + 1) * TILE_SIZE - 1, 85 | fx * TILE_SIZE : (fx + 1) * TILE_SIZE - 1, 86 | :, 87 | ] = col 88 | fx, fy = food.position 89 | frame[ 90 | fy * TILE_SIZE : (fy + 1) * TILE_SIZE, fx * TILE_SIZE : (fx + 1) * TILE_SIZE, : 91 | ] = food.color 92 | cv2.imshow("Snake", frame) 93 | 94 | 95 | color_gen = cycle(RAINBOW) 96 | 97 | snake = Snake(tail=[(5, 5)], colors=[(255, 0, 255)]) 98 | food = Food(position=(10, 5), color=next(color_gen)) 99 | 100 | frame_tick = cycle([True] + [False] * FRAME_SPEED) 101 | move_tick = cycle([True] + [False] * MOVE_SPEED) 102 | 103 | while not snake.collides(): 104 | # draw everything 105 | if next(frame_tick): 106 | draw(snake, food) 107 | 108 | # handle keyboard input 109 | key = chr(cv2.waitKey(1) & 0xFF) 110 | if key in MOVES: 111 | snake.direction = MOVES[key] 112 | elif key == "q": 113 | break 114 | 115 | if next(move_tick): 116 | snake.move(food) 117 | 118 | if snake.head == food.position: 119 | food = Food( 120 | position=(randint(1, COLS - 1), randint(1, ROWS - 1)), 121 | color=next(color_gen) 122 | ) 123 | 124 | cv2.destroyAllWindows() 125 | -------------------------------------------------------------------------------- /examples/tetris.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/examples/tetris.png -------------------------------------------------------------------------------- /examples/tetris.py: -------------------------------------------------------------------------------- 1 | """ 2 | Tetris 3 | 4 | Install dependencies: 5 | 6 | pip install opencv-python 7 | """ 8 | import numpy as np 9 | import cv2 10 | import random 11 | 12 | ROWS, COLS = 17, 10 13 | TILE_SIZE = 32 14 | SCREEN_SIZE_X, SCREEN_SIZE_Y = COLS * TILE_SIZE, ROWS * TILE_SIZE 15 | 16 | # box with three extra layers of wall for rotations and moves 17 | box = np.ones((ROWS + 2, COLS + 4), dtype=np.uint8) 18 | box[:-3, 3:-3] = 0 19 | 20 | # definition of bricks 21 | BRICKS = [ 22 | """ 23 | .#. 24 | ### 25 | ... 26 | """, """ 27 | .#. 28 | ##. 29 | #.. 30 | """, """ 31 | .#. 32 | .## 33 | ..# 34 | """, """ 35 | .## 36 | .#. 37 | .#. 38 | """, """ 39 | ##. 40 | .#. 41 | .#. 42 | """, """ 43 | ## 44 | ## 45 | """, """ 46 | ..#. 47 | ..#. 48 | ..#. 49 | ..#. 50 | """, """ 51 | #. 52 | #. 53 | """ 54 | ] 55 | BRICKS = [ 56 | (np.array([list(row) for row in b.strip().split()]) == "#").astype(np.uint8) 57 | for b in BRICKS 58 | ] 59 | 60 | 61 | # create first brick 62 | brick = random.choice(BRICKS) 63 | x, y = 5, 0 64 | 65 | timer = 150 66 | box_full = False 67 | while not box_full: 68 | 69 | # draw everything 70 | frame = np.zeros((SCREEN_SIZE_Y, SCREEN_SIZE_X, 3), np.uint8) 71 | box_cut = box[:-2, 2:-2] 72 | bb = np.kron(box_cut, np.ones((TILE_SIZE, TILE_SIZE), dtype=np.uint8)) * 192 73 | frame[:,:,0] = bb 74 | frame[:,:,1] = bb 75 | frame[:,:,2] = bb 76 | 77 | br = np.kron(brick, np.ones((TILE_SIZE, TILE_SIZE), dtype=np.uint8)) * 255 78 | frame[y*TILE_SIZE:y*TILE_SIZE+br.shape[0], (x-2)*TILE_SIZE:(x-2)*TILE_SIZE+br.shape[1],0] += br 79 | 80 | cv2.imshow("Tetris", frame) 81 | 82 | # handle keyboard input 83 | key = chr(cv2.waitKey(1) & 0xFF) 84 | newbrick = brick 85 | newx, newy = x, y 86 | if key == "d": 87 | newx += 1 88 | elif key == "a": 89 | newx -= 1 90 | elif key == "s": 91 | newy += 1 92 | elif key == "w": 93 | newbrick = brick.T[::-1] # swap x & y, then mirror y 94 | elif key == "q": 95 | box_full = True 96 | 97 | # place brick in new position if it does not collide 98 | check = box.copy() 99 | check[newy:newy+newbrick.shape[0], newx:newx+newbrick.shape[1]] += newbrick 100 | if not (check == 2).any(): 101 | brick = newbrick 102 | x, y = newx, newy 103 | 104 | timer -= 1 105 | if timer == 0: 106 | # brick drops automatically 107 | timer = 150 108 | y += 1 109 | check = box.copy() 110 | check[y:y+brick.shape[0], newx:newx+brick.shape[1]] += brick 111 | if (check == 2).any(): 112 | # brick stops moving, place it in the box 113 | if y == 1: 114 | box_full = True 115 | y -= 1 116 | box[y:y+brick.shape[0], newx:newx+brick.shape[1]] += brick 117 | 118 | # remove rows 119 | for row in range(box.shape[0] - 3): 120 | if box[row].sum() == box.shape[1]: 121 | box[1:row + 1] = box[:row] 122 | box[0, 3:-3] = 0 123 | 124 | x, y = 4, 0 125 | brick = random.choice(BRICKS) 126 | 127 | cv2.destroyAllWindows() 128 | -------------------------------------------------------------------------------- /examples/tetris.rst: -------------------------------------------------------------------------------- 1 | Tetris 2 | ====== 3 | 4 | .. image:: tetris.png 5 | 6 | In :download:`tetris.py`, you find a complete Tetris game. 7 | It makes heavy use of the Numpy library. 8 | The code was created in a coding speedrun; it almost has no structure. 9 | 10 | .. note:: 11 | 12 | There is also a bug when dropping long bricks to the ground. 13 | Maybe some refactoring makes this issue easier to debug. 14 | 15 | .. literalinclude:: tetris.py 16 | -------------------------------------------------------------------------------- /exercises/class_inheritance.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/exercises/class_inheritance.py -------------------------------------------------------------------------------- /exercises/collection_exercise.py: -------------------------------------------------------------------------------- 1 | """ 2 | Create a dictionary that 3 | has names from the list as keys 4 | and is initalized with a random number for each name. 5 | 6 | Then double that number for each name. 7 | """ 8 | 9 | 10 | names = [ 11 | "Adam", "Bea", "Charlie", "Danielle", 12 | "Eve", "Frantz", "Gustav", "Helena" 13 | ] 14 | -------------------------------------------------------------------------------- /exercises/comprehension_exercise.py: -------------------------------------------------------------------------------- 1 | """ 2 | Exercise: 3 | 4 | Write one-liners that: 5 | 6 | * produce a list with the length of each name 7 | * produce a list with the first character of each name 8 | * produce a list with all names having max 6 characters 9 | * produce a dict with {name:length} 10 | """ 11 | 12 | names = [ 13 | 'Emily', 'Hannah', 'Madison', 'Ashley', 'Sarah', 14 | 'Alexis', 'Samantha', 'Jessica', 'Elizabeth', 'Taylor', 15 | 'Lauren', 'Alyssa', 'Kayla', 'Abigail', 'Brianna', 16 | 'Olivia', 'Emma', 'Megan', 'Grace', 'Victoria' 17 | ] 18 | -------------------------------------------------------------------------------- /exercises/exercise_fibonacci.py: -------------------------------------------------------------------------------- 1 | 2 | ''' 3 | Write a generator function that generates fibonacci numbers. 4 | 5 | Retrieve a few values using next(). 6 | Retrieve values using a loop. 7 | ''' 8 | 9 | def fibonacci(): 10 | pass 11 | 12 | -------------------------------------------------------------------------------- /exercises/exercise_logging.py: -------------------------------------------------------------------------------- 1 | 2 | ''' 3 | Write a decorator which wraps functions to log function arguments 4 | and the return value on each call. 5 | 6 | Implement the 'logged' decorator so that the program produces 7 | the following output: 8 | 9 | you called func(4, 4, 4) 10 | it returned 6 11 | 6 12 | 13 | ''' 14 | 15 | @logged 16 | def func(*args): 17 | return 3 + len(args) 18 | 19 | 20 | print func(4, 4, 4) 21 | 22 | -------------------------------------------------------------------------------- /exercises/exercises.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/exercises/exercises.pdf -------------------------------------------------------------------------------- /exercises/exercises_and_solutions.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/exercises/exercises_and_solutions.pdf -------------------------------------------------------------------------------- /functions/decorators.rst: -------------------------------------------------------------------------------- 1 | Decorators 2 | ========== 3 | 4 | In general, **decorators are functions that manipulate functions**. 5 | More specifically, a decorator wraps a function to add extra behavior to a 6 | function call. 7 | In the examples below, we will decorate a Python function that 8 | calculates fibonacci numbers. 9 | 10 | Using built-in decorators 11 | ------------------------- 12 | 13 | Most of the time, you will use built-in decorators. One example is 14 | ``functools.lru_cache`` that memorizes the output of a function to save 15 | time later. Let's decorate a function with it: 16 | 17 | .. code:: python3 18 | 19 | from functools import lru_cache 20 | 21 | @lru_cache 22 | def fibonacci(n): 23 | """Recursively calculates fibonacci numbers""" 24 | if n < 2: 25 | return n 26 | return fibonacci(n-1) + fibonacci(n-2) 27 | 28 | Try calculating a recursive fibonacci number for ``n=50`` without the 29 | decorator. It takes forever! By default, ``lru_cache`` memorizes the 30 | last 128 results, so above that the recursive fibonacci becomes slow 31 | again. 32 | 33 | Built-in decorators are also used in: 34 | 35 | - web frameworks like **Flask** and **FastAPI** to assign URLs to Python functions 36 | - the **pytest** framework to create compact test code 37 | - classes with ``@property`` and ``@staticmethod`` (described in the OOP section) 38 | 39 | -------------- 40 | 41 | Writing your own decorators 42 | --------------------------- 43 | 44 | If want to add functionality for which no decorator exists, 45 | e.g. printing a timestamp for every addition, you could define a new function: 46 | 47 | .. code:: python3 48 | 49 | import time 50 | 51 | def fibonacci_with_timestamp(n): 52 | print(time.asctime()) 53 | return fibonacci(n) 54 | 55 | If you want to **add the timestamp feature to many functions**, consider 56 | using a decorator: 57 | 58 | .. code:: python3 59 | 60 | def print_timestamp(func): 61 | def wrapper(*args): 62 | print(time.asctime()) # done before addition 63 | result = func(*args) # calls the addition function 64 | ... # actions after addition 65 | return result 66 | return wrapper 67 | 68 | 69 | @print_timestamp 70 | def fibonacci(n): 71 | """Recursively calculates fibonacci numbers""" 72 | if n < 2: 73 | return n 74 | return fibonacci(n-1) + fibonacci(n-2) 75 | 76 | You can argue that this does not simplify the code. 77 | Decorators pays off in bigger programs, when they are used often. 78 | Logging function calls is a good example for using a decorator. 79 | 80 | 81 | -------------- 82 | 83 | Wrapping Decorators 84 | ------------------- 85 | 86 | The **``wraps``** decorator copies documentation strings into the 87 | decorator function, so that the decorated function looks like the 88 | original one. It is useful when writing your own decorators. 89 | 90 | .. code:: python3 91 | 92 | import functools 93 | 94 | def print_timestamp(func): 95 | @functools.wraps(func) 96 | def wrapper(*args): 97 | print(time.asctime()) # done before addition 98 | result = func(*args) # calls the addition function 99 | ... # actions after addition 100 | return result 101 | return wrapper 102 | 103 | @print_timestamp 104 | def fibonacci(n): 105 | """Recursively calculates fibonacci numbers""" 106 | if n < 2: 107 | return n 108 | return fibonacci(n-1) + fibonacci(n-2) 109 | 110 | 111 | # check docstring - would not work without @wraps 112 | print(help(addition)) 113 | -------------------------------------------------------------------------------- /functions/function_parameters.rst: -------------------------------------------------------------------------------- 1 | Function parameters 2 | =================== 3 | 4 | There are four types of function parameters in Python: 5 | 6 | - obligatory parameters 7 | - optional parameters 8 | - list parameters (`*args`) 9 | - keyword parameters (`**kwargs`) 10 | 11 | The following example uses all four of them: 12 | 13 | .. code:: python3 14 | 15 | def example(obligatory, optional=77, *args, **kwargs): 16 | print("obligatory: ", obligatory) 17 | print("optional : ", optional) 18 | print("args : ", args) 19 | print("kwargs : ", kwargs) 20 | 21 | 22 | example(True, 99, 77, 55, a=33, b=11) 23 | 24 | -------------- 25 | 26 | Mutable and immutable parameters 27 | -------------------------------- 28 | 29 | Depending on their data type, some function parameters are **mutable**, 30 | others are not: 31 | 32 | - lists, dictionaries and sets are **mutable** 33 | - integers, floats, strings and tuples are **immutable** 34 | 35 | Here is a small illustration: 36 | 37 | .. code:: python3 38 | 39 | def change(var1, var2): 40 | """Change two values.""" 41 | var1 += [7] 42 | var2 += 7 43 | 44 | 45 | data = [3,4,5] 46 | number = 6 47 | 48 | change(data, number) 49 | 50 | print("The list now contains", data) 51 | print("The number is now", number) 52 | 53 | 54 | .. topic:: Good function style 55 | 56 | - Arguments for input only. 57 | - Return statement for output only. 58 | - No global variables. 59 | - One function serves exactly one purpose. 60 | - Write documentation string first. 61 | - keep functions small (below one screen page) 62 | - If you have too many parameters, make a new class. 63 | - Do not mix mutable types and return output. 64 | -------------------------------------------------------------------------------- /functions/functools.rst: -------------------------------------------------------------------------------- 1 | Map-Filter-Reduce 2 | ================= 3 | 4 | Python offers many functions that can be used to implement patterns from 5 | *functional programming*. 6 | 7 | **Functional Programming** is a programming philosophy where a program 8 | is defined by functions. These functions are stateless (i.e. there are 9 | no global variables). This paradigm helps designing distributed and 10 | concurrent algorithms. 11 | 12 | Although one would rarely implement these in pure Python, it is a good 13 | exercise. Functional code often comes out very clean and is easy to 14 | debug and maintain. 15 | 16 | A side effect is that it helps you to avoid class-based implementations, 17 | which often are more wordy. 18 | 19 | ---- 20 | 21 | Map 22 | --- 23 | 24 | The ``map()`` function applies a function to all elements of an 25 | iterable. It is returning a generator. ``map()`` is an alternative to a 26 | comprehension. 27 | 28 | .. code:: python3 29 | 30 | def square(x): 31 | return x ** 2 32 | 33 | squares = map(square, numbers) 34 | print(squares) 35 | print(list(squares)) 36 | 37 | -------------- 38 | 39 | Filter 40 | ------ 41 | 42 | The ``filter()`` function returns all elements of an iterable for which 43 | a function returns ``True``. It is the functional equivalent of the 44 | ``if`` clause in a comprehension. 45 | 46 | .. code:: python3 47 | 48 | def odd(x): 49 | return x % 2 != 0 50 | 51 | odd_numbers = filter(odd, range(10)) 52 | print(odd_numbers) 53 | print(list(odd_numbers)) 54 | 55 | -------------- 56 | 57 | Reduce 58 | ------ 59 | 60 | The ``reduce`` function aggregates data by recursively calling the same 61 | function. For instance, you could use it to add numbers and concatenate 62 | lists: 63 | 64 | .. code:: python3 65 | 66 | from functools import reduce 67 | 68 | numbers = [1, 2, 3, 4, 5, 6, 7, 8] 69 | 70 | 71 | def add(x, y): 72 | return x + y 73 | 74 | print(reduce(add, numbers)) 75 | print(reduce(add, [[1, 2, 3], [4, 5, 6]])) 76 | 77 | ``reduce`` allows for quite sophisticated patterns: 78 | 79 | .. code:: python3 80 | 81 | from functools import reduce 82 | 83 | words = [ 84 | (455, 'sea'), 85 | (336, 'boat'), 86 | (281, 'white'), 87 | (1226, 'whale'), 88 | (329, 'captain'), 89 | (510, 'Ahab') 90 | ] 91 | 92 | 93 | def newlines(a, b): 94 | return a + '\n' + b 95 | 96 | 97 | def wordformat(x): 98 | number, word = x 99 | return f"{word:>10s}:{number:5}" 100 | 101 | 102 | print(reduce(newlines, map(wordformat, words))) 103 | 104 | -------------- 105 | 106 | Partial 107 | ------- 108 | 109 | The ``functools`` module contains a couple of ways to manipulate 110 | functions. One of them is ``partial`` that fills in a part of the 111 | parameters, resulting in a new function: 112 | 113 | .. code:: python3 114 | 115 | from functools import partial 116 | 117 | add3 = partial(add, 3) # add3 is a function 118 | print(add3(5)) # results in 8 119 | 120 | add5 = partial(add, 5) # can be done more than once 121 | print(add5(8)) # results in 13 122 | -------------------------------------------------------------------------------- /functions/generators.rst: -------------------------------------------------------------------------------- 1 | Generators 2 | ========== 3 | 4 | Generators are *lazy* functions. They produce results like normal 5 | Python functions, but only when they are needed. The main purpose of 6 | using generators is to save memory and calculation time when processing 7 | big datasets. 8 | 9 | A Python generator is indicated by the ``yield`` keyword. 10 | A ``return`` statement in a generator function still terminates it. 11 | 12 | ---- 13 | 14 | A minimalistic example 15 | ---------------------- 16 | 17 | Calling the ``powers()`` function initializes the generator. Every time 18 | ``next()`` is called, it pulls out one value from the generator. 19 | 20 | .. code:: python3 21 | 22 | def powers(): 23 | yield 1 24 | yield 2 25 | yield 4 26 | yield 8 27 | yield 16 28 | 29 | 30 | gen = powers() 31 | print(next(gen)) 32 | print(next(gen)) 33 | print(next(gen)) 34 | 35 | -------------- 36 | 37 | Lazy evaluation 38 | --------------- 39 | 40 | Note that the results of the generator are not pre-calculated. Every 41 | call of ``next()`` executes the code inside the generator function until 42 | it encounters the next ``yield`` statement. 43 | 44 | The following code proves that: 45 | 46 | .. code:: python3 47 | 48 | def count(): 49 | for i in range(10): 50 | print('checkpoint A') 51 | yield i 52 | 53 | gen = count() 54 | print('checkpoint B') 55 | print(next(gen)) 56 | print(next(gen)) 57 | 58 | The first call of the generator does nothing yet. Only when ``next()`` 59 | requests the next value, the generator function is executed until the 60 | ``yield`` statement. Then it pauses until the next ``yield`` and so on. 61 | 62 | -------------- 63 | 64 | Consuming values 65 | ---------------- 66 | 67 | To get all values out of a generator, you can use a ``for`` loop or 68 | convert to a list: 69 | 70 | .. code:: python3 71 | 72 | for x in count(): 73 | print(x, ) 74 | 75 | print(list(count())) 76 | 77 | -------------- 78 | 79 | Endless loops 80 | ------------- 81 | 82 | A common generator pattern is using an endless ``while`` loop to 83 | generate a series. 84 | 85 | .. code:: python3 86 | 87 | def powers(): 88 | """generates infinite powers of two""" 89 | n = 1 90 | while True: 91 | yield n 92 | n *= 2 93 | 94 | Pulling out values with ``next()`` does not result in an endless loop. 95 | Of course, you shouldn’t try to consume this generator in a loop or 96 | using ``list()``. 97 | 98 | -------------- 99 | 100 | Generator Expressions 101 | --------------------- 102 | 103 | You can create ad-hoc generators similar to a list comprehension: 104 | 105 | .. code:: python3 106 | 107 | squares = (x ** 2 for x in range(100)) 108 | 109 | print(next(squares)) 110 | 111 | -------------- 112 | 113 | Iterators 114 | --------- 115 | 116 | The thing returned by a generator is called an **iterator**. Many 117 | functions in Python return iterators (e.g. ``range()``, ``enumerate()`` 118 | and ``zip()``). 119 | 120 | Among the things you can do to iterators are: 121 | 122 | - request values with ``next``. 123 | - use them in a ``for`` loop. 124 | - convert them to lists with ``list()``. 125 | 126 | The ``iter()`` function creates a generator from any iterable: 127 | 128 | .. code:: python3 129 | 130 | gen = iter("Hello World") 131 | print(next(gen)) # -> 'H' 132 | print(next(gen)) # -> 'e' 133 | -------------------------------------------------------------------------------- /functions/levels.rst: -------------------------------------------------------------------------------- 1 | Writing a new Function 2 | ====================== 3 | 4 | In this exercise, you will rehearse writing a new Python function. 5 | 6 | Requirement: Edit Levels 7 | ------------------------ 8 | 9 | The Dungeon levels should be easy to edit, so that we can add multiple levels more easily. 10 | 11 | The Design 12 | ---------- 13 | 14 | At this stage, you should have a class ``DungeonExplorer`` that is the top-level class 15 | for the game logic. 16 | It has defined attributes that contain everything: the player, walls, coins and anything 17 | else you have implemented already. 18 | The exact attributes are not important, all we need to know is that they are somewhere 19 | inside the ``DungeonExplorer`` class and you know their names. 20 | 21 | For creating levels, we want to define a new function ``start_level()`` with the following signature: 22 | 23 | .. code:: python3 24 | 25 | def start_level( 26 | dungeon: DungeonExplorer, 27 | level: list[str], 28 | start_position: list[int], 29 | **kwargs 30 | ) -> None: 31 | ... 32 | 33 | Here, **dungeon** is a DungeonExplorer game object, **level** should be a map of the level, **start_position** where the player starts. 34 | The ``**kwargs`` parameter is a placeholder for things we might add later. 35 | The function does not return anything. 36 | 37 | Step 1: Create a test 38 | --------------------- 39 | 40 | Create a test function in a new file ``test_level.py``: 41 | 42 | .. code:: python3 43 | 44 | def test_start_level(): 45 | d = DungeonExplorer(...) # add necessary parameters but they can be empty 46 | level=[ 47 | "########", 48 | "#.#....#", 49 | "#.#.##.#", 50 | "#.#..#.#", 51 | "#.##.#.#", 52 | "#....#x#", 53 | "########", 54 | ] 55 | start_level(d, level=level, start_position={"x": 1, "y": 1}) 56 | assert [1, 1, "player"] in d.get_objects() # change if your interface is different 57 | 58 | The **#** symbols are walls, the **.** are floor, and the **x** is an exit to the next level. 59 | With such a data structure, we should be able to edit levels in a text editor easily! 60 | 61 | Add imports as necessary. 62 | Run the test with ``pytest`` from the command line. 63 | It should fail. 64 | 65 | Step 2: Implement the function 66 | ------------------------------ 67 | 68 | First, copy the empty function from the design into the code with the game logic. 69 | For now, we keep the function outside of the ``DungeonExplorer`` class. 70 | 71 | Then, write code in the function body that changes the position of the player in the ``dungeon`` object. 72 | 73 | Run the test again. They should pass now. 74 | 75 | .. note:: 76 | 77 | You now have a function that *modifies* one of its arguments. 78 | It is a good practice to not modify arguments and return results at the same time. 79 | 80 | 81 | Step 3: Handle walls 82 | -------------------- 83 | 84 | Add the following line to ``test_start_level()``: 85 | 86 | .. code:: python3 87 | 88 | assert [4, 2, "wall"] in d.get_objects() # an example wall 89 | 90 | Modify the function body to set the walls of the ``dungeon`` object as well. 91 | To find all walls, you probably will have to iterate through all the positions 92 | of the dungeon somewhere (in ``start_level()`` or ``DungeonExplorer.get_objects()``). 93 | You can loop over a list of strings using the ``enumerate`` function with the following pattern: 94 | 95 | .. code:: python3 96 | 97 | for y, row in enumerate(level): # y is a row number 0, 1, 2, ... 98 | for x, tile in enumerate(row): # x is a column number 0, 1, 2, ... 99 | ... # tile is a single character (#, . or x) 100 | 101 | Run all tests afterwards and make sure they pass. 102 | 103 | Step 4: Handle coins 104 | -------------------- 105 | 106 | To include a coin in the test, modify the call in ``test_start_level()``: 107 | 108 | .. code:: python3 109 | 110 | coin = {'position': {"x": 1, "y": 5}, 'value': 100} 111 | start_level(d, level=level, start_position={"x": 1, "y": 1}, coins=[coin]) 112 | 113 | And add an assertion to the test function: 114 | 115 | .. code:: python3 116 | 117 | assert [1, 5, "coin"] in d.get_objects() 118 | 119 | Even though we haven't defined coins as part of the function signature, we can use them. 120 | This is what the ``**kwargs`` is for. 121 | Inside the ``start_level()`` function, check whether there are coins in the ``**kwargs`` dictionary: 122 | 123 | .. code:: python3 124 | 125 | if "coins" in kwargs: 126 | for coin in kwargs["coins"]: 127 | ... # coin is a dictionary with the fields 'position' and 'value' 128 | 129 | Implement the coins and make sure the tests pass. 130 | 131 | Step 5: Use the function 132 | ------------------------ 133 | 134 | Modify the ``graphics_engine.py`` module so that: 135 | 136 | - it imports the ``start_level()`` function. 137 | - it calls ``start_level()`` to define a dungeon. 138 | 139 | To make things a bit easier, you might want to add default values to 140 | the attributes of ``DungeonExplorer`` so that it becomes easier to leave them away. 141 | 142 | Make sure the game is still playable. 143 | 144 | Step 6: Load levels from a file 145 | ------------------------------- 146 | 147 | Create a text file that you call ``level_01.json``. 148 | Place the following there. 149 | 150 | .. code:: json 151 | 152 | { 153 | "level": [ 154 | "########", 155 | "#.#....#", 156 | "#.#.##.#", 157 | "#.#..#.#", 158 | "#.##.#.#", 159 | "#....#x#", 160 | "########" 161 | ], 162 | "start_position": {"x": 1, "y": 1}, 163 | "coins": [ 164 | { 165 | "position": {"x": 1, "y": 5}, 166 | "value": 100 167 | } 168 | ] 169 | } 170 | 171 | Now you can use the following phrase to load the level: 172 | 173 | .. code:: 174 | 175 | import json 176 | 177 | ... 178 | 179 | level_data = json.load(open("level_01.json")) 180 | start_level(dungeon, **level_data) 181 | 182 | 183 | Step 7: More Levels 184 | ------------------- 185 | 186 | Edit another level in a new JSON file. 187 | See if you can switch between levels by changing the file name. 188 | 189 | .. note:: 190 | 191 | Be aware that JSON only understands double quotes around strings. 192 | Also, there must not be any spare commas at the end of a dictionary. 193 | -------------------------------------------------------------------------------- /functions/scope.rst: -------------------------------------------------------------------------------- 1 | Variable Scope 2 | ============== 3 | 4 | Guess the value of the variable ‘a’ in the three example programs. 5 | Check your guess by running the code. 6 | 7 | Example 1: 8 | ^^^^^^^^^^ 9 | 10 | .. code:: python3 11 | 12 | def addition(number1, number2): 13 | a = number1 + number2 14 | return a 15 | 16 | a = 3 17 | b = 5 18 | c = addition(a, b) 19 | print(a) 20 | 21 | Example 2: 22 | ^^^^^^^^^^ 23 | 24 | .. code:: python3 25 | 26 | def addition(number1, number2): 27 | global a 28 | a = number1 + number2 29 | return a 30 | 31 | a = 3 32 | b = 5 33 | c = addition(a, b) 34 | print(a) 35 | 36 | Example 3: 37 | ^^^^^^^^^^ 38 | 39 | .. code:: python3 40 | 41 | def addition(number1, number2): 42 | a = number1 + number2 43 | global a 44 | return a 45 | 46 | a = 3 47 | b = 5 48 | c = addition(a, b) 49 | print(a) 50 | -------------------------------------------------------------------------------- /getting_started/create_repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/getting_started/create_repo.png -------------------------------------------------------------------------------- /getting_started/git_dialog.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/getting_started/git_dialog.png -------------------------------------------------------------------------------- /getting_started/git_repo.rst: -------------------------------------------------------------------------------- 1 | Set up a GitHub Repository 2 | ========================== 3 | 4 | Version Control is in my opinion the single most important tool of 5 | modern software development. Most of the time, using Version Control 6 | means using **git**. In this recipe, you will set up your own project 7 | repository on `GitHub `__. 8 | 9 | I won’t be describing here what git is or how to use it. There are lots 10 | of high quality tutorials for it. If you have never used git before or 11 | need a refresher, check out the `Introduction to Git and 12 | GitHub `__ by Jim 13 | Anderson. 14 | 15 | Let’s go through the steps setting git/GitHub for your project: 16 | 17 | Step 1: Create a repository 18 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 19 | 20 | There are two ways to create a new repository: 21 | 22 | 1. using the GitHub website 23 | 2. using ``git init`` in the terminal 24 | 25 | I recommend starting on GitHub, because you can create a couple of 26 | useful files there right away. 27 | 28 | Log in to your account on `github.com `__ (or 29 | create one if you are there for the first time). Then, find the **button 30 | with a big plus (+)** in the top right corner and select **New 31 | Repository**. You should see a dialog where you can enter the name and 32 | description of your project: 33 | 34 | .. figure:: create_repo.png 35 | :alt: create a repository 36 | 37 | I recommend you use names in ``lowercase_with_underscores``. This may 38 | avoid random bugs on some operating systems. 39 | 40 | Step 2: Decide whether your project is public 41 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 42 | 43 | When you scroll down the creation dialog, there are several control 44 | boxes. The first one selects whether your project is visible to the rest 45 | of the world. There is a simple guideline: 46 | 47 | - If your project is work for a company or you feel you want to keep it 48 | confidential, choose **Private**. 49 | - If you would like to share it, choose **Public**. 50 | 51 | It is fine to have public projects that are *incomplete*, *simple* or 52 | *not practically useful*. Having a simple project that is cleaned up 53 | well may be a good advertisement. The Private/Public setting is easy to 54 | change later on as well. 55 | 56 | Step 3: Add a README file 57 | ~~~~~~~~~~~~~~~~~~~~~~~~~ 58 | 59 | This is a no-brainer. Tick this box. 60 | 61 | Step 4: Add a .gitignore file 62 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 63 | 64 | The ``.gitignore`` file prevents that many types of temporary or 65 | auto-generated files are added to your repository. It is another 66 | must-have. Choose **Python** from the dialog. 67 | 68 | Step 5: Choose a license 69 | ~~~~~~~~~~~~~~~~~~~~~~~~ 70 | 71 | GitHub offers a selection of time-tested open-source licenses. They are 72 | legally watertight, but differ in subtle aspects. For a hobby project, I 73 | recommend the **MIT License**. It roughly says: 74 | 75 | - other people may use your code for whatever they want 76 | - they have to keep the MIT License in 77 | - you cannot be held legally liable 78 | 79 | If you find out later that you need a different license, you as the 80 | author are allowed to change it. 81 | 82 | Step 6: Create the repo 83 | ~~~~~~~~~~~~~~~~~~~~~~~ 84 | 85 | Now you are ready to go. The dialog should look similar to this: 86 | 87 | .. figure:: git_dialog.png 88 | :alt: Git setup dialog 89 | 90 | Press the **Create Repository** button. After a few seconds, you should 91 | see a page listing the files in your new project repo: 92 | 93 | :: 94 | 95 | .gitignore 96 | LICENSE 97 | README.md 98 | 99 | Step 7: Clone the repository 100 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 101 | 102 | Next, create a local working copy of your project. For that, you need to 103 | **clone** the repository. You need to have git installed on your 104 | computer. You find a `git installation guide on 105 | git-scm.com `__. 106 | 107 | To clone the repository, you need the address under the big button 108 | labeled **Code**: 109 | 110 | .. figure:: git_url.png 111 | :alt: Copy the URL 112 | 113 | Do the following: 114 | 115 | 1. Copy the address starting with ``git@github..`` from the GitHub page 116 | 2. Open a terminal on your computer 117 | 3. Use ``cd`` to navigate to the folder where you keep your projects 118 | 4. Type ``git clone`` followed by the address you just copied 119 | 120 | You should see a message similar to: 121 | 122 | :: 123 | 124 | kristian@mylaptop:~/projects$ git clone git@github.com:krother/dungeon_explorer.git 125 | Cloning into 'dungeon_explorer'... 126 | remote: Enumerating objects: 5, done. 127 | remote: Counting objects: 100% (5/5), done. 128 | remote: Compressing objects: 100% (4/4), done. 129 | Receiving objects: 100% (5/5), done. 130 | remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0 131 | 132 | There also should be a new folder: 133 | 134 | :: 135 | 136 | kristian@mylaptop:~/projects$ ls -la dungeon_explorer 137 | total 24 138 | drwxrwxr-x 3 kristian kristian 4096 Mai 28 11:33 . 139 | drwxrwxr-x 50 kristian kristian 4096 Mai 28 11:33 .. 140 | drwxrwxr-x 8 kristian kristian 4096 Mai 28 11:33 .git 141 | -rw-rw-r-- 1 kristian kristian 1799 Mai 28 11:33 .gitignore 142 | -rw-rw-r-- 1 kristian kristian 1072 Mai 28 11:33 LICENSE 143 | -rw-rw-r-- 1 kristian kristian 35 Mai 28 11:33 README.md 144 | 145 | Step 8: Add your code 146 | ~~~~~~~~~~~~~~~~~~~~~ 147 | 148 | Now you can start adding code to your repository. For instance you could 149 | add a prototype if you have one. The sequence of commands might look 150 | like this: 151 | 152 | :: 153 | 154 | cd dungeon_explorer/ 155 | cp ~/Desktop/prototype.py . 156 | git status 157 | git add prototype.py 158 | git commit -m "add a dungeon_explorer prototype" 159 | git push 160 | 161 | To exectute ``git push``, you may need to `Add SSH keys to your GitHub 162 | account `__. 163 | 164 | In the end, you should see the code of your prototype on your GitHub 165 | page. **Congratulations!** 166 | 167 | 168 | .. seealso:: 169 | 170 | - `Git Introduction `__ 171 | - `Try GitHub - Online-Tutorial `__ 172 | - `Pro Git `__ – the book by Scott Chacon 173 | -------------------------------------------------------------------------------- /getting_started/git_url.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/getting_started/git_url.png -------------------------------------------------------------------------------- /getting_started/structure.rst: -------------------------------------------------------------------------------- 1 | Create a Folder Structure 2 | ========================= 3 | 4 | A small but important part of a project is creating folders for your 5 | code. In the Python world, there is a standard structure that you will 6 | find in many other projects. The next few steps let you create one. 7 | 8 | Step 1: A package folder 9 | ~~~~~~~~~~~~~~~~~~~~~~~~ 10 | 11 | First, you need a place for the package you want to write. A **Python 12 | package** is simply a folder that contains ``.py`` files. Create a 13 | folder ``dungeon_explorer`` inside your repository. On the bash terminal, you would 14 | use 15 | 16 | :: 17 | 18 | mkdir dungeon_explorer 19 | 20 | If your project folder (with the git repo) is also called ``dungeon_explorer``, 21 | you may want to rename the project folder to something else like ``dungeon_project`` or similar. 22 | Having two folders called ``dungeon_explorer`` 23 | inside each other could lead to strange import bugs later 24 | 25 | -------------- 26 | 27 | Step 2: A folder for tests 28 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ 29 | 30 | You will also want to have a place where you add test code later. Name 31 | that folder ``tests/``. We will leave it empty for now. 32 | 33 | :: 34 | 35 | mkdir tests 36 | 37 | -------------- 38 | 39 | Step 3: Create a Python module 40 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 41 | 42 | You may want to create a Python module (a ``.py`` file) to make sure 43 | everything is set up correctly. Create a file ``game.py`` inside the 44 | ``dungeon_explorer/`` folder. Add a placeholder function to it: 45 | 46 | :: 47 | 48 | def play(): 49 | print('this is the Dungeon Explorer game') 50 | 51 | Now start Python in your main project folder (above the package) through 52 | the terminal. **It is important that you start Python in your project 53 | folder. It will probably not work from your IDE at this point.** The 54 | code that you want to get running is: 55 | 56 | :: 57 | 58 | from dungeon_explorer.game import play 59 | 60 | play() 61 | 62 | You should see the message from the print statement. 63 | 64 | -------------- 65 | 66 | Step 4: main Python file 67 | ~~~~~~~~~~~~~~~~~~~~~~~~ 68 | 69 | Importing the ``play()`` function to play the game is a bit 70 | inconvenient. Let’s create a shortcut. Create a file named 71 | ``__main__.py`` (with double underscores on both ends) in the package 72 | folder that contains the following code: 73 | 74 | :: 75 | 76 | from game import play 77 | 78 | play() 79 | 80 | Now it should be possible to start the game by typing: 81 | 82 | :: 83 | 84 | python dungeon_explorer 85 | 86 | -------------- 87 | 88 | Summary 89 | ~~~~~~~ 90 | 91 | At this point, your project folder should contain: 92 | 93 | :: 94 | 95 | LICENSE 96 | prototype.py 97 | README.md 98 | dungeon_explorer/ 99 | game.py 100 | __main__.py 101 | tests/ 102 | 103 | 104 | .. seealso:: 105 | 106 | You find detailed info on importing stuff in 107 | `Python Modules and Packages on realpython.com `__ 108 | -------------------------------------------------------------------------------- /getting_started/virtualenv.rst: -------------------------------------------------------------------------------- 1 | Virtual Environments 2 | ==================== 3 | 4 | When developing software, you often need a specific combination of 5 | Python libraries. Sometimes this is difficult, because you require a 6 | specific version of a library, want to test your program on multiple 7 | Python versions, or simply need to develop your program further, while a 8 | stable version is installed on the same machine. In these cases, 9 | **virtual environments** come to the rescue. 10 | 11 | -------------- 12 | 13 | What is a virtual environment? 14 | ------------------------------ 15 | 16 | A virtual environment manages multiple parallel installations of Python 17 | interpreters and libraries, so that you can switch between them. The 18 | virtual environment consists of a folder per project, in which Python 19 | libraries for that project are installed. 20 | 21 | -------------- 22 | 23 | How to install a virtual environment? 24 | ------------------------------------- 25 | 26 | There are many Python tools to manage virtual environments: venv, 27 | virtualenv, Pipenv and Poetry. A beginner-friendly tool is to use 28 | **conda**. If you haven’t installed Anaconda already, you can find the 29 | **Miniconda installer** at https://conda.io/miniconda.html. 30 | 31 | -------------- 32 | 33 | How to set up a project with conda? 34 | ----------------------------------- 35 | 36 | Once the installer finishes and you open a new terminal, you should see 37 | ``(base)`` before the prompt: 38 | 39 | :: 40 | 41 | (base) ada@adas_laptop:~$ 42 | 43 | This means you are in an virtual environment called *“base”*. 44 | 45 | Let’s create a new one for a project called **dungeon_explorer**, specifying a 46 | Python version: 47 | 48 | :: 49 | 50 | conda create -n dungeon_explorer python=3.11 51 | 52 | Behind the scenes **conda** creates a new subdirectory. This is where 53 | libraries for your project will be stored. There are also scripts to 54 | activate the environment. 55 | 56 | -------------- 57 | 58 | How to work with an environment 59 | ------------------------------- 60 | 61 | To start working with your project, type: 62 | 63 | :: 64 | 65 | conda activate dungeon_explorer 66 | 67 | You should see *(dungeon_explorer)* appearing before your prompt. Now, whenever you 68 | use *pip* to install something, it will be installed only for 69 | *myproject*. 70 | 71 | Now check which libraries you have installed: 72 | 73 | :: 74 | 75 | pip freeze 76 | 77 | You can install additional libraries with ``pip`` or ``conda``. 78 | For instance, you need to install OpenCV even if you already installed it in your base environment: 79 | 80 | :: 81 | 82 | conda install opencv-python 83 | 84 | When you want to switch back to the base environment, type: 85 | 86 | :: 87 | 88 | conda activate base 89 | 90 | The virtual environment is specific for a terminal session. Thus, you 91 | can work on as many projects simultaneously as you have terminals open. 92 | -------------------------------------------------------------------------------- /hall_of_fame.rst: -------------------------------------------------------------------------------- 1 | 2 | Hall of Fame 3 | ============ 4 | 5 | Game projects completed by course participants 6 | ---------------------------------------------- 7 | 8 | - `MantiGo `__ – a frantic quest for fast food at Kottbusser Tor by Naive Bayleaves 9 | - `Rock-Paper-Scissors `__ – by Shreyaasri Prakash 10 | - `Breakout `__ – by Alexander Rykov and Chen-Yu Liu 11 | - `Wild West `__ – by Sultan Mirzoev 12 | - `Minensuche `__ – minesweeper clone by Benny Henning 13 | - `Mondrian 2.0 `__ – an image generator by Christin Jacobsen 14 | - `Games in C++ `__ – project highlights from my 2016/2017 C++ course 15 | 16 | 17 | Game projects I wrote myself 18 | ---------------------------- 19 | 20 | .. toctree:: 21 | :maxdepth: 1 22 | 23 | examples/breakout.rst 24 | examples/tetris.rst 25 | 26 | - `Pandas go to Space `__ - a space traveling adventure with a web frontend 27 | - `Snake `__ - a classic snake game 28 | -------------------------------------------------------------------------------- /index.rst: -------------------------------------------------------------------------------- 1 | Advanced Python 2 | =============== 3 | 4 | This coursebook is for you if you want to write programs with more than 1000 lines. 5 | 6 | You feel comfortable with writing basic Python code, but have realized that creating a 7 | piece of software is more complex. You are facing questions like: 8 | 9 | - How to organize code into functions, classes and modules? 10 | - How to clean up my code? 11 | - How to make sure my program works? 12 | - How to keep the program running over time? 13 | - How to install my program on multiple computers? 14 | 15 | Below you find development tools and techniques that help you to write 16 | programs that get the job done and don’t fall apart. 17 | 18 | Developing Software 19 | ------------------- 20 | 21 | .. toctree:: 22 | :maxdepth: 1 23 | 24 | software_engineering/README.rst 25 | software_engineering/prototype.rst 26 | software_engineering/code_review.rst 27 | 28 | Setting up a Python Project 29 | --------------------------- 30 | 31 | .. toctree:: 32 | :maxdepth: 1 33 | 34 | getting_started/virtualenv.rst 35 | getting_started/git_repo.rst 36 | getting_started/structure.rst 37 | 38 | Writing Automated Tests 39 | ----------------------- 40 | 41 | .. toctree:: 42 | :maxdepth: 1 43 | 44 | testing/facade.rst 45 | testing/unit_test.rst 46 | 47 | Functions 48 | --------- 49 | 50 | .. toctree:: 51 | :maxdepth: 1 52 | 53 | functions/levels.rst 54 | functions/function_parameters.rst 55 | functions/scope.rst 56 | functions/generators.rst 57 | functions/functools.rst 58 | functions/decorators.rst 59 | 60 | Shortcuts 61 | --------- 62 | 63 | .. toctree:: 64 | :maxdepth: 1 65 | 66 | shortcuts/collections.rst 67 | shortcuts/comprehensions.rst 68 | shortcuts/enums.rst 69 | 70 | Structuring Programs 71 | -------------------- 72 | 73 | .. toctree:: 74 | :maxdepth: 1 75 | 76 | structure/main_block.rst 77 | structure/commandline_args.rst 78 | structure/modules.rst 79 | structure/namespaces.rst 80 | 81 | Object-Oriented Programming 82 | --------------------------- 83 | 84 | .. toctree:: 85 | :maxdepth: 1 86 | 87 | classes/classes.rst 88 | classes/inheritance.rst 89 | classes/composition.rst 90 | classes/class_diagram.md 91 | classes/operator_overloading.rst 92 | classes/abc.rst 93 | classes/decorator_class.rst 94 | classes/metaclasses.rst 95 | 96 | Software Quality 97 | ---------------- 98 | 99 | .. toctree:: 100 | :maxdepth: 1 101 | 102 | quality/continuous_integration.rst 103 | quality/ 104 | 105 | Error Handling 106 | -------------- 107 | 108 | .. toctree:: 109 | :maxdepth: 1 110 | 111 | error_handling/debugging.rst 112 | error_handling/interactive_debugger.rst 113 | error_handling/exceptions.rst 114 | error_handling/warnings.rst 115 | error_handling/logging.rst 116 | 117 | Performance Optimization 118 | ------------------------ 119 | 120 | .. toctree:: 121 | :maxdepth: 1 122 | 123 | performance/profiling.rst 124 | concurrency/README.rst 125 | 126 | Challenges 127 | ---------- 128 | 129 | .. toctree:: 130 | :maxdepth: 1 131 | 132 | challenges/factorial.rst 133 | challenges/sorting.rst 134 | challenges/dice/dice.rst 135 | challenges/tennis.rst 136 | challenges/memory/memory.rst 137 | challenges/magic_square.rst 138 | challenges/josephus.rst 139 | challenges/binary_search.rst 140 | challenges/tree_traversal.rst 141 | challenges/maze.rst 142 | challenges/backpack_problem.rst 143 | challenges/chained_list.rst 144 | challenges/tsp.rst 145 | challenges/blockchain.rst 146 | challenges/metaclass.rst 147 | 148 | Coding Project 149 | -------------- 150 | 151 | .. toctree:: 152 | :maxdepth: 1 153 | 154 | project.rst 155 | hall_of_fame.rst 156 | 157 | 158 | Appendix 159 | --------- 160 | 161 | .. toctree:: 162 | :maxdepth: 1 163 | 164 | links.rst 165 | 166 | .. topic:: Source 167 | 168 | `github.com/krother/advanced_python `__ 169 | 170 | 171 | .. topic:: License 172 | 173 | © 2023 `Kristian Rother `__ 174 | 175 | Usable under the conditions of the MIT License. 176 | -------------------------------------------------------------------------------- /introspection.rst: -------------------------------------------------------------------------------- 1 | Introspection 2 | ============= 3 | 4 | Introspection is a feature of Python by which you can examine objects 5 | (including variables, functions, classes, modules) inside a running 6 | Python environment (a program or shell session). 7 | 8 | Exploring the namespace 9 | ----------------------- 10 | 11 | In Python all objects (variables, modules, classes, functions and your 12 | main program) are boxes called **namespaces**. You can imagine the 13 | namespace of an object as the data and functions inside than object. You 14 | can explore a namespace with the ``dir()`` function. 15 | 16 | Exploring the namespace of a variable 17 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 18 | 19 | With a string object, you see all the string methods: 20 | 21 | 22 | .. code:: python3 23 | 24 | s = "Emily" 25 | print(dir(s)) 26 | 27 | Exploring the namespace of a module: 28 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 29 | 30 | The same works for a module you import: 31 | 32 | 33 | .. code:: python3 34 | 35 | import time 36 | print(dir(time)) 37 | 38 | Listing the builtin functions: 39 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 40 | 41 | You also can view all builtin functions: 42 | 43 | 44 | .. code:: python3 45 | 46 | print(dir(__builtins__)) 47 | 48 | The help function 49 | ----------------- 50 | 51 | You can get context-sensitive help to functions, methods and classes 52 | with ``help()`` function. 53 | 54 | 55 | .. code:: python3 56 | 57 | import time 58 | print help(time.asctime) 59 | 60 | ``help()`` utilizes the triple-quoted comments called **docstrings**, so 61 | that documentation you write for your own functions is also availabel 62 | through ``help()``: 63 | 64 | Everything is an object 65 | ----------------------- 66 | 67 | One consequence of the dynamic typing is that Python can treat 68 | everything it manages technically in the same way. **Everything is an 69 | object** is a common phrase describing how Python works. There is no 70 | fundamental difference between a function and an integer. Many advanced 71 | features of Python are built on this concept. 72 | -------------------------------------------------------------------------------- /links.rst: -------------------------------------------------------------------------------- 1 | Links 2 | ===== 3 | 4 | https://github.com/pablito56/decorators.git 5 | 6 | Pythonic objects https://www.youtube.com/watch?v=k55d3ZUF3ZQ 7 | 8 | Mike Metaclasses https://www.youtube.com/watch?v=7PzeZQGVPKc 9 | 10 | https://github.com/cosmologicon/pywat 11 | 12 | http://pymust.watch/ 13 | 14 | Raymond on Concurrency: https://www.youtube.com/watch?v=Bv25Dwe84g0 15 | 16 | https://www.pythonsheets.com/ 17 | 18 | https://codesachin.wordpress.com/2016/04/03/a-practical-introduction-to-functional-programming-for-python-coders/ 19 | 20 | https://github.com/aosabook/500lines 21 | 22 | how-to-create-read-only-attributes-and-restrict-setting-attribute-values-on-object-in-python/ 23 | 24 | http://slides.com/meilitriantafyllidi/be-more-pythonic#/25 25 | 26 | https://www.codementor.io/moyosore/a-dive-into-python-closures-and-decorators-part-1-9mpr98pgr 27 | 28 | https://nbviewer.jupyter.org/github/austin-taylor/code-vault/blob/master/python_expert_notebook.ipynb 29 | -------------------------------------------------------------------------------- /performance/mandelbrot.py: -------------------------------------------------------------------------------- 1 | """ 2 | Drawing the Mandelbrot set 3 | 4 | based on R code by Myles Harrison 5 | http://www.everydayanalytics.ca 6 | 7 | original source of the Python code: 8 | https://github.com/krother/Python3_Package_Examples 9 | MIT License 10 | """ 11 | import numpy as np 12 | from PIL import Image 13 | 14 | def get_next_iter(z, c, index): 15 | """calculate the next generation of the entire matrix""" 16 | newz = [] 17 | for x in range(z.shape[0]): 18 | for y in range(z.shape[1]): 19 | if index[x, y]: 20 | newz.append(z[x, y] ** 2 + c[x, y]) 21 | else: 22 | newz.append(999) 23 | z = np.array(newz).reshape(z.shape) 24 | return z 25 | 26 | 27 | def calculate(z, k, c): 28 | index = z < 2 29 | z = get_next_iter(z, c, index) 30 | k[index] = k[index] + 1 31 | return z, k 32 | 33 | 34 | def draw_mandelbrot(xmin=-2, xmax=1.0, nx=500, 35 | ymin=-1.5, ymax=1.5, ny=500, 36 | n=100): 37 | x = np.linspace(xmin, xmax, nx) 38 | real = np.outer(x, np.ones(ny)) 39 | 40 | y = np.linspace(ymin, ymax, ny) 41 | imag = 1j * np.outer(np.ones(nx), y) 42 | 43 | c = real + imag 44 | 45 | z = np.zeros((nx, ny)) * 1j 46 | k = np.zeros((nx, ny)) 47 | 48 | for recursion in range(1, n): 49 | z, k = calculate(z, k, c) 50 | 51 | return k 52 | 53 | 54 | if __name__ == '__main__': 55 | mtx = draw_mandelbrot() 56 | mtx = 255 * mtx / mtx.max() 57 | mtx = mtx.astype(np.uint8) 58 | im = Image.fromarray(mtx, 'L') 59 | im.save('mandelbrot.png') 60 | -------------------------------------------------------------------------------- /performance/profiling.rst: -------------------------------------------------------------------------------- 1 | 2 | Measuring Performance 3 | ===================== 4 | 5 | Profiling Python Scripts 6 | ------------------------ 7 | 8 | Run the code in :download:`mandelbrot.py` and time its execution with `cProfile`, the Python profiler: 9 | 10 | .. code:: 11 | 12 | python -m cProfile -s cumtime mandelbrot.py > profile.txt 13 | 14 | Inspect the output and look for bottlenecks. 15 | 16 | Then insert the line: 17 | 18 | .. code:: python3 19 | 20 | z[index] = z[index] \*\* 2 + c[index] 21 | 22 | Re-run the profiling. 23 | 24 | .. hint:: 25 | 26 | cProfile also works inside a program: 27 | 28 | .. code:: python3 29 | 30 | import cProfile 31 | cProfile.run("[x for x in range(1500)]") 32 | 33 | Timing in Jupyter / IPython 34 | --------------------------- 35 | 36 | IPython (including Jupyter notebooks) has two magic commands for measuring execution time. 37 | 38 | .. code:: ipython3 39 | 40 | %time len(range(100000)) 41 | 42 | compare the output to 43 | 44 | .. code:: ipython3 45 | 46 | %timeit 47 | 48 | 49 | C Extensions 50 | ------------ 51 | 52 | Inspect how a Python-C interfacee looks like. 53 | Examine the code at `https://github.com/biopython/biopython/blob/master/Bio/PDB/ `__ 54 | 55 | In particular, inspect the files: 56 | 57 | - NeighborSearch.py 58 | - kdtrees.c 59 | - setup.py (in the main directory) 60 | - README.md (in the main directory) 61 | -------------------------------------------------------------------------------- /pip.rst: -------------------------------------------------------------------------------- 1 | Installing Packages with pip 2 | ============================ 3 | 4 | ``pip`` is a tool to install Python packages and resolve dependencies 5 | automatically. This section lists a couple of things you can do with 6 | ``pip``: 7 | 8 | Install a package 9 | ~~~~~~~~~~~~~~~~~ 10 | 11 | To install a Python package, call ``pip`` with the package name: 12 | 13 | :: 14 | 15 | pip install pandas 16 | 17 | You can specify the exact version of a package: 18 | 19 | :: 20 | 21 | pip install pandas==0.25.0 22 | 23 | -------------- 24 | 25 | Install many packages 26 | ~~~~~~~~~~~~~~~~~~~~~ 27 | 28 | First, create a file ``requirements.txt`` in your project directory. The 29 | file should look similar to this: 30 | 31 | :: 32 | 33 | pandas==0.25 34 | numpy>=1.17 35 | scikit-learn 36 | requests 37 | 38 | Second, ask ``pip`` to install everything: 39 | 40 | :: 41 | 42 | pip -r requirements.txt 43 | 44 | -------------- 45 | 46 | Install from a git repo 47 | ~~~~~~~~~~~~~~~~~~~~~~~ 48 | 49 | If a repository has a ``setup.py`` file, you could install directly from 50 | git. This is useful to install branches, forks and other work in 51 | progress: 52 | 53 | :: 54 | 55 | pip install git+https://github.com/pandas-dev/pandas.git 56 | 57 | -------------- 58 | 59 | Install a package you are developing 60 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 61 | 62 | When developing, you might want to pip-install a working copy. This 63 | allows you to import your package (e.g. for testing). Changes to the 64 | code directly take effect in the installation. 65 | 66 | For the following to work, your project folder needs to have a 67 | ``setup.py``: 68 | 69 | :: 70 | 71 | pip install --editable . 72 | 73 | -------------- 74 | 75 | List all installed packages 76 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 77 | 78 | This one prints all packages you have installed and their versions: 79 | 80 | :: 81 | 82 | pip freeze 83 | 84 | To search for a pacakge, use ``grep``: 85 | 86 | :: 87 | 88 | pip freeze | grep pandas 89 | 90 | 91 | Where does pip store its files? 92 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 93 | 94 | Usually, packages are stored in the ``site_packages/`` folder. Where 95 | this one is depends on your distribution and your virtual environment. 96 | 97 | You might want to check your ``PYTHONPATH`` environment variable. To do 98 | so from Python, use: 99 | 100 | .. code:: python3 101 | 102 | import sys 103 | 104 | print(sys.path) 105 | 106 | -------------- 107 | 108 | .. seealso:: 109 | 110 | - The ``conda`` program (part of the Anaconda distribution) is often a viable alternative to pip 111 | - You find all installable packages on the `Python Package Index pypi.org `__ 112 | -------------------------------------------------------------------------------- /project.rst: -------------------------------------------------------------------------------- 1 | 2 | Project: Implement a Game 3 | ========================= 4 | 5 | .. topic:: Goal 6 | 7 | **Implement a computer game in Python.** 8 | 9 | Get together in a team of 2-3. 10 | Find an idea for a game that you would like to implement. 11 | Then implement it. 12 | 13 | Some ideas 14 | ---------- 15 | 16 | - a text adventure with pictures (see `Quest for the Dragon Egg `__) 17 | - Connect Four 18 | - a Candy Crush-like game 19 | - a Tetris-like game 20 | - Pac man/woman/non-binary 21 | 22 | .. hint:: 23 | 24 | A smaller idea implemented well is worth more than a large program that is left half-finished. 25 | 26 | Assessment Criteria 27 | ------------------- 28 | 29 | - the game is working 30 | - there is a git repository 31 | - there is a README file including installation instructions 32 | - there is a ``requirements.txt`` or ``pyproject.toml`` file 33 | - no functions is longer than 20 lines 34 | - there is at least one class with 3+ attributes and/or methods 35 | - there is a class diagram, sequence diagram, state diagram or other type of design diagram 36 | - there are 3+ automated tests that pass and check the code of the game 37 | - the project uses Continuous Integration 38 | - Python style checks with black pass 39 | - at least 3 functions contain type annotations 40 | - the game contains custom graphics 41 | - the game contains sound effects and/or music 42 | - great gameplay (up to 2 points) 43 | 44 | .. warning:: 45 | 46 | When submitting a project as a piece of academic work, you must make sure to reference: 47 | 48 | - sources of images, animations and music 49 | - projects the code is based upon 50 | - tutorials that were directly used 51 | - large language models (LLMs) that were used to generate artwork or code 52 | 53 | Failure to comply may consist a plagiarism case and may violate the rights of copyright holders. 54 | -------------------------------------------------------------------------------- /quality/continuous_integration.rst: -------------------------------------------------------------------------------- 1 | Continuous Integration 2 | ====================== 3 | 4 | In this exercise you will create a simple **Continuous Integration** 5 | workflow. 6 | 7 | What CI does 8 | ------------ 9 | 10 | When anyone pushes to a **remote git repository**, the git server should: 11 | 12 | 1. create an empty virtual computer 13 | 2. clone the repository 14 | 3. install all dependencies 15 | 4. run automated tests 16 | 5. report whether the tests succeed or fail 17 | 18 | Step 1: Preparations 19 | -------------------- 20 | 21 | You need: 22 | 23 | - a GitHub repo for your project 24 | - a ``requirements.txt`` file 25 | - at least one automated test 26 | 27 | Step 2: Create a workflow 28 | ------------------------- 29 | 30 | GitHub Actions needs instructions how to install the program. 31 | 32 | Create a folder ``.github/workflows/``. Place a text file ``run_tests.yml`` 33 | into that folder containing the following: 34 | 35 | :: 36 | 37 | name: run_tests 38 | 39 | on: 40 | push: 41 | branches: [ main ] 42 | 43 | jobs: 44 | build: 45 | 46 | runs-on: ubuntu-latest 47 | 48 | steps: 49 | - uses: actions/checkout@v2 50 | - name: Set up Python 3.11 51 | uses: actions/setup-python@v2 52 | with: 53 | python-version: 3.11 54 | - name: Install dependencies 55 | run: | 56 | python -m pip install --upgrade pip 57 | pip install -r requirements.txt 58 | - name: Test with pytest 59 | run: | 60 | pytest 61 | 62 | 63 | Step 3: Requirements 64 | -------------------- 65 | 66 | A best practice is to have a separate file for **development requirements**. 67 | 68 | - Create a separate file ``dev_requirements.txt`` 69 | - Add ``pytest`` to it 70 | - Add a **pip install** line to the yml file. 71 | 72 | Step 4: Push 73 | ------------ 74 | 75 | Commit and push the changes. 76 | 77 | Step 5: Observe 78 | --------------- 79 | 80 | - Go to GitHub. Check the **Actions** tab. 81 | - Watch the output of your project building. 82 | - Also check your mailbox. 83 | 84 | Step 6: Badge 85 | ------------- 86 | 87 | Copy the following code into your ``README.md`` file: 88 | 89 | :: 90 | 91 | ![Python application](https://github.com/USER/REPO/workflows/run_tests/badge.svg) 92 | 93 | Replace **USER** and **REPO** by the data of your project. **run_tests** 94 | is the name from the workflow file. 95 | 96 | You should see a red or green badge in the README that updates itself. 97 | 98 | .. topic:: Authors 99 | 100 | Malte Bonart participated in the writing of this chapter. 101 | -------------------------------------------------------------------------------- /quality/packaging.rst: -------------------------------------------------------------------------------- 1 | Create a pip-installable Package 2 | ================================ 3 | 4 | Once a python package (a folder with ``.py`` files), you may want to 5 | make it available to other programs – and people. Making your code 6 | pip-installable can be done by adding an extra configuration files 7 | called ``setup.py``. 8 | 9 | Assume your project folder contains: 10 | 11 | :: 12 | 13 | dungeon_explorer/ - module folder you want to import 14 | tests/ - the test code for pytest 15 | .git/ - the commit history (managed by git) 16 | README.md - documentation 17 | LICENSE - legal info 18 | setup.py - used by pip (see below) 19 | .gitignore - choose one on Github 20 | 21 | The Project Folder 22 | ------------------ 23 | 24 | Your project folder should contain a sub-directory with a name in 25 | **lowercase_with_underscores**. This sub-directory is your python 26 | package! Add your own Python files inside this package folder. The 27 | ``setup.py`` script will look for the source code there. 28 | 29 | The setup.py file 30 | ----------------- 31 | 32 | **setuptools** is a Python library that builds and installs Python 33 | packages. You may need to install it first: 34 | 35 | :: 36 | 37 | pip install setuptools 38 | 39 | In order to use setuptools, you need a file called ``setup.py`` that 40 | tells the installer what to install. You can use the following 41 | ``setup.py`` file as a starting point: 42 | 43 | :: 44 | 45 | from setuptools import setup 46 | import os 47 | 48 | def get_readme(): 49 | """returns the contents of the README file""" 50 | return open(os.path.join(os.path.dirname(__file__), "README.md")).read() 51 | 52 | setup( 53 | name="dungeon_explorer", # name used on PyPi 54 | version="0.0.1", # uses *semantic versioning* 55 | description="a simpl dungeon RPG", 56 | long_description=get_readme(), 57 | author="your_name", 58 | author_email="your@name.com", 59 | packages=["dungeon_explorer"], # the folder with .py modules 60 | url="https://github.com/...", 61 | license="MIT", 62 | classifiers=[ 63 | "Programming Language :: Python :: 3.10", 64 | "Programming Language :: Python :: 3.11", 65 | "Programming Language :: Python :: 3.12", 66 | ] 67 | ) 68 | 69 | Copy this code to a ``setup.py`` file in the top-level folder of your 70 | project and save it. 71 | 72 | Here is a `video explaining how setup.py 73 | works `__. 74 | 75 | -------------- 76 | 77 | Install your program 78 | -------------------- 79 | 80 | When developing a program, the first thing you want to do is to install 81 | your program in development mode. Go to the folder where the 82 | ``setup.py`` file is located and run the command: 83 | 84 | :: 85 | 86 | python setup.py develop 87 | 88 | OR 89 | 90 | pip install --editable . 91 | 92 | This makes your project available to the rest of your Python environment 93 | (Python creates a link to your project somewhere in the PYTHONPATH). Now 94 | you should be able to run from any other Python program: 95 | 96 | :: 97 | 98 | import dungeon_explorer 99 | 100 | In other words, you don’t actually need to be in your project folder to 101 | use your program. This is super convenient! You can use your package 102 | from anywhere as if it were an official library, like **numpy** or 103 | **pydantic**. You should also see your package in the output of 104 | ``pip list`` or ``pip freeze``. 105 | 106 | This method has the advantage that you can still edit your code, and the 107 | next time you import the library again. 108 | 109 | **WARNING:** for the re-import to work, you need to restart the Python 110 | interpreter. Just executing the import twice does not work. 111 | 112 | -------------- 113 | 114 | Installation on other machines 115 | ------------------------------ 116 | 117 | If you want to use your library but not edit it (e.g. in a production 118 | environment), you may want to copy it to where Python stores all the 119 | other packages. This can be done with another one-liner. 120 | 121 | :: 122 | 123 | python setup.py install 124 | 125 | OR 126 | 127 | pip install . 128 | 129 | The files are copied to a folder called ``site-packages/`` . The 130 | location of it depends on your operating system and Python distribution. 131 | 132 | -------------- 133 | 134 | Installation from GitHub 135 | ------------------------ 136 | 137 | If you have a ``setup.py``, you can pip-install your package directly 138 | from GitHub: 139 | 140 | :: 141 | 142 | pip install 143 | 144 | -------------- 145 | 146 | Creating a distribution 147 | ----------------------- 148 | 149 | If you want to package all files of your projects into an archive, you 150 | can do this with: 151 | 152 | :: 153 | 154 | python setup.py sdist 155 | 156 | This creates a ``dist/`` folder with a ``.tar.gz`` file that you can 157 | move around easily. 158 | 159 | -------------- 160 | 161 | Further Reading 162 | --------------- 163 | 164 | If you would like to upload your program to PyPi, so that anyone can 165 | install it with 166 | 167 | :: 168 | 169 | pip install dungeon_explorer 170 | 171 | you need to follow a few more steps. This is not difficult but a bit 172 | tedious. We recommend the official 173 | ``Packaging Python Projects Tutorial ``\ \__. 174 | 175 | .. topic:: Authors 176 | 177 | This guide was written together with Paul Wlodkowski and Malte Bonart. 178 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | sphinx 2 | sphinx-design 3 | sphinx-copybutton 4 | myst-parser 5 | sphinxcontrib-svg2pdfconverter[CairoSVG] 6 | furo 7 | -------------------------------------------------------------------------------- /shortcuts/collections.rst: -------------------------------------------------------------------------------- 1 | Collections 2 | =========== 3 | 4 | The ``collections`` module contains a few very useful dictionaries: 5 | 6 | =========== ===================================== 7 | type description 8 | =========== ===================================== 9 | Counter dictionary for counting things 10 | defaultdict dictionary with a preset value 11 | OrderedDict dictionary preserving insertion order 12 | =========== ===================================== 13 | 14 | Counter 15 | ------- 16 | 17 | The ``Counter`` is a special dictionary for counting things. 18 | 19 | .. code:: python3 20 | 21 | from collections import Counter 22 | 23 | # generate 100 random names 24 | names = ["Adam", "Bea", "Charlie", "Danielle", 25 | "Eve", "Frantz", "Gustav", "Helena"] 26 | from random import choice 27 | data = [choice(names) for i in range(100)] 28 | 29 | You create a ``Counter`` by giving it an iterable: 30 | 31 | .. code:: python3 32 | 33 | c = Counter(data) 34 | 35 | The most important new method is ``most_common``. The rest works like a 36 | normal dictionary. 37 | 38 | .. code:: python3 39 | 40 | print(c) 41 | print(c.most_common(3)) 42 | print(c.get('Adam')) 43 | 44 | ---- 45 | 46 | Defaultdict 47 | ----------- 48 | 49 | A ``defaultdict`` inserts a default value for each key that is used the first time. 50 | It therefore never throws a ``KeyError`` when requesting a value. 51 | 52 | Creating a defaultdict 53 | ~~~~~~~~~~~~~~~~~~~~~~ 54 | 55 | A ``defaultdict`` is created with a function that creates the default 56 | valued. 57 | 58 | The following dictionary defaults to a zero integer: 59 | 60 | .. code:: python3 61 | 62 | from collections import defaultdict 63 | 64 | d = defaultdict(int) 65 | d['Adam'] = 33 66 | d['Eve'] = 55 67 | 68 | print(d['Adam']) # -> 33 like a normal dict 69 | print(d['Guido']) # -> 0 because it's a new key 70 | 71 | Most times a ``defaultdict`` is initialized with a data, but any Python 72 | function works. Try: 73 | 74 | .. code:: python3 75 | 76 | from random import random 77 | 78 | d = defaultdict(random) 79 | d['dummy'] 80 | 81 | Collecting lists 82 | ~~~~~~~~~~~~~~~~ 83 | 84 | A common use case is to create a dict of lists. This becomes very easy 85 | with a ``defaultdict``. 86 | 87 | The following example sorts names by their initial: 88 | 89 | .. code:: python3 90 | 91 | names = [ 92 | "Athene", "Ada", "Hypathia", 93 | "Anna", "Helena", "Thetis" 94 | ] 95 | 96 | d = defaultdict(list) 97 | 98 | for n in names: 99 | key = n[0] 100 | d[key].append(n) 101 | 102 | This results in the following data: 103 | 104 | .. code:: python3 105 | 106 | defaultdict(list, 107 | {'A': ['Athene', 'Ada', 'Anna'], 108 | 'H': ['Hypathia', 'Helena'], 109 | 'T': ['Thetis']}) 110 | 111 | ---- 112 | 113 | OrderedDict 114 | ----------- 115 | 116 | OrderedDict is a special kind of dictionary that preserves the insertion 117 | order. 118 | 119 | .. code:: python3 120 | 121 | from collections import OrderedDict 122 | 123 | od = OrderedDict() 124 | 125 | names = ["Adam", "Bea", "Charlie", "Danielle", "Eve", "Frantz", "Gustav", "Helena"] 126 | for i, name in enumerate(names): 127 | od[i] = name 128 | 129 | With an OrderedDict, you have a guarantee that the output of the 130 | following command is always the same (which you don’t have with a normal 131 | dictionary): 132 | 133 | .. code:: python3 134 | 135 | print(od) 136 | 137 | You have an additional method for changing the order: 138 | 139 | .. code:: python3 140 | 141 | od.move_to_end(2) 142 | print(od) 143 | -------------------------------------------------------------------------------- /shortcuts/comprehensions.rst: -------------------------------------------------------------------------------- 1 | Comprehensions 2 | ============== 3 | 4 | **Comprehensions condense for loops into a one-line expression.** 5 | 6 | List Comprehensions 7 | ------------------- 8 | 9 | A list comprehension creates a list: 10 | 11 | .. code:: python3 12 | 13 | squares = [x ** 2 for x in range(10)] 14 | 15 | Produces the same result as: 16 | 17 | .. code:: python3 18 | 19 | squares = [] 20 | for x in range(10): 21 | squares.append(x ** 2) 22 | 23 | Conditionals 24 | ~~~~~~~~~~~~ 25 | 26 | The comprehension may contain an ``if`` clause to filter the list: 27 | 28 | .. code:: python3 29 | 30 | squares = [x ** 2 for x in range(10) if x % 3 == 0] 31 | 32 | Note that you can also place an ``if`` on the other side of the ``for`` 33 | as a ternary expression. This leads to a different result: 34 | 35 | .. code:: python3 36 | 37 | squares = [x ** 2 if x % 3 == 0 else -1 for x in range(10)] 38 | 39 | Nested Comprehensions 40 | ~~~~~~~~~~~~~~~~~~~~~ 41 | 42 | It is perfectly fine to place one comprehension inside another. The 43 | following code creates a 5x5 matrix: 44 | 45 | .. code:: python3 46 | 47 | [[x * y for x in range(5)] for y in range(5)] 48 | 49 | Note that you can concatenate the ``for`` statements without the extra 50 | brackets. In that case the result is a flat list: 51 | 52 | .. code:: python3 53 | 54 | [x * y for x in range(5) for y in range(5)] 55 | 56 | The join pattern 57 | ~~~~~~~~~~~~~~~~ 58 | 59 | Comprehensions that produce a list of strings are often found together 60 | with ``join()``, e.g. to format text output: 61 | 62 | .. code:: python3 63 | 64 | ';'.join([char.upper() for char in 'abcde']) 65 | 66 | ---- 67 | 68 | Dict comprehensions 69 | ------------------- 70 | 71 | This variant produces dictionaries: 72 | 73 | .. code:: python3 74 | 75 | ascii_table = {x: chr(x) for x in range(65, 91)} 76 | 77 | ---- 78 | 79 | Set comprehensions 80 | ------------------ 81 | 82 | The same with a set: 83 | 84 | .. code:: python3 85 | 86 | unique = {x.upper() for x in 'Hello World'} 87 | 88 | ---- 89 | 90 | Generator expressions 91 | --------------------- 92 | 93 | Finally, you can use a comprehension to define generators: 94 | 95 | .. code:: python3 96 | 97 | squares = (x ** 2 for x in range(1_000_000_000_000_000)) 98 | print(next(squares)) 99 | 100 | ---- 101 | 102 | 103 | Recap: Number Comprehensions 104 | ---------------------------- 105 | 106 | Create the following data with one-liners: 107 | 108 | .. code:: python3 109 | 110 | # 1. integers 111 | a = ... 112 | assert a == [1, 2, 3, 4, 5, 6] 113 | 114 | # 2. squares 115 | a = ... 116 | assert a == [1, 4, 9, 16, 25, 36] 117 | 118 | # 3. fractions 119 | a = ... 120 | assert a == [1.0, 0.5, 0.33, 0.25, 0.2, 0.17, 0.14] 121 | 122 | # 4. filtering 123 | a = range(100) 124 | b = ... a ... 125 | assert b == [11, 22, 33, 44, 55, 66, 77, 88, 99] 126 | 127 | # 5. dictionary of squares 128 | a = ... 129 | assert a == {1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36} 130 | 131 | # 6. split 132 | b = ... 133 | assert b == [[1, 2], [3, 4], [5, 6]] 134 | 135 | # 7. concatenate 136 | c = ... b ... 137 | assert c == [1, 2, 3, 4, 5, 6] 138 | -------------------------------------------------------------------------------- /shortcuts/enums.rst: -------------------------------------------------------------------------------- 1 | Enumerations 2 | ============ 3 | 4 | **Enumerations or Enums** are a way to manage strictly defined states. 5 | They avoid ambiguities and accidental bugs when using standard data 6 | types. 7 | 8 | Defining an Enum 9 | ---------------- 10 | 11 | An ``Enum`` is defined by calling the ``Enum`` as a function with the 12 | possible states as as a list: 13 | 14 | .. code:: python3 15 | 16 | TrafficLight = Enum('TrafficLight', ["RED", "AMBER", "GREEN"]) 17 | 18 | Alternatively you can define an ``Enum`` as a subclass, with the 19 | possible states as attributes: 20 | 21 | .. code:: python3 22 | 23 | from enum import Enum 24 | 25 | class TrafficLight(Enum): 26 | RED = 1 27 | AMBER = 2 28 | GREEN = 3 29 | 30 | ---- 31 | 32 | Using an Enum 33 | ------------- 34 | 35 | To use the states in an ``Enum``, assign it to a state variable: 36 | 37 | .. code:: python3 38 | 39 | a = TrafficLight.RED 40 | 41 | Then compare values to another state: 42 | 43 | .. code:: python3 44 | 45 | print(a is TrafficLight.GREEN) 46 | print(a TrafficLight.RED) 47 | 48 | ---- 49 | 50 | Inspecting Enums 51 | ---------------- 52 | 53 | Also try: 54 | 55 | .. code:: python3 56 | 57 | print(TrafficLight.RED.name) 58 | print(TrafficLight.RED == 1) 59 | 60 | ---- 61 | 62 | Iterating over an Enums 63 | ----------------------- 64 | 65 | iterating over an Enum gives you all states: 66 | 67 | .. code:: python3 68 | 69 | for x in TrafficLight: 70 | print(x) 71 | -------------------------------------------------------------------------------- /software_engineering/README.rst: -------------------------------------------------------------------------------- 1 | 2 | The Dungeon Explorer Game 3 | ========================= 4 | 5 | .. image:: title.png 6 | 7 | *image generated with dreamstudio* 8 | 9 | In the following chapters, we will write a small game called **Dungeon Explorer**. 10 | Writing a game is a great way to both become better at programming and to exemplify a systematic approach to programming. 11 | 12 | What Software Development is about 13 | ---------------------------------- 14 | 15 | .. image:: software_engineering.png 16 | 17 | Software development is more than writing code. 18 | While most courses and online resources focus very much on the process of writing code, 19 | it is easy to forget that several things that are necessary before the code can be written: 20 | 21 | * an **idea** that provides a reason for the program to exist (*"why"*) 22 | * **requirements** that describe what the program should do (*"what"*) 23 | * a **design** that describes core mechanics of the program (*"how"*) 24 | * the actual **implementation** 25 | 26 | When moving from the idea towards code, one gradually translates the informal, prose description of the problem into a *formal language*. Much of it requires gaining a deeper understanding of a task that is unknown at first. This is what much of a software developers work is really about. 27 | 28 | In a simple program, like many learning examples and coding challenges, the first three steps are nonexistent or at least very easy. 29 | In any real-world software project, the first three steps are often more difficult than the implementation. 30 | When writing requirements or designing a program, the software developer does not only have to find a path that brings the original idea to life, but also anticipate what forces the software will have to adapt to in the future. 31 | 32 | Let's look at an example of how the first three steps would look for a **Dungeon Explorer** game: 33 | 34 | The Idea 35 | -------- 36 | 37 | *"In the Dungeon Explorer game, an adventurer roams a dungeon, solves puzzles, avoids traps and finds treasures."* 38 | 39 | The Requirements 40 | ---------------- 41 | 42 | Here is a short example of how the idea could be fleshed out more: 43 | 44 | 1. User Interface 45 | ~~~~~~~~~~~~~~~~~ 46 | 47 | === ============================================================== 48 | # description 49 | === ============================================================== 50 | 1.1 The dungeon is seen from the top as a 2D grid 51 | 1.2 The dungeon is drawn from square graphic tiles 52 | 1.3 The adventurer is controlled by the player by pressing keys 53 | === ============================================================== 54 | 55 | 2. Dungeon Elements 56 | ~~~~~~~~~~~~~~~~~~~ 57 | 58 | === ============================================================== 59 | # description 60 | === ============================================================== 61 | 2.1 walls are impassable 62 | 2.2 an exit square leads to the next level 63 | 2.3 keys that are needed to open doors 64 | === ============================================================== 65 | 66 | 3. Traps 67 | ~~~~~~~~ 68 | 69 | === ============================================================== 70 | # description 71 | === ============================================================== 72 | 3.1 if the player falls into a pit, they die 73 | 3.2 if the player steps into green slime, they lose health 74 | 3.3 there are fireballs that travel in straight lines 75 | === ============================================================== 76 | 77 | The Design 78 | ---------- 79 | 80 | Two key elements of the game deserve to be described better. 81 | First, the grid that is used for the dungeon is based on *integer x/y coordinates*: 82 | 83 | .. image:: grid_model.png 84 | 85 | Second, the following **flowchart** describes how the program actually works: 86 | 87 | .. image:: flowchart.png 88 | 89 | .. note:: 90 | 91 | At this point, we have not decided what libraries or even what programming language to use. This is good, because the design makes the task more specific without constraining the developer. 92 | 93 | These are just two examples of design techniques. 94 | For a first implementation, see the next chapter. 95 | 96 | Reflection Questions 97 | -------------------- 98 | 99 | - which of the in the software development diagram could be automated with the help of a Large Language Model (LLM)? -------------------------------------------------------------------------------- /software_engineering/code_review.rst: -------------------------------------------------------------------------------- 1 | Code Reviews 2 | ============ 3 | 4 | A **Code review** means that another person reads your code. This could 5 | be: 6 | 7 | - a senior engineer 8 | - a programmer with similar experience 9 | - a junior developer 10 | 11 | All three provide complementary feedback that is useful in many ways. 12 | Besides discovering bugs, they also expose general design weaknesses 13 | (that might become bugs in the future) or simply learn you 14 | alternative/better ways to solve the problem. 15 | 16 | Because of that, many engineers see code reviews as the 17 | **number one technique to build high-quality software.** 18 | 19 | 20 | Exercise 1: Code Review 21 | ----------------------- 22 | 23 | Inspect the Python code in :download:`prototype_opencv.py`. 24 | Answer the following questions: 25 | 26 | * What libraries does the program use? 27 | * What variables does the program define? 28 | * What do the functions do? 29 | * Which parts of the code correspond to the flowchart? 30 | * How are the grid coordinates represented? 31 | * What would you like to know more about? 32 | 33 | Exercise 2: Modify 34 | ------------------ 35 | 36 | Extend the code such that: 37 | 38 | * the player can be moved up and down. 39 | * the player stops at the border of the screen 40 | * the player respawns at the left if they leave the grid on the right side 41 | * there is a "teleport" key that jumps to a random grid point 42 | 43 | 44 | .. topic:: What to look for in a code review? 45 | 46 | Here are a few things to pay attention to in a code review. 47 | The list is by far not exhaustive: 48 | 49 | - Does the module header explain understandably what the code does? 50 | - Are constants listed right after the import statements? 51 | - Is the code sufficiently documented? 52 | - Are the functions understandable? 53 | - Are the variable names descriptive 54 | - Is the code formatted in a consistent way? 55 | - Are there code duplications? 56 | - Is there dead code that does nothing? 57 | - Are there code sections that are unnecessarily long? 58 | - Are there while loops that could run forever? 59 | - Are there deeply nested code sections? 60 | - Do all functions have exactly one way to return data (either by 61 | return value OR by modifying an object, not both). 62 | -------------------------------------------------------------------------------- /software_engineering/flowchart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/flowchart.png -------------------------------------------------------------------------------- /software_engineering/grid_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/grid_model.png -------------------------------------------------------------------------------- /software_engineering/prototype.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/prototype.png -------------------------------------------------------------------------------- /software_engineering/prototype.rst: -------------------------------------------------------------------------------- 1 | The Prototype 2 | ============= 3 | 4 | Before attempting a costly clean implementation, you may want to 5 | check whether the project is feasible at all. You can do such a check by 6 | implementing a quick and dirty proof of concept: a **prototype**. 7 | The goal of a prototype is usually to reduce risks in a project. 8 | 9 | A prototype can answer questions like: 10 | 11 | - is my programming environment set up properly? 12 | - can we solve a particular algorithmic problem? 13 | - does a library do what we need? 14 | - is the algorithm/library fast enough? 15 | - what safety/security risks are there? 16 | - did we understand the customer correctly? 17 | 18 | A Prototype for Dungeon Explorer 19 | -------------------------------- 20 | 21 | Let's look at a prototype for a Dungeon Explorer game. 22 | In the program you move a graphical icon using the keyboard. 23 | The goal of this prototype is to prove that you can process keyboard input 24 | and draw 2D graphics. 25 | 26 | The prototype helps us to get rid of installation issues right away. 27 | 28 | Exercise 1: Save the code 29 | ------------------------- 30 | 31 | Save the code in :download:`prototype_opencv.py` to a Python file. 32 | 33 | Also unzip the tile images in :download:`tiles.zip` to the folder containing the Python file. 34 | 35 | Exercise 2: Install Dependencies 36 | -------------------------------- 37 | 38 | You need to install some libraries. 39 | Open a terminal (on Windows: Anaconda Prompt) and type: 40 | 41 | :: 42 | 43 | pip install opencv-python 44 | 45 | .. hint:: 46 | 47 | In case the installation, try the fallback code in :download:`prototype_curses.py`. 48 | That second prototype runs in the terminal and does not require OpenCV. 49 | See the source code for further info. 50 | 51 | 52 | Exercise 3: Execute the prototype 53 | --------------------------------- 54 | 55 | Change to the directory with the ``.py`` file and execute the code with: 56 | 57 | :: 58 | 59 | python prototype_opencv.py 60 | 61 | You should see a screen where you can move a character with the keys **A and D**: 62 | 63 | .. image:: prototype.png 64 | 65 | 66 | Reflection Questions 67 | -------------------- 68 | 69 | Discuss the following questions: 70 | 71 | - Can you think of any software projects with special risks? 72 | - Could these risks be mitigated by writing a prototype? 73 | - Do you know other engineering disciplines where prototypes are used? 74 | -------------------------------------------------------------------------------- /software_engineering/prototype_curses.py: -------------------------------------------------------------------------------- 1 | """ 2 | Proof-of-concept: move around in a 2D frame 3 | 4 | On Windows, you need to install `windows-curses`: 5 | 6 | pip install windows-curses 7 | 8 | """ 9 | import curses 10 | 11 | # WASD keys 12 | KEY_COMMANDS = {97: "left", 100: "right", 119: "up", 115: "down"} 13 | 14 | # prepare the screen 15 | screen = curses.initscr() 16 | curses.start_color() 17 | curses.init_pair(1, curses.COLOR_GREEN, curses.COLOR_BLACK) 18 | curses.curs_set(0) 19 | curses.noecho() 20 | curses.raw() 21 | screen.keypad(False) 22 | 23 | win = curses.newwin(20, 20, 0, 0) 24 | win.nodelay(True) 25 | 26 | 27 | def game_loop(screen): 28 | """called by curses""" 29 | x, y = 5, 5 30 | 31 | # draw 32 | screen.clear() 33 | screen.addch(y, x, "O", curses.color_pair(1)) 34 | win.refresh() 35 | screen.refresh() 36 | 37 | while True: 38 | 39 | # handle moves 40 | char = win.getch() 41 | direction = KEY_COMMANDS.get(char) 42 | if direction == "left": 43 | x -= 1 44 | elif direction == "right": 45 | x += 1 46 | else: 47 | continue 48 | 49 | # draw 50 | screen.clear() 51 | screen.addch(y, x, "O", curses.color_pair(1)) 52 | win.refresh() 53 | screen.refresh() 54 | 55 | 56 | if __name__ == "__main__": 57 | curses.wrapper(game_loop) 58 | curses.endwin() 59 | -------------------------------------------------------------------------------- /software_engineering/prototype_opencv.py: -------------------------------------------------------------------------------- 1 | """ 2 | Proof-of-concept: move around on a 2D grid 3 | 4 | Install dependencies: 5 | 6 | pip install opencv-python 7 | """ 8 | import numpy as np 9 | import cv2 10 | 11 | 12 | # constants measured in pixel 13 | SCREEN_SIZE_X, SCREEN_SIZE_Y = 640, 640 14 | TILE_SIZE = 64 15 | 16 | 17 | 18 | def read_image(filename): 19 | """returns an image twice as big""" 20 | img = cv2.imread(filename) 21 | return np.kron(img, np.ones((2, 2, 1), dtype=img.dtype)) 22 | 23 | 24 | player_img = read_image("tiles/deep_elf_high_priest.png") 25 | 26 | def draw(x, y): 27 | """draws the player image on the screen""" 28 | frame = np.zeros((SCREEN_SIZE_Y, SCREEN_SIZE_X, 3), np.uint8) 29 | xpos, ypos = x * TILE_SIZE, y * TILE_SIZE 30 | frame[ypos : ypos + TILE_SIZE, xpos : xpos + TILE_SIZE] = player_img 31 | cv2.imshow("frame", frame) 32 | 33 | 34 | # starting position of the player in dungeon 35 | x, y = 4, 4 36 | 37 | exit_pressed = False 38 | 39 | while not exit_pressed: 40 | draw(x, y) 41 | 42 | # handle keyboard input 43 | key = chr(cv2.waitKey(1) & 0xFF) 44 | if key == "d": 45 | x += 1 46 | elif key == "a": 47 | x -= 1 48 | elif key == "q": 49 | exit_pressed = True 50 | 51 | cv2.destroyAllWindows() 52 | -------------------------------------------------------------------------------- /software_engineering/software_engineering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/software_engineering.png -------------------------------------------------------------------------------- /software_engineering/software_engineering.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 22 | 24 | 42 | 44 | 45 | 47 | image/svg+xml 48 | 50 | 51 | 52 | 53 | 54 | 59 | 64 | 67 | Idea Requirements Design Implementation 114 | 121 | 138 | 139 | 142 | 149 | 166 | 167 | 170 | 177 | 194 | 195 | 196 | 201 | formal language prose 242 | 243 | -------------------------------------------------------------------------------- /software_engineering/tiles.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/tiles.zip -------------------------------------------------------------------------------- /software_engineering/title.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/software_engineering/title.png -------------------------------------------------------------------------------- /structure/commandline_args.rst: -------------------------------------------------------------------------------- 1 | Command-line Arguments 2 | ====================== 3 | 4 | One easy way to use your programs more flexibly is through calling them 5 | with command-line arguments from a terminal: 6 | 7 | Two ways to implement CL arguments are **sys.argv** and **argparse**. 8 | 9 | Option 1: sys.argv 10 | ------------------ 11 | 12 | The ``sys.argv`` method is a quick-and-dirty approach. It is quick to 13 | implement, but not very clean in the long run. 14 | 15 | The list ``sys.argv`` contains everything that was entered on the 16 | terminal. For instance, if you write 17 | 18 | :: 19 | 20 | python hello.py Kristian 21 | 22 | the ``sys.argv`` list will contain two strings: 23 | 24 | .. code:: python3 25 | 26 | ['hello.py', 'Kristian'] 27 | 28 | You can access these values in your code like this: 29 | 30 | .. code:: python3 31 | 32 | import sys 33 | 34 | if len(sys.argv) == 2: 35 | name = sys.argv[1] 36 | print(f'Hello {name}') 37 | else: 38 | print("usage: hello.py ") 39 | 40 | ---- 41 | 42 | Option 2: argparse 43 | ------------------ 44 | 45 | This is a cleaner version. ``argparse`` allows you to define 46 | command-line parameters for a Python application. The module takes care 47 | of reading the parameters into variables, checking data types and 48 | generating help text. 49 | 50 | ``argparse`` is installed with Python by default. 51 | 52 | Usage 53 | ~~~~~ 54 | 55 | 1. Define the ``argparse`` options in your code (in a function or your 56 | main block). 57 | 2. Call ``args = parser.parse_args()`` 58 | 3. Access your options as attributes of ``args`` 59 | 60 | In addition to using your option, your program will print usage 61 | instructions when you type 62 | 63 | :: 64 | 65 | python myprog.py --help 66 | 67 | Example 68 | ~~~~~~~ 69 | 70 | Here is a hello world program that you can use as a starting point: 71 | 72 | .. code:: python3 73 | 74 | import argparse 75 | 76 | parser = argparse.ArgumentParser(description='A Hello World program with arguments.') 77 | 78 | parser.add_argument('-m', '--message', type=str, default="Hello ", 79 | help='message to be written') 80 | 81 | parser.add_argument('-c', '--capitals', type=bool, 82 | help='write capitals') 83 | 84 | parser.add_argument('name', type=str, nargs='+', 85 | help='name(s) of the user') 86 | 87 | args = parser.parse_args() 88 | 89 | message = args.message 90 | if args.name: 91 | message = message + ' '.join(args.name) 92 | if args.capitals: 93 | names = names.upper() 94 | print(f"{message} {names}") 95 | 96 | .. seealso:: 97 | 98 | `docs.python.org/3/howto/argparse.html `__ 99 | -------------------------------------------------------------------------------- /structure/main_block.rst: -------------------------------------------------------------------------------- 1 | Structure of a Python script 2 | ============================ 3 | 4 | There is a standard structure recommended for Python scripts: 5 | 6 | 1. imports 7 | 2. constants 8 | 3. functions and classes 9 | 4. ``__main__`` block 10 | 11 | Example 12 | ------- 13 | 14 | The following program contains most of these elements 15 | 16 | .. code:: python3 17 | 18 | from pprint import pprint 19 | 20 | MESSAGE = "Good morning {}. Nice to see you!" 21 | 22 | def hello(name): 23 | """Writes a hello world message""" 24 | msg = MESSAGE.format(name) 25 | pprint(msg) 26 | 27 | if __name__ == '__main__': 28 | name = input("please enter your name: ") 29 | hello(name) 30 | 31 | ---- 32 | 33 | Imports 34 | ------- 35 | 36 | All import statements should be grouped together at the beginning of a 37 | Python file. If you have lots of imports, a standard for sorting them 38 | might be useful. The tools **isort** and **black** help with that. 39 | 40 | ---- 41 | 42 | Constants 43 | --------- 44 | 45 | In Python, constants do not exist as a syntax element. 46 | But it is a good practice to group variables 47 | *“that the programmer does not intend to change”*. 48 | 49 | These constants should be written in capitals to distinguish them from 50 | variables that change during their lifetime: 51 | 52 | .. code:: python3 53 | 54 | PI = 3.14159 55 | 56 | ---- 57 | 58 | Function and class definitions 59 | ------------------------------ 60 | 61 | Functions and classes should be defined in nested-first-order. That 62 | means functions called by other functions go first, functions called 63 | from the main program only go last. 64 | 65 | ---- 66 | 67 | The **main** block 68 | ------------------ 69 | 70 | At the end of the program, there is a strange construct: The 71 | ``if __name__ == '__main__':`` block indicates the main program. 72 | 73 | The ``__main__`` block is only executed when you run the entire program 74 | with ``python my_program.py`` (or the equivalent in your editor). 75 | However, it is **not** executed when you import functions or classes 76 | from it. This way you can have a main program and reusable functions in 77 | the same file. 78 | 79 | It you can keep an entire code block here, or only call a ``main()`` 80 | function. Sometimes the ``__main__`` block is used for other things 81 | (e.g. test code). 82 | -------------------------------------------------------------------------------- /structure/modules.rst: -------------------------------------------------------------------------------- 1 | Modules 2 | ======= 3 | 4 | Any Python file (ending with ``.py``) can be imported by Python script. 5 | A single Python file is also called a **module**. This helps you to 6 | divide a bigger program into several smaller pieces. 7 | 8 | For instance if you have a file ``names.py`` containing the following: 9 | 10 | .. code:: python3 11 | 12 | FIRST_NAMES = ['Alice', 'Bob', 'Charlie'] 13 | 14 | Then you can write (e.g. in a second Python file in the same directory): 15 | 16 | .. code:: python3 17 | 18 | import names 19 | print(names.FIRST_NAMES) 20 | 21 | ---- 22 | 23 | Packages 24 | ======== 25 | 26 | For big programs, it is useful to divide up the code among several 27 | directories. A directory from which you import Python modules is called a **package**. 28 | For instance, you could have the following files in a package ``namedata``: 29 | 30 | :: 31 | 32 | namedata/ 33 | __init__.py 34 | names.py 35 | 36 | ---- 37 | 38 | Importing modules and packages 39 | ------------------------------ 40 | 41 | To import from a module, a package or their contents, place its name 42 | (without .py) needs to be given in the import statement. Import 43 | statements can look like this: 44 | 45 | .. code:: python3 46 | 47 | import names 48 | import names as n 49 | from names import FIRST_NAMES 50 | from namedata.names import FIRST_NAMES 51 | 52 | It is strongly recommended to list the imported variables and functions 53 | explicitly and not write 54 | 55 | .. code:: python3 56 | 57 | from names import * 58 | 59 | The latter makes debugging difficult. 60 | 61 | When importing, Python generates intermediate files (bytecode) in the 62 | ``__pycache__`` directory that help to execute programs more 63 | efficiently. It is managed automatically, and you can safely ignore it. 64 | 65 | ---- 66 | 67 | The __init__ file 68 | ----------------- 69 | 70 | You can define the file ``__init__.py`` to make importing from packages more flexible. 71 | It is executed automatically when you import anything from that package. 72 | Suppose you want to use an object from a package, you would import it with: 73 | 74 | .. code:: python3 75 | 76 | from namedata.names import FIRST_NAMES 77 | 78 | Now you create a ``__init__.py`` file with a single line: 79 | 80 | .. code:: python3 81 | 82 | from names import FIRST_NAMES 83 | 84 | That import is made available in the namespace of the package, so that the following works: 85 | 86 | .. code:: python3 87 | 88 | from namedata import FIRST_NAMES 89 | 90 | 91 | The ``__init__.py`` file makes is easier to move objects around inside a package. 92 | 93 | ---- 94 | 95 | How does Python find modules and packages? 96 | ------------------------------------------ 97 | 98 | When importing modules or packages, Python needs to know where to find 99 | them. There is a certain sequence of directories in which Python looks 100 | for things to import: 101 | 102 | - The current directory. 103 | - The site-packages folder (where Python is installed). 104 | - In directories in the ``PYTHONPATH`` environment variable. 105 | 106 | You can see all directories from within Python by checking the ``sys.path`` variable: 107 | 108 | .. code:: python3 109 | 110 | import sys 111 | print sys.path 112 | -------------------------------------------------------------------------------- /structure/namespaces.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/structure/namespaces.png -------------------------------------------------------------------------------- /structure/namespaces.rst: -------------------------------------------------------------------------------- 1 | Namespaces 2 | ========== 3 | 4 | .. image:: namespaces.png 5 | 6 | Exercise 1 7 | ---------- 8 | 9 | Create two new Python files in your working directory: 10 | 11 | :: 12 | 13 | triangle.py 14 | 15 | a = 3 16 | b = 4 17 | 18 | and 19 | 20 | :: 21 | 22 | rectangle.py 23 | 24 | a = 4 25 | b = 3 26 | 27 | Exercise 2 28 | ---------- 29 | 30 | Import both modules from the command line or a third script and print 31 | the value of ``a``. Consider the following variations: 32 | 33 | :: 34 | 35 | from triangle import a 36 | 37 | import triangle 38 | 39 | Exercise 3 40 | ---------- 41 | 42 | Check the contents of the **namespace** before and after the above 43 | imports with the function ``dir()``: 44 | 45 | :: 46 | 47 | print(dir()) 48 | 49 | Exercise 4 50 | ---------- 51 | 52 | Add the follow function to ``triangle.py``: 53 | 54 | :: 55 | 56 | import math 57 | 58 | def calc_hypothenuse(a, b): 59 | c = math.sqrt(a ** 2 + b ** 2) 60 | return c 61 | 62 | Import and call the function in your main module: 63 | 64 | :: 65 | 66 | c = calc_hypothenuse(4, 5) 67 | return c 68 | 69 | Exercise 5 70 | ---------- 71 | 72 | Write a function ``calc_area(a, b)`` in ``rectangle.py``. 73 | 74 | Exercise 6 75 | ---------- 76 | 77 | Discuss the concepts of **variable scope** and **namespaces** 78 | -------------------------------------------------------------------------------- /structure/structuring_python_programs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/krother/advanced_python/ccdd898605d4fc1adacbe4f5cd4e0d42510c4f7e/structure/structuring_python_programs.png -------------------------------------------------------------------------------- /testing/facade.rst: -------------------------------------------------------------------------------- 1 | The Facade Pattern 2 | ================== 3 | 4 | Before you can write automated tests, you need to make sure that your code is testable. 5 | It is not a great idea to test any function or class in your program, 6 | because that makes the program harder to modify in the future. 7 | Whatever you test, you want to be stable and not change very often. 8 | Testable code means that you need to define an **interface** you are testing against. 9 | 10 | Also, some things are harder to test than others, graphics and keyboard input for instance. 11 | We won't test them for now. Instead, we want to make the code more testable 12 | by **separating the graphics engine and game logic**. 13 | 14 | The Design 15 | ---------- 16 | 17 | In the `Facade Pattern `__, you define a single class 18 | that serves as the interface to an entire subsysten. 19 | We will define such a Facade class for the game logic named ``DungeonExplorer``: 20 | 21 | .. code:: python3 22 | 23 | class DungeonExplorer(BaseModel): 24 | 25 | player: Player 26 | level: Level 27 | 28 | def get_objects() -> list[DungeonObject]: 29 | """Returns everything in the dungeon to be used by a graphics engine" 30 | ... 31 | 32 | def execute_command(cmd: str) -> None: 33 | """Performs a player action, such as 'left', 'right', 'jump', 'fireball'""" 34 | ... 35 | 36 | 37 | Note that the attributes ``player`` and ``level`` of the game might change in the future. 38 | We will treat them as private. 39 | All the communication should happen through the two methods. 40 | 41 | In the following exercise, you refactor the code to use the Facade pattern. 42 | 43 | Step 1: Separate Modules 44 | ------------------------ 45 | 46 | Split the existing code into two Python modules ``graphics_engine.py`` and ``game_logic.py``. 47 | For each paragraph of code decide, which of the two modules it belongs to. 48 | 49 | Step 2: Implement the Facade class 50 | ---------------------------------- 51 | 52 | Copy the skeleton code for the ``DungeonExplorer`` class to ``game_logic.py``. 53 | Leave the methods empty for now. 54 | 55 | Step 3: Define a class for data exchange 56 | ---------------------------------------- 57 | 58 | In the ``get_objects()`` method, we use the type ``DungeonObject`` to send everything that 59 | should be drawn to the graphics engine. 60 | This includes walls, the player for now, but will include more stuff later. 61 | Define it as follows: 62 | 63 | .. code:: python3 64 | 65 | class DungeonObject(BaseModel): 66 | position: Position 67 | name: str 68 | 69 | Example objects could be: 70 | 71 | .. code:: python3 72 | 73 | DungeonObject(Position(x=1, y=1), "wall") 74 | DungeonObject(Position(x=4, y=4), "player") 75 | 76 | .. note:: 77 | 78 | This is really a very straightforward approach to send the information for drawing. 79 | In fact, it makes a couple of things very hard, e.g. animation. 80 | This is an example of a design decision: we choose that we do not want animations in the game. 81 | Our design makes adding them more expensive. 82 | 83 | Step 4: Implement the get_objects method 84 | ---------------------------------------- 85 | 86 | Implement the ``get_objects()`` method from scratch. 87 | Create a list of the player and all walls as a list of ``DungeonObject``. 88 | 89 | Step 5: Implement the execute_command method 90 | -------------------------------------------- 91 | 92 | Move the code you have for handling keyboard input into the ``execute_command()`` method. 93 | Replace the keys by explicit commands like `"left"`, `"right"` etc. 94 | The idea behind that is that we do not want the game logic to know anything about 95 | which key you press to walk right. This belongs to the user interface. 96 | 97 | Step 6: Import the Facade class 98 | ------------------------------- 99 | 100 | In the module ``graphics_engine.py``, import the Facade class ``DungeonExplorer``. 101 | The only things the user interface needs to know about are the Facade class and 102 | the data exchange class ``DungeonObject`` (although we do not have to import the latter). 103 | 104 | Create an instance of it. 105 | 106 | Step 7: Adjust the graphics engine 107 | ---------------------------------- 108 | 109 | Make sure the graphics engine does the following: 110 | 111 | - it calls ``DungeonExplorer.get_objects`` in the draw function. 112 | - it does not access the level or player attributes in the draw function. 113 | - it translates the keys to commands 114 | - it calls the ``DungeonExplorer.execute_command`` method. 115 | -------------------------------------------------------------------------------- /testing/unit_test.rst: -------------------------------------------------------------------------------- 1 | Unit Tests 2 | ========== 3 | 4 | In this short exercise, we will write a test against the Facade. 5 | 6 | Step 1: Install pytest 7 | ---------------------- 8 | 9 | Make sure pytest is installed: 10 | 11 | :: 12 | 13 | pip install pytest 14 | 15 | Step 2: Create a test 16 | --------------------- 17 | 18 | Create a file ``test_game_logic.py``. In it, you need the folowing code: 19 | 20 | .. code:: python3 21 | 22 | from game_logic import DungeonExplorer, DungeonObject 23 | 24 | def test_move(): 25 | dungeon = DungeonExplorer( 26 | player=Player(Position(x=1, y=1), 27 | ... # add other attributes if necessary 28 | dungeon.execute_command("right") 29 | assert DungeonObject(Position(x=2, y=1), "player") in dungeon.get_objects() 30 | 31 | A typical automated test consists of three parts: 32 | 33 | 1. setting up test data (fixtures) 34 | 2. executing the code that is tested 35 | 3. checking the results against expected values 36 | 37 | Step 3: Run the test 38 | -------------------- 39 | 40 | Run the tests from the terminal with: 41 | 42 | :: 43 | 44 | pytest 45 | 46 | You should see a message that the test either passes or fails. 47 | --------------------------------------------------------------------------------