├── standard_lib ├── empty_set_literal.py ├── print_human_friendly_numbers.py ├── read_files_using_iterator.py ├── chained_assignments.py ├── fill_zeros.py ├── tuple_comprehension.py ├── variable_swapping.py ├── hex_decode.py ├── get_password_input.py ├── uuid1_example.py ├── regular_expression_debug.py ├── iterable_unpacking.py ├── update_dict_using_tuples.py ├── list_unpacking.py ├── reverse_iterables.py ├── crazy_dict_expression.py ├── count_thresholds.py ├── string_capitalize.py ├── unicode_source_code.py ├── for_loop_index.py ├── open_browser_tab.py ├── overwrite_dictionary.py ├── partial_function.py ├── parse_query_string.py ├── dict_based_on_lists.py ├── chained_comparison.py ├── remove_elements_list.py ├── sort_complex_tuples.py ├── max_int_in_list_of_str.py ├── set_union_intersection.py ├── zip_safe.py ├── builtins_manipulation.py ├── merge_arbitrary_number_of_dicts.py ├── multidict_with_default_init.py ├── remove_duplicates_list.py ├── combine_iterables.py ├── file_matching_regex.py ├── check_pattern.py ├── split_preserving_sub-strings.py ├── slice_generators.py ├── drawing_turtle.py ├── pathlib_relpath.py ├── compare_strings.py ├── pass_multiple_dicts.py ├── tree_clone.py ├── timing_context_manager.py ├── human_readable_bytecode.py ├── reduce_memory_consumption_iterator.py ├── emulate_switch_case.py ├── disassemble_bytecode.py ├── abc_class.py ├── multi_open_files.py ├── flatten.py ├── capture_output.py ├── save_dict_update_without_loosing_original.py ├── bit_flipper.py ├── priority_queue.py ├── provide_default_config_values.py ├── scopes_namespaces.py ├── keep_metadata_on_decorator_usage.py ├── trace_decorator.py ├── function_arguments.py ├── hash_file.py ├── temptable_contextmanager.py ├── port_scanner.py ├── async_sleep_sync.py ├── async_sleep_async.py ├── binary_search.py ├── bitmask.py ├── deprecated_decorator.py ├── reduce_memory_consumption.py ├── README.md └── MicroWebServer.py ├── ebook ├── python-snippets.epub ├── python-snippets.mobi ├── python-snippets.pdf ├── generate.sh ├── .latexmkrc ├── chapters │ ├── third_party.tex │ ├── standard_lib_chapters │ │ ├── search_algorithms.tex │ │ ├── abc.tex │ │ ├── async.tex │ │ ├── playing_bits.tex │ │ ├── files.tex │ │ ├── bytecode.tex │ │ ├── context_manager.tex │ │ ├── dealing_builtins.tex │ │ ├── decorators.tex │ │ ├── lists.tex │ │ ├── string_operations.tex │ │ ├── dicts.tex │ │ └── non_categorized.tex │ ├── standard_lib.tex │ └── third_party_chapters │ │ ├── timing.tex │ │ ├── image_animation.tex │ │ ├── commandline.tex │ │ └── non_categorized.tex └── python-snippets.tex ├── third_party ├── src │ ├── python.png │ ├── samples.xlsx │ └── text.txt ├── fancy_cli_header.py ├── simple_progressbar.py ├── attention_message.py ├── numpy_array_operations.py ├── show_progress.py ├── folium_snippet.py ├── mypy_example.py ├── is_holiday.py ├── timing_tool.py ├── simple_debugger.py ├── inspect_docker.py ├── cooked_input_example.py ├── display_tables.py ├── parse_complex_excel_sheets.py ├── f-strings_vs_str.py ├── colored_python.py ├── range_vs_repeat.py ├── formatted_json.py ├── text_analysis.py ├── change_bg.py ├── resize_images.py ├── count_python_bytes.py ├── pytest_rename_class_backwards_compatibility.py ├── complex_list_ops.py ├── tensorflow_image.py ├── animated_graphics.py ├── reduce_pandas_df_memory.py ├── world_bank_data.py ├── clint_cli_tool.py ├── test_gevent.py ├── port_scanner_nmap.py ├── test_tornado.py ├── manipulate_images.py ├── cli_help_strings.py ├── test_asyncio.py ├── auto_login_website.py ├── refactoring_code │ ├── refactoring_code.py │ └── testutils.py ├── mocking_requests.py ├── pyproject.toml ├── interactive_cli.py ├── img_to_ascii_2.py ├── README.md ├── mathematicians.py ├── img_to_ascii.py ├── steganography.py └── medium.py ├── .gitignore ├── LICENSE └── README.md /standard_lib/empty_set_literal.py: -------------------------------------------------------------------------------- 1 | empty_set = {*[]} 2 | print(empty_set) 3 | -------------------------------------------------------------------------------- /standard_lib/print_human_friendly_numbers.py: -------------------------------------------------------------------------------- 1 | n = 123123123 2 | print(format(n, ",")) 3 | -------------------------------------------------------------------------------- /standard_lib/read_files_using_iterator.py: -------------------------------------------------------------------------------- 1 | print(next(open("huge_log_file.txt"))) 2 | -------------------------------------------------------------------------------- /standard_lib/chained_assignments.py: -------------------------------------------------------------------------------- 1 | x = y = z = 26 2 | print(x) 3 | print(y) 4 | print(z) 5 | -------------------------------------------------------------------------------- /standard_lib/fill_zeros.py: -------------------------------------------------------------------------------- 1 | for i in [1, 11, 222, "aaa", 1234]: 2 | print(str(i).zfill(4)) 3 | -------------------------------------------------------------------------------- /standard_lib/tuple_comprehension.py: -------------------------------------------------------------------------------- 1 | result = (*(x**2 for x in range(5)),) 2 | print(result) 3 | -------------------------------------------------------------------------------- /standard_lib/variable_swapping.py: -------------------------------------------------------------------------------- 1 | x = 4 2 | y = 5 3 | x, y = y, x 4 | print(x) 5 | print(y) 6 | -------------------------------------------------------------------------------- /standard_lib/hex_decode.py: -------------------------------------------------------------------------------- 1 | print(bytes.fromhex("4d 65 72 72 79 20 43 68 72 69 73 74 6d 61 73 21")) 2 | -------------------------------------------------------------------------------- /standard_lib/get_password_input.py: -------------------------------------------------------------------------------- 1 | import getpass 2 | 3 | 4 | pwd = getpass.getpass() 5 | print(pwd) 6 | -------------------------------------------------------------------------------- /standard_lib/uuid1_example.py: -------------------------------------------------------------------------------- 1 | from uuid import uuid1 2 | 3 | for i in range(5): 4 | print(uuid1()) 5 | -------------------------------------------------------------------------------- /ebook/python-snippets.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahlitzFlorian/python-snippets/HEAD/ebook/python-snippets.epub -------------------------------------------------------------------------------- /ebook/python-snippets.mobi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahlitzFlorian/python-snippets/HEAD/ebook/python-snippets.mobi -------------------------------------------------------------------------------- /ebook/python-snippets.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahlitzFlorian/python-snippets/HEAD/ebook/python-snippets.pdf -------------------------------------------------------------------------------- /standard_lib/regular_expression_debug.py: -------------------------------------------------------------------------------- 1 | import re 2 | 3 | 4 | re.compile("([\w\.-]+)@([\w\.-]+)", re.DEBUG) 5 | -------------------------------------------------------------------------------- /third_party/src/python.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahlitzFlorian/python-snippets/HEAD/third_party/src/python.png -------------------------------------------------------------------------------- /standard_lib/iterable_unpacking.py: -------------------------------------------------------------------------------- 1 | numbers = [1, 2, 3] 2 | x, y, z = numbers 3 | print(x) 4 | print(y) 5 | print(z) 6 | -------------------------------------------------------------------------------- /third_party/src/samples.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahlitzFlorian/python-snippets/HEAD/third_party/src/samples.xlsx -------------------------------------------------------------------------------- /standard_lib/update_dict_using_tuples.py: -------------------------------------------------------------------------------- 1 | a = {1: "1"} 2 | b = [(2, "2"), (3, "3")] 3 | 4 | a.update(b) 5 | 6 | print(a) 7 | -------------------------------------------------------------------------------- /standard_lib/list_unpacking.py: -------------------------------------------------------------------------------- 1 | a, *b, c = [1, 2, 3, 4, 5] 2 | 3 | print(f"a = {a}") 4 | print(f"b = {b}") 5 | print(f"c = {c}") 6 | -------------------------------------------------------------------------------- /standard_lib/reverse_iterables.py: -------------------------------------------------------------------------------- 1 | names = ["Lea", "Mario", "Nicole"] 2 | 3 | for name in reversed(names): 4 | print(name) 5 | -------------------------------------------------------------------------------- /third_party/fancy_cli_header.py: -------------------------------------------------------------------------------- 1 | from pyfiglet import Figlet 2 | 3 | f = Figlet(font="slant") 4 | print(f.renderText("text to render")) 5 | -------------------------------------------------------------------------------- /third_party/simple_progressbar.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | from tqdm import tqdm 3 | 4 | 5 | for i in tqdm(range(100)): 6 | sleep(0.2) 7 | -------------------------------------------------------------------------------- /ebook/generate.sh: -------------------------------------------------------------------------------- 1 | NAME=$(basename -s .tex "$1") 2 | latexmk "$1" 3 | pandoc --toc $NAME.tex -o $NAME.epub 4 | ebook-convert $NAME.epub $NAME.mobi 5 | -------------------------------------------------------------------------------- /standard_lib/crazy_dict_expression.py: -------------------------------------------------------------------------------- 1 | from pprint import pprint 2 | 3 | crazy_dict = {True: "yes", 1: "no", 1.0: "maybe"} 4 | pprint(crazy_dict) 5 | -------------------------------------------------------------------------------- /third_party/attention_message.py: -------------------------------------------------------------------------------- 1 | import termcolor 2 | 3 | 4 | termcolor.cprint("Hey, this is important!!!", "white", "on_red", attrs=["blink"]) 5 | -------------------------------------------------------------------------------- /third_party/numpy_array_operations.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | 4 | x = np.array([1, 2, 3, 4, 5]) 5 | 6 | print(f"{x * 2}") 7 | print(f"{x * x}") 8 | -------------------------------------------------------------------------------- /standard_lib/count_thresholds.py: -------------------------------------------------------------------------------- 1 | data = [1, 2, 3, 4, 5, 6, 7] 2 | low, high = 3, 6 3 | result = sum(low <= x < high for x in data) 4 | 5 | print(result) 6 | -------------------------------------------------------------------------------- /standard_lib/string_capitalize.py: -------------------------------------------------------------------------------- 1 | string = "i love coffee" 2 | 3 | print(f"capitalize(): {string.capitalize()}") 4 | print(f"title(): {string.title()}") 5 | -------------------------------------------------------------------------------- /standard_lib/unicode_source_code.py: -------------------------------------------------------------------------------- 1 | # Only works in Python REPL 2 | 3 | def ^2(' ): 4 | f"result" = ' * ' 5 | return f"result" 6 | 7 | 8 | ^2(4) -------------------------------------------------------------------------------- /standard_lib/for_loop_index.py: -------------------------------------------------------------------------------- 1 | names = ["Sandy", "Florian", "Mike", "Katie", "Maria"] 2 | 3 | for i, name in enumerate(names): 4 | print(f"{i+1}. {name}") 5 | -------------------------------------------------------------------------------- /standard_lib/open_browser_tab.py: -------------------------------------------------------------------------------- 1 | import webbrowser 2 | 3 | 4 | controller = webbrowser.get("firefox") 5 | controller.open_new_tab("https://stackoverflow.com") 6 | -------------------------------------------------------------------------------- /third_party/show_progress.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | from progressbar import ProgressBar 3 | 4 | bar = ProgressBar() 5 | for i in bar(range(50)): 6 | sleep(0.5) 7 | -------------------------------------------------------------------------------- /third_party/folium_snippet.py: -------------------------------------------------------------------------------- 1 | import folium 2 | 3 | mapOBJ = folium.Map(location=[39.5, -98.35], zoom_start=4, tiles="Stamen Terrain") 4 | mapOBJ.save(outfile="map.html") 5 | -------------------------------------------------------------------------------- /third_party/mypy_example.py: -------------------------------------------------------------------------------- 1 | import mypy 2 | 3 | 4 | def add(a: int, b: int) -> int: 5 | return a + b 6 | 7 | 8 | print(add(1, 5)) 9 | print(add("a", 3)) 10 | -------------------------------------------------------------------------------- /standard_lib/overwrite_dictionary.py: -------------------------------------------------------------------------------- 1 | defaults = {"lenny": "white", "carl": "black"} 2 | overwrite = {"lenny": "yellow"} 3 | new = {**defaults, **overwrite} 4 | 5 | print(new) 6 | -------------------------------------------------------------------------------- /standard_lib/partial_function.py: -------------------------------------------------------------------------------- 1 | from functools import partial 2 | from itertools import repeat 3 | 4 | times = partial(repeat, None) 5 | min(random.random() for _ in times(100_000)) 6 | -------------------------------------------------------------------------------- /standard_lib/parse_query_string.py: -------------------------------------------------------------------------------- 1 | from urllib.parse import parse_qs 2 | 3 | 4 | input_string = "host=171.25.2.7&port=8080&port=8081" 5 | result = parse_qs(input_string) 6 | print(result) 7 | -------------------------------------------------------------------------------- /third_party/is_holiday.py: -------------------------------------------------------------------------------- 1 | from datetime import date 2 | 3 | import holidays 4 | 5 | 6 | today = date.today() 7 | is_holiday = today in holidays.Germany() 8 | 9 | print(is_holiday) 10 | -------------------------------------------------------------------------------- /third_party/timing_tool.py: -------------------------------------------------------------------------------- 1 | from boxx import timeit 2 | from time import sleep 3 | 4 | 5 | with timeit(): 6 | sleep(0.01) 7 | 8 | with timeit(name="sleep"): 9 | sleep(0.1) 10 | -------------------------------------------------------------------------------- /standard_lib/dict_based_on_lists.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | 4 | nums = [1, 2, 3] 5 | chars = ["a", "b", "c"] 6 | 7 | alph = dict(zip(nums, chars)) 8 | print(json.dumps(alph, indent=4, sort_keys=True)) 9 | -------------------------------------------------------------------------------- /standard_lib/chained_comparison.py: -------------------------------------------------------------------------------- 1 | x = 13 2 | lower_bound, upper_bound = 10, 20 3 | print(lower_bound, upper_bound) 4 | print(x > lower_bound and x < upper_bound) 5 | print(lower_bound < x < upper_bound) 6 | -------------------------------------------------------------------------------- /third_party/simple_debugger.py: -------------------------------------------------------------------------------- 1 | import pysnooper 2 | 3 | 4 | @pysnooper.snoop() 5 | def foo(): 6 | a = [x for x in range(100)] 7 | return [x for x in a if x % 2 == 0] 8 | 9 | 10 | foo() 11 | -------------------------------------------------------------------------------- /standard_lib/remove_elements_list.py: -------------------------------------------------------------------------------- 1 | data = [1, 2, 3, 4, 1, 2, 3, 4] 2 | target = 1 3 | 4 | print(f"Original: {data}") 5 | 6 | data[:] = [elem for elem in data if elem != target] 7 | 8 | print(f"New: {data}") 9 | -------------------------------------------------------------------------------- /standard_lib/sort_complex_tuples.py: -------------------------------------------------------------------------------- 1 | student_tuples = [("John", "A", 15), ("Jane", "B", 12), ("Dave", "B", 10)] 2 | 3 | sorted_list = sorted(student_tuples, key=lambda student: student[2]) 4 | 5 | print(sorted_list) 6 | -------------------------------------------------------------------------------- /standard_lib/max_int_in_list_of_str.py: -------------------------------------------------------------------------------- 1 | numbers = ["9", "44", "8", "222"] 2 | 3 | max_string = max(numbers) 4 | print(f"Max string: {max_string}") 5 | 6 | max_number = max(numbers, key=int) 7 | print(f"Max number: {max_number}") 8 | -------------------------------------------------------------------------------- /third_party/inspect_docker.py: -------------------------------------------------------------------------------- 1 | import docker 2 | 3 | 4 | client = docker.from_env() 5 | 6 | print(client.containers.run("ubuntu:16.04", "echo hello world")) 7 | print(client.containers.list()) 8 | print(client.images.list()) 9 | -------------------------------------------------------------------------------- /standard_lib/set_union_intersection.py: -------------------------------------------------------------------------------- 1 | set_a = {1, 2} 2 | set_b = {2, 3} 3 | 4 | print("Use | and & for set union and intersection.") 5 | print(f"{set_a} & {set_b} = {set_a & set_b}") 6 | print(f"{set_a} | {set_b} = {set_a | set_b}") 7 | -------------------------------------------------------------------------------- /standard_lib/zip_safe.py: -------------------------------------------------------------------------------- 1 | import itertools 2 | 3 | 4 | a = [1, 2, 3] 5 | b = ["One", "Two"] 6 | 7 | result1 = list(zip(a, b)) 8 | result2 = list(itertools.zip_longest(a, b)) 9 | 10 | print(result1) 11 | print(result2) 12 | -------------------------------------------------------------------------------- /standard_lib/builtins_manipulation.py: -------------------------------------------------------------------------------- 1 | _print = print 2 | 3 | 4 | def print(*args, **kwargs): 5 | nargs = (*args, "\nNumber of arguments:", len(args)) 6 | _print(*nargs, **kwargs) 7 | 8 | 9 | print("One", "Two", "Three") 10 | -------------------------------------------------------------------------------- /standard_lib/merge_arbitrary_number_of_dicts.py: -------------------------------------------------------------------------------- 1 | dict1 = {"a": 1, "b": 2} 2 | dict2 = {"b": 3, "c": 4, 3: 9} 3 | dict3 = {"a": 2} 4 | dict4 = {"d": 8, "e": 1} 5 | 6 | print("Result of merging dict 1-4:", {**dict1, **dict2, **dict3, **dict4}) 7 | -------------------------------------------------------------------------------- /third_party/cooked_input_example.py: -------------------------------------------------------------------------------- 1 | import cooked_input as ci 2 | 3 | 4 | cap_cleaner = ci.CapitalizationCleaner(style=ci.ALL_WORDS_CAP_STYLE) 5 | name = ci.get_string(prompt="What is your name?", cleaners=[cap_cleaner]) 6 | 7 | print(name) 8 | -------------------------------------------------------------------------------- /standard_lib/multidict_with_default_init.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict 2 | 3 | 4 | rd = defaultdict(list) 5 | 6 | for name, num in [("ichi", 1), ("one", 1), ("uno", 1), ("un", 1)]: 7 | rd[num].append(name) 8 | 9 | print(rd) 10 | -------------------------------------------------------------------------------- /third_party/display_tables.py: -------------------------------------------------------------------------------- 1 | from prettytable import PrettyTable 2 | 3 | 4 | table = PrettyTable(["food", "price"]) 5 | table.add_row(["ham", "$2"]) 6 | table.add_row(["eggs", "$1"]) 7 | table.add_row(["spam", "$4"]) 8 | 9 | print(table) 10 | -------------------------------------------------------------------------------- /standard_lib/remove_duplicates_list.py: -------------------------------------------------------------------------------- 1 | from collections import OrderedDict 2 | 3 | 4 | a = ["foo", "Alice", "bar", "foo", "Bob"] 5 | print(f"List with duplicates: {a}") 6 | 7 | a = list(OrderedDict.fromkeys(a).keys()) 8 | print(f"Without duplicates: {a}") 9 | -------------------------------------------------------------------------------- /ebook/.latexmkrc: -------------------------------------------------------------------------------- 1 | $pdflatex = "xelatex %O %S"; 2 | $pdf_mode = 1; 3 | $dvi_mode = 0; 4 | $postscript_mode = 0; 5 | add_cus_dep('glo', 'gls', 0, 'makeglo2gls'); 6 | sub makeglo2gls { 7 | system("makeindex -s '$_[0]'.ist -t '$_[0]'.glg -o '$_[0]'.gls '$_[0]'.glo"); 8 | } 9 | -------------------------------------------------------------------------------- /standard_lib/combine_iterables.py: -------------------------------------------------------------------------------- 1 | grades = [1, 2, 3, 4, 5] 2 | phrases = ["very good", "good", "okay", "at least not failed", "failed"] 3 | 4 | combined = list(zip(grades, phrases)) 5 | print(combined) 6 | 7 | for grade, phrase in combined: 8 | print(grade, phrase) 9 | -------------------------------------------------------------------------------- /third_party/parse_complex_excel_sheets.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | import pandas as pd 4 | 5 | 6 | path = Path("src/samples.xlsx") 7 | xls = pd.ExcelFile(path) 8 | print(xls.sheet_names) 9 | 10 | df = xls.parse(xls.sheet_names[0]) 11 | print(df.head()) 12 | -------------------------------------------------------------------------------- /standard_lib/file_matching_regex.py: -------------------------------------------------------------------------------- 1 | import fnmatch 2 | import os 3 | import re 4 | 5 | 6 | reg = "({})|({})".format(fnmatch.translate("*.md"), fnmatch.translate("*.git*")) 7 | markdown_files = [f for f in os.listdir() if re.match(reg, f)] 8 | 9 | print(markdown_files) 10 | -------------------------------------------------------------------------------- /third_party/f-strings_vs_str.py: -------------------------------------------------------------------------------- 1 | from boxx import timeit 2 | 3 | 4 | with timeit(name="f-strings"): 5 | for i in range(500_000): 6 | x = 6 7 | f"{x}" 8 | 9 | with timeit(name="str"): 10 | for i in range(500_000): 11 | x = 6 12 | str(x) 13 | -------------------------------------------------------------------------------- /standard_lib/check_pattern.py: -------------------------------------------------------------------------------- 1 | """ 2 | Check for multiple string patterns. 3 | """ 4 | lst = ["hello", "fafaea", "hello world", "xxx world", "zzz"] 5 | patterns = ["hello", "world"] 6 | 7 | result = [item for item in lst if any(pattern in item for pattern in patterns)] 8 | 9 | print(result) 10 | -------------------------------------------------------------------------------- /third_party/colored_python.py: -------------------------------------------------------------------------------- 1 | from termcolor import colored, cprint 2 | 3 | # Normal print 4 | print("Hello World!") 5 | 6 | # Normal print with colored text 7 | text = colored("Hello Red World!", "red") 8 | print(text) 9 | 10 | # Termcolor colored print 11 | cprint("Hello Blue World!", "blue") 12 | -------------------------------------------------------------------------------- /third_party/range_vs_repeat.py: -------------------------------------------------------------------------------- 1 | import itertools 2 | import random 3 | 4 | from boxx import timeit 5 | 6 | with timeit(name="Range"): 7 | min(random.random() for i in range(100_000)) 8 | 9 | with timeit(name="Repeat"): 10 | min(random.random() for _ in itertools.repeat(None, 100_000)) 11 | -------------------------------------------------------------------------------- /standard_lib/split_preserving_sub-strings.py: -------------------------------------------------------------------------------- 1 | import shlex 2 | 3 | 4 | sample = 'This is "Berlin Cafe"' 5 | split_str = sample.split() 6 | shlex_str = shlex.split(sample) 7 | some = shlex.quote(sample) 8 | 9 | print(f"Split string output: {split_str}") 10 | print(f"Shlex string outpur: {shlex_str}") 11 | -------------------------------------------------------------------------------- /third_party/formatted_json.py: -------------------------------------------------------------------------------- 1 | import json 2 | import requests 3 | 4 | 5 | def sample_request(url: str): 6 | resp = requests.get(url) 7 | 8 | return resp.json() 9 | 10 | 11 | response = sample_request("https://jsonplaceholder.typicode.com/todos/1") 12 | print(json.dumps(response, indent=4)) 13 | -------------------------------------------------------------------------------- /standard_lib/slice_generators.py: -------------------------------------------------------------------------------- 1 | import itertools 2 | 3 | 4 | def fibonacci(): 5 | current, nxt = 0, 1 6 | while True: 7 | current, nxt = nxt, nxt + current 8 | yield current 9 | 10 | 11 | first_fove = list(itertools.islice(fibonacci(), 5)) 12 | print(f"First five: {first_fove}") 13 | -------------------------------------------------------------------------------- /third_party/text_analysis.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | from textblob import TextBlob 4 | 5 | 6 | path = Path("src/text.txt") 7 | 8 | with open(path) as f: 9 | text = f.read() 10 | 11 | blob = TextBlob(text) 12 | 13 | for sentence in blob.sentences: 14 | print(sentence.sentiment.polarity) 15 | -------------------------------------------------------------------------------- /third_party/change_bg.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | 4 | 5 | img = cv2.imread(open("")) 6 | 7 | for i in range(img.shape[0]): 8 | for j in range(img.shape[1]): 9 | if all(img[i][j] == np.array([114, 89, 47])): 10 | img[i][j] = np.array([200, 0, 0]) 11 | 12 | cv2.imwrite("new.jpg", img) 13 | -------------------------------------------------------------------------------- /third_party/resize_images.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import glob 3 | 4 | 5 | images = glob.glob("*.jpg") 6 | 7 | for image in images: 8 | img = cv2.imread(image, 0) 9 | re = cv2.resize(img, (100, 100)) 10 | cv2.imshow("Hey", re) 11 | cv2.waitKey(500) 12 | cv2.destroyAllWindows() 13 | cv2.imwrite("resized_" + image, re) 14 | -------------------------------------------------------------------------------- /standard_lib/drawing_turtle.py: -------------------------------------------------------------------------------- 1 | import turtle 2 | 3 | 4 | turtle.tracer(False) 5 | 6 | turtle.penup() 7 | turtle.goto(-50, -50) 8 | turtle.pendown() 9 | 10 | for i in range(1, 2 ** 16): 11 | turtle.forward(3) 12 | if ((i & -i) << 1) & i: 13 | turtle.lt(90) 14 | else: 15 | turtle.rt(90) 16 | 17 | turtle.done() 18 | -------------------------------------------------------------------------------- /standard_lib/pathlib_relpath.py: -------------------------------------------------------------------------------- 1 | from itertools import takewhile 2 | from pathlib import Path 3 | 4 | 5 | foo = Path("foo") 6 | baz = Path("baz") 7 | 8 | common_rel_path = Path( 9 | *[ 10 | set(i).pop() 11 | for i in (takewhile(lambda x: x[0] == x[1], zip(foo.parts, baz.parts))) 12 | ] 13 | ) 14 | print(common_rel_path) 15 | -------------------------------------------------------------------------------- /standard_lib/compare_strings.py: -------------------------------------------------------------------------------- 1 | from difflib import SequenceMatcher 2 | 3 | 4 | s1 = "This is my string" 5 | s2 = "This is my new string" 6 | 7 | s = SequenceMatcher(None, s1, s2) 8 | 9 | print(round(s.ratio(), 3)) 10 | 11 | for block in s.get_matching_blocks(): 12 | a, b, size = block 13 | print(f"a[{a}] and b[{b}] match for {size} elements") 14 | -------------------------------------------------------------------------------- /standard_lib/pass_multiple_dicts.py: -------------------------------------------------------------------------------- 1 | from collections import ChainMap 2 | 3 | 4 | def f(a, b, c, d): 5 | print(a, b, c, d) 6 | 7 | 8 | d1 = dict(a=5, b=6) 9 | d2 = dict(b=7, c=8, d=9) 10 | 11 | # The old may without overwriting previous values 12 | f(**ChainMap(d1, d2)) 13 | 14 | # Since Python 3.5 allowing value overwriting 15 | f(**{**d1, **d2}) 16 | -------------------------------------------------------------------------------- /standard_lib/tree_clone.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | 4 | def tree(directory): 5 | print(f"{directory}") 6 | for path in sorted(directory.rglob("*")): 7 | depth = len(path.relative_to(directory).parts) 8 | spacer = " " * (depth - 1) 9 | print(f"{spacer}\u2514---- {path.name}") 10 | 11 | 12 | tree(Path().cwd()) 13 | -------------------------------------------------------------------------------- /standard_lib/timing_context_manager.py: -------------------------------------------------------------------------------- 1 | import time 2 | 3 | from contextlib import contextmanager 4 | 5 | 6 | @contextmanager 7 | def timing(description: str): 8 | start = time.time() 9 | yield 10 | print(f"{description}: {time.time() - start}") 11 | 12 | 13 | with timing("Time List Comprehension"): 14 | s = [x for x in range(10_000_000)] 15 | -------------------------------------------------------------------------------- /standard_lib/human_readable_bytecode.py: -------------------------------------------------------------------------------- 1 | from struct import unpack 2 | from dis import opmap 3 | 4 | reverse_opmap = {v: k for k, v in opmap.items()} 5 | 6 | 7 | def foo(): 8 | pass 9 | 10 | 11 | foo_code = foo.__code__.co_code 12 | for pos in range(0, len(foo_code), 2): 13 | inst = unpack("BB", foo_code[pos : pos + 2]) 14 | print(f"{reverse_opmap[inst[0]]}: {inst[1]}") 15 | -------------------------------------------------------------------------------- /standard_lib/reduce_memory_consumption_iterator.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | from itertools import repeat 4 | 5 | 6 | lots_of_fours = repeat(4, times=100_000_000) 7 | print(f"Using itertools.repeat: {sys.getsizeof(lots_of_fours)} bytes") 8 | 9 | lots_of_fours = [4] * 100_000_000 10 | print( 11 | f"Using list with 100.000.000 elements: {sys.getsizeof(lots_of_fours) / (1024**2)} MB" 12 | ) 13 | -------------------------------------------------------------------------------- /third_party/count_python_bytes.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | from fs import open_fs 4 | from fs.filesize import traditional 5 | 6 | 7 | fs_url = sys.argv[1] 8 | count = 0 9 | 10 | with open_fs(fs_url) as fs: 11 | for _path, info in fs.walk.info(filter=["*.py"], namespaces=["details"]): 12 | count += info.size 13 | 14 | print(f'There is {traditional(count)} of Python in "{fs_url}"') 15 | -------------------------------------------------------------------------------- /third_party/pytest_rename_class_backwards_compatibility.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | 4 | class Mario: 5 | pass 6 | 7 | 8 | class Luigi: 9 | pass 10 | 11 | 12 | @pytest.fixture(params=[Mario, Luigi]) 13 | def plumber(request): 14 | plumber_cls = request.param 15 | 16 | return plumber_cls() 17 | 18 | 19 | def test_backwards_compatibility(plumber): 20 | assert 1 == 1 21 | -------------------------------------------------------------------------------- /standard_lib/emulate_switch_case.py: -------------------------------------------------------------------------------- 1 | import operator 2 | 3 | 4 | dispatch_dict = { 5 | "add": operator.add, 6 | "sub": operator.sub, 7 | "mul": operator.mul, 8 | "div": operator.floordiv, 9 | } 10 | 11 | 12 | def dispatch_func(op, x, y): 13 | return dispatch_dict.get(op, lambda: None)(x, y) 14 | 15 | 16 | print(f"dispatch_func('mul', 4, 5) = {dispatch_func('mul', 4, 5)}") 17 | -------------------------------------------------------------------------------- /third_party/complex_list_ops.py: -------------------------------------------------------------------------------- 1 | from blist import blist 2 | from boxx import timeit 3 | 4 | 5 | def complex_list_ops(x: list) -> list: 6 | x *= 2**29 7 | x.append(5) 8 | y = x[4:-234234] 9 | del x[3:1024] 10 | return x 11 | 12 | 13 | with timeit(name="Builtin"): 14 | y = complex_list_ops([0]) 15 | 16 | with timeit(name="Blist"): 17 | y = complex_list_ops(blist([0])) 18 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # vscode 2 | .vscode/ 3 | 4 | # wily 5 | .wily/ 6 | 7 | # idea 8 | .idea/ 9 | 10 | #__pycache__ 11 | __pycache__/ 12 | */__pycache__/ 13 | 14 | # large data sets 15 | third_party/data/indicators.csv 16 | 17 | # pytest 18 | .pytest_cache/ 19 | */.pytest_cache/ 20 | 21 | # ebook 22 | build.sh 23 | *.aux 24 | *.dvi 25 | *.fdb_latexmk 26 | *.fls 27 | *.log 28 | *.out 29 | *.synctex.gz 30 | *.toc 31 | -------------------------------------------------------------------------------- /standard_lib/disassemble_bytecode.py: -------------------------------------------------------------------------------- 1 | import dis 2 | 3 | 4 | def f_string(number: int): 5 | return f"{number}" 6 | 7 | 8 | def str_string(number: int): 9 | return str(number) 10 | 11 | 12 | print("============================== f-strings ==============================") 13 | dis.dis(f_string) 14 | print("================================ str() ================================") 15 | dis.dis(str_string) 16 | -------------------------------------------------------------------------------- /third_party/tensorflow_image.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import matplotlib.pyplot as plt 3 | import numpy as np 4 | from PIL import Image 5 | 6 | 7 | randomInt = np.random.randint(256, size=(1000, 1000)) 8 | randomFloat = tf.to_float(randomInt) 9 | 10 | with tf.Session() as session: 11 | result = session.run(randomFloat) 12 | im = Image.fromarray(result, "RGB") 13 | 14 | plt.imshow(im) 15 | plt.show() 16 | -------------------------------------------------------------------------------- /standard_lib/abc_class.py: -------------------------------------------------------------------------------- 1 | from abc import ABCMeta, abstractmethod 2 | 3 | 4 | class BaseClass(metaclass=ABCMeta): 5 | @abstractmethod 6 | def foo(self): 7 | pass 8 | 9 | @abstractmethod 10 | def bar(self): 11 | pass 12 | 13 | 14 | class ConcreteClass(BaseClass): 15 | def foo(self): 16 | pass 17 | 18 | def bar(self): 19 | pass 20 | 21 | 22 | instance = ConcreteClass() 23 | -------------------------------------------------------------------------------- /standard_lib/multi_open_files.py: -------------------------------------------------------------------------------- 1 | from contextlib import ExitStack, contextmanager 2 | 3 | 4 | @contextmanager 5 | def multi_open(paths, mode="r"): 6 | with ExitStack() as stack: 7 | yield [stack.enter_context(open(path, mode)) for path in paths] 8 | 9 | 10 | paths = (f"file_{n}.txt" for n in range(10)) 11 | 12 | with multi_open(paths, "w") as files: 13 | for file in files: 14 | file.write("Enter some text to several files.") 15 | -------------------------------------------------------------------------------- /third_party/animated_graphics.py: -------------------------------------------------------------------------------- 1 | import animatplot as amp 2 | import matplotlib.pyplot as plt 3 | import numpy as np 4 | 5 | 6 | x = np.linspace(0, 1, 50) 7 | t = np.linspace(0, 1, 20) 8 | 9 | X, T = np.meshgrid(x, t) 10 | Y = np.sin(2 * np.pi * (X + T)) 11 | 12 | block = amp.blocks.Line(X, Y) 13 | anim = amp.Animation([block]) 14 | 15 | anim.controls() # creates a timeline_slider and a play/pause toggle 16 | anim.save_gif("images/line2") # save animation for docs 17 | plt.show() 18 | -------------------------------------------------------------------------------- /third_party/reduce_pandas_df_memory.py: -------------------------------------------------------------------------------- 1 | """The used data set can be found here: https://www.kaggle.com/worldbank/world-development-indicators#Indicators.csv""" 2 | import pandas as pd 3 | 4 | 5 | df = pd.read_csv("data/indicators.csv") 6 | print("========== Before (with object) ==========") 7 | print(df.info()) 8 | 9 | for column in df.select_dtypes("object"): 10 | df[column] = df[column].astype("category") 11 | 12 | print("========== After (with category) ==========") 13 | print(df.info()) 14 | -------------------------------------------------------------------------------- /standard_lib/flatten.py: -------------------------------------------------------------------------------- 1 | from collections.abc import Iterable 2 | 3 | 4 | def flatten(input_arr, output_arr=None): 5 | if output_arr is None: 6 | output_arr = [] 7 | for ele in input_arr: 8 | if isinstance(ele, Iterable): 9 | flatten(ele, output_arr) # tail-recursion 10 | else: 11 | output_arr.append(ele) # produce the result 12 | 13 | return output_arr 14 | 15 | 16 | sample_list = [1, [2], [[3, 4], 5], 6] 17 | print(flatten(sample_list)) 18 | -------------------------------------------------------------------------------- /third_party/src/text.txt: -------------------------------------------------------------------------------- 1 | The titular threat of The Blob has always struck me as the ultimate movie 2 | monster: an insatiably hungry, amoeba-like mass able to penetrate 3 | virtually any safeguard, capable of--as a doomed doctor chillingly 4 | describes it--"assimilating flesh on contact. 5 | Snide comparisons to gelatin be damned, it's a concept with the most 6 | devastating of potential consequences, not unlike the grey goo scenario 7 | proposed by technological theorists fearful of 8 | artificial intelligence run rampant. 9 | -------------------------------------------------------------------------------- /standard_lib/capture_output.py: -------------------------------------------------------------------------------- 1 | import contextlib 2 | import io 3 | import sys 4 | 5 | 6 | # Option 1 7 | output_stream = io.StringIO() 8 | with contextlib.redirect_stdout(output_stream): 9 | help(pow) 10 | 11 | output_stream = output_stream.getvalue() 12 | print("value:", output_stream) 13 | 14 | 15 | # Option 2 16 | with open("help.txt", "w") as f: 17 | with contextlib.redirect_stdout(f): 18 | help(pow) 19 | 20 | 21 | # Option 3 22 | with contextlib.redirect_stdout(sys.stderr): 23 | help(pow) 24 | -------------------------------------------------------------------------------- /standard_lib/save_dict_update_without_loosing_original.py: -------------------------------------------------------------------------------- 1 | from collections import ChainMap 2 | 3 | 4 | population = {"italy": 60, "japan": 127, "uk": 65} 5 | 6 | changes = dict() 7 | editable = ChainMap(changes, population) 8 | 9 | print(f"Before update: {editable['japan']}") 10 | 11 | editable["japan"] += 1 12 | print(f"Updated population: {editable['japan']}") 13 | print(f"Original population: {population['japan']}") 14 | 15 | print(f"Changes: {changes.keys()}") 16 | print(f"Did not changed: {population.keys() - changes.keys()}") 17 | -------------------------------------------------------------------------------- /standard_lib/bit_flipper.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | # https://forum.omz-software.com/topic/2943/trying-to-make-an-encrypted-message-system 4 | # a poor man's encryption 5 | 6 | 7 | def bit_flipper(s, salt=1): 8 | return "".join([chr(ord(x) ^ salt) for x in s]) 9 | 10 | 11 | salt = 1 # try 1, 6, 7 12 | # for instance, salt = 2 gives you an encrypted string with no printable chars 13 | # (disappearing ink)! 14 | s = "Pythonista rules!" 15 | print(s) 16 | s = bit_flipper(s, salt) 17 | print(s) 18 | s = bit_flipper(s, salt) 19 | print(s) 20 | -------------------------------------------------------------------------------- /standard_lib/priority_queue.py: -------------------------------------------------------------------------------- 1 | import time 2 | import heapq 3 | 4 | 5 | class PriorityQueue: 6 | def __init__(self): 7 | self._q = [] 8 | 9 | def add(self, value, priority=0): 10 | heapq.heappush(self._q, (priority, time.time(), value)) 11 | 12 | def pop(self): 13 | return heapq.heappop(self._q)[-1] 14 | 15 | 16 | f1 = lambda: print("hello") 17 | f2 = lambda: print("world") 18 | 19 | pq = PriorityQueue() 20 | pq.add(f2, priority=1) 21 | pq.add(f1, priority=0) 22 | 23 | print(pq.pop()()) 24 | print(pq.pop()()) 25 | -------------------------------------------------------------------------------- /standard_lib/provide_default_config_values.py: -------------------------------------------------------------------------------- 1 | from collections import ChainMap 2 | 3 | 4 | def main(): 5 | command_line_args = get_command_line_args() 6 | config_file = get_config() 7 | default_config = get_default_config() 8 | 9 | config_dict = ChainMap(command_line_args, config_file, default_config) 10 | 11 | 12 | def get_command_line_args() -> dict: 13 | pass 14 | 15 | 16 | def get_config() -> dict: 17 | pass 18 | 19 | 20 | def get_default_config() -> dict: 21 | pass 22 | 23 | 24 | if __name__ == "__main__": 25 | main() 26 | -------------------------------------------------------------------------------- /standard_lib/scopes_namespaces.py: -------------------------------------------------------------------------------- 1 | def scope_test(): 2 | def do_local(): 3 | spam = "local spam" 4 | 5 | def do_nonlocal(): 6 | nonlocal spam 7 | spam = "nonlocal spam" 8 | 9 | def do_global(): 10 | global spam 11 | spam = "global spam" 12 | 13 | spam = "test spam" 14 | do_local() 15 | print("After local assignment:", spam) 16 | do_nonlocal() 17 | print("After nonlocal assignment:", spam) 18 | do_global() 19 | print("After global assignment:", spam) 20 | 21 | 22 | scope_test() 23 | print("In global scope:", spam) 24 | -------------------------------------------------------------------------------- /standard_lib/keep_metadata_on_decorator_usage.py: -------------------------------------------------------------------------------- 1 | from functools import wraps 2 | 3 | 4 | def tags(tag_name): 5 | """Wraps another function""" 6 | 7 | def tags_decorator(func): 8 | @wraps(func) 9 | def func_wrapper(name): 10 | return f"<{tag_name}>{func(name)}" 11 | 12 | return func_wrapper 13 | 14 | return tags_decorator 15 | 16 | 17 | @tags("p") 18 | def get_text(name): 19 | """Returns some text""" 20 | return "Hello " + name 21 | 22 | 23 | print(get_text("World")) 24 | 25 | print(get_text.__name__) 26 | print(get_text.__doc__) 27 | -------------------------------------------------------------------------------- /third_party/world_bank_data.py: -------------------------------------------------------------------------------- 1 | import csv 2 | import io 3 | import zipfile 4 | 5 | import requests 6 | 7 | 8 | url = "http://databank.worldbank.org/data/download/WDI_csv.zip" 9 | filename = "WDIData.csv" 10 | 11 | # Warning: this can take two minutes to download!! 12 | with zipfile.ZipFile(io.BytesIO(requests.get(url).content)) as zip_file: 13 | print("\n".join(name for name in zip_file.namelist())) 14 | zip_file.extractall() 15 | 16 | # On Python 2, remove: ", newline=''" 17 | with open(filename, newline="") as in_file: 18 | for row in csv.reader(in_file): 19 | print(", ".join(row)) 20 | -------------------------------------------------------------------------------- /third_party/clint_cli_tool.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | 4 | from clint.arguments import Args 5 | from clint.textui import puts, colored, indent 6 | 7 | sys.path.insert(0, os.path.abspath("..")) 8 | 9 | args = Args() 10 | 11 | with indent(4, quote=">>>"): 12 | puts(colored.blue("Arguments passed in: ") + str(args.all)) 13 | puts(colored.blue("Flags detected: ") + str(args.flags)) 14 | puts(colored.blue("Files detected: ") + str(args.files)) 15 | puts(colored.blue("NOT Files detected: ") + str(args.not_files)) 16 | puts(colored.blue("Grouped Arguments: ") + str(dict(args.grouped))) 17 | 18 | print() 19 | -------------------------------------------------------------------------------- /third_party/test_gevent.py: -------------------------------------------------------------------------------- 1 | import gevent.monkey 2 | 3 | from urllib.request import urlopen 4 | from urllib.error import URLError 5 | 6 | 7 | gevent.monkey.patch_all() 8 | urls = ["http://www.google.com", "http://www.yandex.ru", "http://www.python.org"] 9 | 10 | 11 | def print_head(url): 12 | print("Starting {}".format(url)) 13 | try: 14 | data = urlopen(url).read() 15 | except URLError: 16 | data = "None" 17 | finally: 18 | print("{}: {} bytes: {}".format(url, len(data), data)) 19 | 20 | 21 | jobs = [gevent.spawn(print_head, _url) for _url in urls] 22 | 23 | gevent.wait(jobs) 24 | -------------------------------------------------------------------------------- /ebook/chapters/third_party.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../python-snippets.tex 2 | 3 | \chapter{Third Party} 4 | 5 | What distinguishs the recipes in this chapter from the ones in the \textit{Standard Library} chapter? 6 | The ones in this chapter make use of third party packages you need to install first. 7 | You can find the used code snippets in the \textit{third\_party} directory of the repository. 8 | 9 | \input{chapters/third_party_chapters/image_animation.tex} 10 | \input{chapters/third_party_chapters/commandline.tex} 11 | \input{chapters/third_party_chapters/timing.tex} 12 | \input{chapters/third_party_chapters/non_categorized.tex} 13 | -------------------------------------------------------------------------------- /third_party/port_scanner_nmap.py: -------------------------------------------------------------------------------- 1 | import nmap 2 | 3 | nm = nmap.PortScanner() 4 | nm.scan("127.0.0.1", "22-443") 5 | 6 | for host in nm.all_hosts(): 7 | print("----------------------------------------------------") 8 | print(f"Host : {host} ({nm[host].hostname()})") 9 | print(f"State : {nm[host].state()}") 10 | 11 | for protocol in nm[host].all_protocols(): 12 | print("----------") 13 | print(f"Protocol : {protocol}") 14 | port_list = sorted(nm[host][protocol].keys()) 15 | 16 | for port in port_list: 17 | print(f"port : {port}\tstate : {nm[host][protocol][port]['state']}") 18 | -------------------------------------------------------------------------------- /third_party/test_tornado.py: -------------------------------------------------------------------------------- 1 | import tornado.ioloop 2 | from tornado.httpclient import AsyncHTTPClient 3 | 4 | 5 | urls = ["http://www.google.com", "http://www.yandex.ru", "http://www.python.org"] 6 | 7 | 8 | def handle_response(response): 9 | if response.error: 10 | print("Error:", response.error) 11 | else: 12 | url = response.request.url 13 | data = response.body 14 | print("{}: {} bytes: {}".format(url, len(data), data)) 15 | 16 | 17 | http_client = AsyncHTTPClient() 18 | for url in urls: 19 | http_client.fetch(url, handle_response) 20 | 21 | tornado.ioloop.IOLoop.instance().start() 22 | -------------------------------------------------------------------------------- /standard_lib/trace_decorator.py: -------------------------------------------------------------------------------- 1 | import functools 2 | 3 | 4 | def trace(func): 5 | @functools.wraps(func) 6 | def wrapper(*args, **kwargs): 7 | print("-" * 20) 8 | print(f"TRACE: calling {func.__name__}() " f"with {args}, {kwargs}") 9 | 10 | original_result = func(*args, **kwargs) 11 | 12 | print(f"TRACE: {func.__name__}() " f"returned {original_result!r}") 13 | print("-" * 20) 14 | 15 | return original_result 16 | 17 | return wrapper 18 | 19 | 20 | @trace 21 | def greet(name, phrase): 22 | return f"{name}, {phrase}" 23 | 24 | 25 | print(greet("Florian", "Nice to see you!")) 26 | -------------------------------------------------------------------------------- /standard_lib/function_arguments.py: -------------------------------------------------------------------------------- 1 | def main(): 2 | positional("Cheese", 21, "Meat") 3 | optional_keyword("Cheese", 21, "Meat") 4 | optional_keyword("Cheese", 21, arg3="Meat") 5 | force_keyword("Cheese", 21, arg3="Meat") 6 | force_keyword("Cheese", 21, "Meat") 7 | 8 | 9 | def positional(arg1: str, arg2: int, arg3: str): 10 | print(arg1, arg2, arg3) 11 | 12 | 13 | def optional_keyword(arg1: str, arg2: int, arg3: str = None): 14 | print(arg1, arg2, arg3) 15 | 16 | 17 | def force_keyword(arg1: str, arg2: int, *, arg3: str = None): 18 | print(arg1, arg2, arg3) 19 | 20 | 21 | if __name__ == "__main__": 22 | main() 23 | -------------------------------------------------------------------------------- /third_party/manipulate_images.py: -------------------------------------------------------------------------------- 1 | import imageio 2 | import numpy as np 3 | import scipy.ndimage 4 | 5 | 6 | start_img = imageio.imread( 7 | "http://static.cricinfo.com/db/PICTURES/CMS/263600/263697.20.jpg" 8 | ) 9 | gray_inv_img = 255 - np.dot(start_img[..., :3], [0.299, 0.587, 0.114]) 10 | blur_img = scipy.ndimage.filters.gaussian_filter(gray_inv_img, sigma=5) 11 | 12 | 13 | def dodge(front, back): 14 | result = front * 255 / (255 - back) 15 | result[np.logical_or(result > 255, back == 255)] = 255 16 | return result.astype("uint8") 17 | 18 | 19 | final_img = dodge(blur_img, gray_inv_img) 20 | imageio.imwrite("final.jpg", final_img) 21 | -------------------------------------------------------------------------------- /third_party/cli_help_strings.py: -------------------------------------------------------------------------------- 1 | """CLI HELP STRINGS 2 | Usage: 3 | cli_help_strings.py 4 | cli_help_strings.py 5 | cli_help_strings.py -h|--help 6 | cli_help_strings.py -v|--version 7 | Options: 8 | Optional name argument. 9 | -h --help Show this screen. 10 | -v --version Show version. 11 | """ 12 | from docopt import docopt 13 | 14 | 15 | def say_hello(name): 16 | return f"Hello {name}!" 17 | 18 | 19 | if __name__ == "__main__": 20 | arguments = docopt(__doc__, version="DEMO 1.0") 21 | if arguments[""]: 22 | print(say_hello(arguments[""])) 23 | else: 24 | print(arguments) 25 | -------------------------------------------------------------------------------- /standard_lib/hash_file.py: -------------------------------------------------------------------------------- 1 | """Reference: https://medium.com/ediblesec/building-a-hashing-tool-with-python-3afe34db74e5""" 2 | import os 3 | import hashlib 4 | import argparse 5 | 6 | parser = argparse.ArgumentParser() 7 | parser.add_argument("file") 8 | args = parser.parse_args() 9 | 10 | md5 = hashlib.md5() 11 | sha1 = hashlib.sha1() 12 | 13 | try: 14 | with open(args.file, "rb") as f: 15 | buf = f.read() 16 | md5.update(buf) 17 | sha1.update(buf) 18 | 19 | print(f"Filename: {os.path.basename(args.file)}") 20 | print(f"MD5-Hash: {md5.hexdigest()}") 21 | print(f"SHA1-Hash: {sha1.hexdigest()}") 22 | except FileNotFoundError as e: 23 | print(e) 24 | -------------------------------------------------------------------------------- /third_party/test_asyncio.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import aiohttp 3 | import async_timeout 4 | 5 | 6 | urls = ["http://www.google.com", "http://www.yandex.ru", "http://www.python.org"] 7 | 8 | 9 | async def call_url(url): 10 | print("Starting {}".format(url)) 11 | response = None 12 | async with aiohttp.ClientSession() as session: 13 | async with async_timeout.timeout(10): 14 | async with session.get(url) as response: 15 | data = await response.text() 16 | print("{}: {} bytes: {}".format(url, len(data), data)) 17 | return data 18 | 19 | 20 | futures = [call_url(url) for url in urls] 21 | 22 | loop = asyncio.get_event_loop() 23 | loop.run_until_complete(asyncio.wait(futures)) 24 | -------------------------------------------------------------------------------- /third_party/auto_login_website.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | 3 | 4 | website_link = "https://github.com/login" 5 | username = "" 6 | password = "" 7 | 8 | element_for_username = "login" 9 | element_for_password = "password" 10 | element_for_submit = "commit" 11 | 12 | browser = webdriver.Firefox() 13 | browser.get(website_link) 14 | 15 | try: 16 | username_field = browser.find_element_by_name(element_for_username) 17 | username_field.send_keys(username) 18 | password_field = browser.find_element_by_name(element_for_password) 19 | password_field.send_keys(password) 20 | signInButton = browser.find_element_by_name(element_for_submit) 21 | signInButton.click() 22 | except Exception as e: 23 | print(e) 24 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/search_algorithms.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Search Algorithms} 4 | 5 | This section contains implementations of certain search algorithms. 6 | 7 | 8 | \subsection{Binary Search} 9 | 10 | The binary search algortihm is a search algorithm, that finds the position of a target value within a sorted array. 11 | It compares the target value to the middle element of the array. 12 | If they are not equal, the half in which the target cannot lie is eliminated and the search continues on the remaining half, again taking the middle element to compare to the target value, and repeating this until the target value is found. 13 | 14 | \lstinputlisting[caption=binary\_search.py]{../standard_lib/binary_search.py} 15 | -------------------------------------------------------------------------------- /standard_lib/temptable_contextmanager.py: -------------------------------------------------------------------------------- 1 | from sqlite3 import connect 2 | from contextlib import contextmanager 3 | 4 | 5 | @contextmanager 6 | def temptable(cur): 7 | cur.execute("create table points(x int, y int)") 8 | try: 9 | yield 10 | finally: 11 | cur.execute("drop table points") 12 | 13 | 14 | with connect("test.db") as conn: 15 | cur = conn.cursor() 16 | with temptable(cur): 17 | cur.execute("insert into points (x, y) values(1, 1)") 18 | cur.execute("insert into points (x, y) values(1, 2)") 19 | cur.execute("insert into points (x, y) values(2, 1)") 20 | for row in cur.execute("select x, y from points"): 21 | print(row) 22 | for row in cur.execute("select sum(x * y) from points"): 23 | print(row) 24 | -------------------------------------------------------------------------------- /standard_lib/port_scanner.py: -------------------------------------------------------------------------------- 1 | """Reference: https://medium.com/ediblesec/building-a-port-scanner-in-16-lines-of-code-26793f53f0b5""" 2 | import argparse 3 | import socket 4 | import sys 5 | 6 | from datetime import datetime 7 | 8 | parser = argparse.ArgumentParser() 9 | parser.add_argument("host") 10 | args = parser.parse_args() 11 | 12 | t1 = datetime.now() 13 | 14 | try: 15 | for port in range(1, 1025): 16 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 17 | sock.settimeout(1) 18 | result = sock.connect_ex((args.host, port)) 19 | if result == 0: 20 | print("Port: {} Open".format(port)) 21 | sock.close() 22 | except KeyboardInterrupt: 23 | sys.exit() 24 | 25 | t2 = datetime.now() 26 | print("Scanning completed in: {}".format(t2 - t1)) 27 | -------------------------------------------------------------------------------- /standard_lib/async_sleep_sync.py: -------------------------------------------------------------------------------- 1 | """ 2 | Async snippet demonstrating the usage of time.sleep() 3 | """ 4 | import asyncio 5 | import time 6 | from datetime import datetime 7 | 8 | 9 | async def custom_sleep(): 10 | print("SLEEP", datetime.now()) 11 | time.sleep(1) 12 | 13 | 14 | async def factorial(name, number): 15 | f = 1 16 | for i in range(2, number + 1): 17 | print(f"Task {name}: Compute factorial({i})") 18 | await custom_sleep() 19 | f *= i 20 | print(f"Task {name}: factorial({number}) is {i}\n") 21 | 22 | 23 | start = time.time() 24 | loop = asyncio.get_event_loop() 25 | 26 | tasks = [ 27 | asyncio.ensure_future(factorial("A", 3)), 28 | asyncio.ensure_future(factorial("B", 4)), 29 | ] 30 | loop.run_until_complete(asyncio.wait(tasks)) 31 | loop.close() 32 | 33 | end = time.time() 34 | print(f"Total time: {end - start}") 35 | -------------------------------------------------------------------------------- /standard_lib/async_sleep_async.py: -------------------------------------------------------------------------------- 1 | """ 2 | Async snippet demonstarting the usage of asyncio.sleep() 3 | """ 4 | import asyncio 5 | import time 6 | from datetime import datetime 7 | 8 | 9 | async def custom_sleep(): 10 | print(f"SLEEP {datetime.now()}\n") 11 | await asyncio.sleep(1) 12 | 13 | 14 | async def factorial(name, number): 15 | f = 1 16 | for i in range(2, number + 1): 17 | print(f"Task {name}: Compute factorial({i})") 18 | await custom_sleep() 19 | f *= i 20 | print(f"Task {name}: factorial({number}) is {f}\n") 21 | 22 | 23 | start = time.time() 24 | loop = asyncio.get_event_loop() 25 | 26 | tasks = [ 27 | asyncio.ensure_future(factorial("A", 3)), 28 | asyncio.ensure_future(factorial("B", 4)), 29 | ] 30 | loop.run_until_complete(asyncio.wait(tasks)) 31 | loop.close() 32 | 33 | end = time.time() 34 | print(f"Total time: {end - start}") 35 | -------------------------------------------------------------------------------- /standard_lib/binary_search.py: -------------------------------------------------------------------------------- 1 | import random 2 | 3 | 4 | def binary_search(lst, integer): 5 | 6 | lower_bound = 0 7 | upper_bound = len(lst) - 1 8 | number_guesses = 0 9 | 10 | while True: 11 | index = (lower_bound + upper_bound) // 2 12 | 13 | guess = lst[index] 14 | number_guesses += 1 15 | 16 | if guess == integer: 17 | break 18 | 19 | if guess > integer: 20 | upper_bound = index - 1 21 | else: 22 | lower_bound = index + 1 23 | 24 | return number_guesses 25 | 26 | 27 | print("Number Guesses") 28 | 29 | for _ in range(30): 30 | integers = random.sample(range(50000), 10000) 31 | 32 | integer_to_find = random.choice(integers) 33 | 34 | sorted_integers = sorted(integers) 35 | 36 | guesses = binary_search(sorted_integers, integer_to_find) 37 | 38 | print("{:6} {:7}".format(integer_to_find, guesses)) 39 | -------------------------------------------------------------------------------- /standard_lib/bitmask.py: -------------------------------------------------------------------------------- 1 | #! /bin/env python3 2 | 3 | 4 | class BitMask(int): 5 | def AND(self, bm): 6 | return BitMask(self & bm) 7 | 8 | def OR(self, bm): 9 | return BitMask(self | bm) 10 | 11 | def XOR(self, bm): 12 | return BitMask(self ^ bm) 13 | 14 | def NOT(self): 15 | return BitMask(~self) 16 | 17 | def shiftleft(self, num): 18 | return BitMask(self << num) 19 | 20 | def shiftright(self, num): 21 | return BitMask(self > num) 22 | 23 | def bit(self, num): 24 | mask = 1 << num 25 | return bool(self & mask) 26 | 27 | def setbit(self, num): 28 | mask = 1 << num 29 | return BitMask(self | mask) 30 | 31 | def zerobit(self, num): 32 | mask = ~(1 << num) 33 | return BitMask(self & mask) 34 | 35 | def listbits(self, start=0, end=-1): 36 | end = end if end < 0 else end + 2 37 | return [int(c) for c in bin(self)[start + 2 : end]] 38 | -------------------------------------------------------------------------------- /standard_lib/deprecated_decorator.py: -------------------------------------------------------------------------------- 1 | import warnings 2 | 3 | 4 | def deprecated(func): 5 | """This is a decorator which can be used to wmark functions as deprecated. 6 | It will result in a warning being emitted when the function is used.""" 7 | 8 | def new_func(*args, **kwargs): 9 | warnings.warn( 10 | f"Call to deprecated function {func.__name__}.", category=DeprecationWarning 11 | ) 12 | 13 | return func(*args, **kwargs) 14 | 15 | new_func.__name__ = func.__name__ 16 | new_func.__doc__ = func.__doc__ 17 | new_func.__dict__.update(func.__dict__) 18 | 19 | return new_func 20 | 21 | 22 | # === Example usages === 23 | 24 | 25 | @deprecated 26 | def some_old_function(x, y): 27 | return x + y 28 | 29 | 30 | class SomeClass: 31 | @deprecated 32 | def some_old_method(self, x, y): 33 | return x + y 34 | 35 | 36 | some_old_function(1, 2) 37 | example = SomeClass() 38 | example.some_old_method(1, 2) 39 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/abc.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Abstract Base Class (ABC)} 4 | 5 | The \lstinline{abc} module provides the infrastructure for defining abstract base classes (ABCs) in Python. 6 | In this section you will find a collection of snippets being useful when dealing with ABCs. 7 | 8 | 9 | \subsection{General Usage} 10 | 11 | Listing~\ref{lst:abcclass} shows the general usage of ABCs by example. 12 | 13 | \lstinputlisting[caption={abc\_class.py},label={lst:abcclass}]{../standard_lib/abc_class.py} 14 | 15 | The snippets contains two classes: \lstinline{BaseClass} and \lstinline{ConcreteClass}, whereas \lstinline{BaseClass} is the ABC of \linebreak\lstinline{ConcreteClass}. 16 | Both methods \lstinline{foo} and \lstinline{bar} are decorated with \lstinline{@abstractmethod} and have to be implemented by the classes inheriting from \lstinline{BaseClass}. 17 | Furthermore, as \lstinline{BaseClass} is an ABC, it can't be instantiated. 18 | -------------------------------------------------------------------------------- /third_party/refactoring_code/refactoring_code.py: -------------------------------------------------------------------------------- 1 | from rope.refactor import change_signature 2 | import testutils 3 | 4 | 5 | project = testutils.sample_project() 6 | pycore = project.pycore 7 | mod = testutils.create_module(project, "mod") 8 | 9 | code = ( 10 | "class A(object):\n" 11 | " def a_func(self, param):\n" 12 | " pass\n" 13 | "a_var = A()\n" 14 | "a_var.a_func(param=1)\n" 15 | ) 16 | 17 | mod.write(code) 18 | signature = change_signature.ChangeSignature( 19 | project, mod, mod.read().index("a_func") + 1 20 | ) 21 | project.do(signature.get_changes([change_signature.ArgumentNormalizer()])) 22 | expected = ( 23 | "class A(object):\n" 24 | " def a_func(self, param):\n" 25 | " pass\n" 26 | "a_var = A()\n" 27 | "a_var.a_func(1)\n" 28 | ) 29 | 30 | print("==================== Before Refactoring ====================") 31 | print(code) 32 | print("==================== After Refactoring =====================") 33 | print(mod.read()) 34 | -------------------------------------------------------------------------------- /third_party/mocking_requests.py: -------------------------------------------------------------------------------- 1 | from unittest.mock import Mock 2 | 3 | import requests 4 | 5 | 6 | requests = Mock() 7 | 8 | 9 | def get_holidays(): 10 | r = requests.get("http://localhost/api/holidays") 11 | if r.status_code == 200: 12 | return r.json() 13 | return None 14 | 15 | 16 | class TestCalendar: 17 | def log_request(self, url): 18 | # Log a fake request for test output purposes 19 | print(f"Making a request to {url}.") 20 | print("Request received!") 21 | 22 | # Create a new Mock to imitate a Response 23 | response_mock = Mock() 24 | response_mock.status_code = 200 25 | response_mock.json.return_value = { 26 | "12/25": "Christmas", 27 | "7/4": "Independence Day", 28 | } 29 | 30 | return response_mock 31 | 32 | def test_get_holidays_logging(self): 33 | requests.get.side_effect = self.log_request 34 | print(requests) 35 | assert get_holidays()["12/25"] == "Christmas" 36 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../python-snippets.tex 2 | 3 | \chapter{Standard Library} 4 | 5 | This chapter contains the Python snippets from the \textit{standard\_lib} directory. 6 | Additionally, explanations are added and possible usages are described. 7 | 8 | 9 | \input{chapters/standard_lib_chapters/abc.tex} 10 | \input{chapters/standard_lib_chapters/async.tex} 11 | \input{chapters/standard_lib_chapters/playing_bits.tex} 12 | \input{chapters/standard_lib_chapters/search_algorithms.tex} 13 | \input{chapters/standard_lib_chapters/dealing_builtins.tex} 14 | \input{chapters/standard_lib_chapters/string_operations.tex} 15 | \input{chapters/standard_lib_chapters/dicts.tex} 16 | \input{chapters/standard_lib_chapters/decorators.tex} 17 | \input{chapters/standard_lib_chapters/bytecode.tex} 18 | \input{chapters/standard_lib_chapters/lists.tex} 19 | \input{chapters/standard_lib_chapters/files.tex} 20 | \input{chapters/standard_lib_chapters/context_manager.tex} 21 | \input{chapters/standard_lib_chapters/non_categorized.tex} 22 | -------------------------------------------------------------------------------- /third_party/pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "third_party" 3 | version = "0.1.0" 4 | description = "" 5 | authors = ["Florian Dahlitz "] 6 | license = "MIT" 7 | 8 | [tool.poetry.dependencies] 9 | python = "^3.6" 10 | folium = "*" 11 | pandas = "*" 12 | imageio = "*" 13 | numpy = "*" 14 | scipy = "*" 15 | matplotlib = "*" 16 | animatplot = "*" 17 | asyncio = "*" 18 | boxx = "*" 19 | click = "*" 20 | termcolor = "*" 21 | python-nmap = "*" 22 | pyinquirer = "*" 23 | pyfiglet = "*" 24 | clint = "*" 25 | docopt = "*" 26 | tqdm = "*" 27 | opencv-python = "*" 28 | aiohttp = "*" 29 | gevent = "*" 30 | tornado = "*" 31 | beautifulsoup4 = "*" 32 | holidays = "*" 33 | selenium = "*" 34 | docker = "*" 35 | progressbar = "*" 36 | rope = "*" 37 | mypy = "*" 38 | prettytable = "*" 39 | fs = "*" 40 | pytest = "*" 41 | requests = "*" 42 | xlrd = "*" 43 | pysnooper = "*" 44 | cooked-input = "*" 45 | textblob = "*" 46 | blist = "*" 47 | 48 | [tool.poetry.dev-dependencies] 49 | 50 | [build-system] 51 | requires = ["poetry>=0.12"] 52 | build-backend = "poetry.masonry.api" 53 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Florian Dahlitz 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 6 | 7 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 10 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/async.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{AsyncIO} 4 | 5 | Since Python~3.5 the module \lstinline{asyncio} provides the keywords \lstinline{async} and \lstinline{await}. 6 | This section provides some useful snippets when operating with them. 7 | 8 | 9 | \subsection{Time Saving} 10 | 11 | Using \lstinline{async} and \lstinline{await} will save you some time especially when requesting data from the internet. 12 | To keep things simple let's simulate requests against a certain API with \lstinline{time.sleep()}. 13 | Listing~\ref{lst:asyncsleep} shows a sample program. 14 | 15 | \lstinputlisting[caption=async\_sleep\_sync.py,label=lst:asyncsleep]{../standard_lib/async_sleep_sync.py} 16 | 17 | Running this snippet takes around five seconds. 18 | Let's modify it a little bit to run the tasks asynchronously, which is shown in Listing~\ref{lst:asyncsleepmodified}. 19 | 20 | \lstinputlisting[caption=async\_sleep\_async.py,label=lst:asyncsleepmodified]{../standard_lib/async_sleep_async.py} 21 | 22 | After the modifications it lasts around three seconds. 23 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/playing_bits.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Playing with Bits} 4 | 5 | In this section you will find a collection of snippets corresponding to bit manipulation. 6 | 7 | 8 | \subsection{Bit Flipper} 9 | 10 | The \lstinline{bit_flipper} function gives you a poor man's encryption. 11 | Making use of a salt, it flips certain bits of each charcter. 12 | The resulting string hides the information from the original one, at least for humans. 13 | Applying the function a second time returns the original string. 14 | 15 | \lstinputlisting[caption=bit\_flipper.py,label=lst:bitflipper]{../standard_lib/bit_flipper.py} 16 | 17 | The output will look like the following: 18 | 19 | \begin{lstlisting}[caption=Output of the bit flipper] 20 | $ python bit_flipper.py 21 | Pythonista rules! 22 | Qxuinohru`!stmdr 23 | Pythonista rules! 24 | \end{lstlisting} 25 | 26 | 27 | \subsection{A Simple Bitmask} 28 | 29 | The \lstinline{BitMask} class represents a simple bit mask. 30 | It has methods representing all the bitwise operations plus some additional features. 31 | The methods return a new BitMask object or a boolean result. 32 | See the bits module for more information about the operations provided. 33 | 34 | \lstinputlisting[caption=bitmask.py]{../standard_lib/bitmask.py} 35 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python Snippets # 2 | ## Description ## 3 | A collection of useful Python snippets. 4 | 5 | **standard_lib:** Contains those snippets, which do not depend 6 | on third-party packages. 7 | 8 | **third_party:** As the name suggests, it contains those snippets 9 | depending on third-party packages, even if it's only `requests` 10 | or `click`. 11 | 12 | **ebook:** Contains an ebook including the provided snippets and additional explanations and insights. 13 | The ebook is available as `.epub`, `.mobi` and `.pdf`. 14 | Currently `pandoc` doesn't support LaTeX `\ref{...}` handling, which is used in the ebook. 15 | That said, the `epub` and `mobi` version won't show the correct Listing numbers in the text, whereas the `pdf` version does. 16 | 17 | 18 | ## Installation 19 | 20 | The code snippets in the [`third_party`](third_party) directory depend on other packages you need to install. 21 | A `pyproject.toml` and a `poetry.lock` are in the directory, too, as poetry is used for dependency management. 22 | 23 | > **Note:** Due to the changed distribution settings of the library `tensorflow`, you need to install it separately on your system in order to run some of the snippets in this repo. 24 | > `tensorflow` is no longer part of the repositories `pyproject.toml`. 25 | 26 | 27 | ## More Tips & Tricks 28 | 29 | More tips & tricks can be found in the respective section on my website: [link](https://florian-dahlitz.de/tips). 30 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/files.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Files} 4 | 5 | In this section you will find a collection of snippets revealing tips for interacting with files. 6 | 7 | 8 | \subsection{Hash a File} 9 | 10 | To ensure a files integrity, you can hash the file and compare it with other hashes. 11 | The following Listing shows you how to hash a certain file using \lstinline{MD5} and \lstinline{SHA1}. 12 | Both hashes are printed to stdout as well as the name of the hashed file. 13 | 14 | \lstinputlisting[caption=hash\_file.py]{../standard_lib/hash_file.py} 15 | 16 | 17 | \subsection{Read Files using Iterator} 18 | 19 | Reveals the usage of iterators to read in a file. 20 | It's useful when dealing with large files. 21 | If not using iterators, the whole file is loaded into memory at once (think of several gigabyte huge files). 22 | If using iterators, only the next line is loaded. 23 | 24 | \lstinputlisting[caption=read\_files\_using\_iterator.py]{../standard_lib/read_files_using_iterator.py} 25 | 26 | 27 | \subsection{File Matching Using \lstinline{fnmatch}} 28 | 29 | Matching certain strings is easy if you use Pythons built-in \lstinline{fnmatch} module. 30 | It even provides you functionality to translate the easy to use \lstinline{fnmatch} patterns into regular expressions. 31 | The following recipe shows you a sample usage filtering markdown files and \lstinline{git}-related files in your current working directory. 32 | 33 | \lstinputlisting[caption=file\_matching\_regex.py]{../standard_lib/file_matching_regex.py} 34 | -------------------------------------------------------------------------------- /third_party/interactive_cli.py: -------------------------------------------------------------------------------- 1 | from PyInquirer import style_from_dict, Token, prompt, Separator 2 | from pprint import pprint 3 | 4 | 5 | style = style_from_dict( 6 | { 7 | Token.Separator: "#cc5454", 8 | Token.QuestionMark: "#673ab7 bold", 9 | Token.Selected: "#cc5454", # default 10 | Token.Pointer: "#673ab7 bold", 11 | Token.Instruction: "", # default 12 | Token.Answer: "#f44336 bold", 13 | Token.Question: "", 14 | } 15 | ) 16 | 17 | 18 | questions = [ 19 | { 20 | "type": "checkbox", 21 | "message": "Select toppings", 22 | "name": "toppings", 23 | "choices": [ 24 | Separator("= The Meats ="), 25 | {"name": "Ham"}, 26 | {"name": "Ground Meat"}, 27 | {"name": "Bacon"}, 28 | Separator("= The Cheeses ="), 29 | {"name": "Mozzarella", "checked": True}, 30 | {"name": "Cheddar"}, 31 | {"name": "Parmesan"}, 32 | Separator("= The usual ="), 33 | {"name": "Mushroom"}, 34 | {"name": "Tomato"}, 35 | {"name": "Pepperoni"}, 36 | Separator("= The extras ="), 37 | {"name": "Pineapple"}, 38 | {"name": "Olives", "disabled": "out of stock"}, 39 | {"name": "Extra cheese"}, 40 | ], 41 | "validate": lambda answer: "You must choose at least one topping." 42 | if len(answer) == 0 43 | else True, 44 | } 45 | ] 46 | 47 | answers = prompt(questions, style=style) 48 | pprint(answers) 49 | -------------------------------------------------------------------------------- /ebook/python-snippets.tex: -------------------------------------------------------------------------------- 1 | \documentclass[ebook,12pt,oneside,openany]{memoir} 2 | \usepackage[english]{babel} 3 | \usepackage{pmboxdraw} 4 | \usepackage{url} 5 | 6 | \DeclareFixedFont{\ttb}{T1}{txtt}{bx}{n}{12} % for bold 7 | \DeclareFixedFont{\ttm}{T1}{txtt}{m}{n}{12} % for normal 8 | 9 | % Custom colors 10 | \usepackage{color} 11 | \definecolor{deepblue}{rgb}{0,0,0.5} 12 | \definecolor{deepred}{rgb}{0.6,0,0} 13 | \definecolor{deepgreen}{rgb}{0,0.5,0} 14 | 15 | \usepackage{listings} 16 | \lstset{ 17 | aboveskip=2em, 18 | belowskip=2em, 19 | breaklines=true, 20 | captionpos=b, 21 | language=Python, 22 | postbreak=\mbox{\textcolor{red}{$\hookrightarrow$}\space}, 23 | basicstyle=\ttm, 24 | otherkeywords={self}, % Add keywords here 25 | keywordstyle=\ttb\color{deepblue}, 26 | emph={MyClass,__init__}, % Custom highlighting 27 | emphstyle=\ttb\color{deepred}, % Custom highlighting style 28 | stringstyle=\color{deepgreen}, 29 | frame=tb, % Any extra options here 30 | showstringspaces=false % 31 | } 32 | 33 | \usepackage{hyperref} 34 | \hypersetup{ 35 | colorlinks, 36 | citecolor=black, 37 | filecolor=black, 38 | linkcolor=black, 39 | urlcolor=black 40 | } 41 | 42 | \setcounter{tocdepth}{5} 43 | \setcounter{secnumdepth}{5} 44 | 45 | \title{% 46 | Python Snippets \\ 47 | \large A Collection of Useful Python Snippets with Explanations} 48 | \author{Florian Dahlitz} 49 | 50 | \begin{document} 51 | \maketitle 52 | \newpage \tableofcontents 53 | 54 | \newpage \input{chapters/standard_lib.tex} 55 | \newpage \input{chapters/third_party.tex} 56 | 57 | \end{document} 58 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/bytecode.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Bytecode} 4 | 5 | This section contains snippets for investigating Pythons bytecode. 6 | 7 | 8 | \subsection{Disassemble Bytecode - String Conversion} 9 | 10 | Ever wanted to know what a string conversion using f-strings or \lstinline{str()} looks like in bytecode? 11 | The following snippet shows you exactly that! 12 | 13 | \lstinputlisting[caption=disassemble\_bytecode.py]{../standard_lib/disassemble_bytecode.py} 14 | 15 | For displaying the bytecode of a function the builtin \lstinline{dis} module is used. 16 | 17 | \begin{lstlisting}[caption=Output of disassemble\_bytecode.py] 18 | $ python disassemble_bytecode.py 19 | ============================== f-strings ============================== 20 | 5 0 LOAD_FAST 0 (number) 21 | 2 FORMAT_VALUE 0 22 | 4 RETURN_VALUE 23 | ================================ str() ================================ 24 | 9 0 LOAD_GLOBAL 0 (str) 25 | 2 LOAD_FAST 0 (number) 26 | 4 CALL_FUNCTION 1 27 | 6 RETURN_VALUE 28 | \end{lstlisting} 29 | 30 | 31 | \subsection{Human Readable Bytecode} 32 | 33 | If you are using Pythons \lstinline{dis} module to get some insights into what's happening behind the scenes, you may like this recipe as it provides a way to print the output in a much more human readable way. 34 | 35 | \lstinputlisting[caption=human\_readable\_bytecode.py]{../standard_lib/human_readable_bytecode.py} 36 | 37 | \begin{lstlisting}[caption=Output of human\_readable\_bytecode.py] 38 | $ python human_readable_bytecode.py 39 | LOAD_CONST: 0 40 | RETURN_VALUE: 0 41 | \end{lstlisting} 42 | 43 | \textbf{Note:} Even though we didn't put a return statement at the end of \lstinline{foo}, Python added it. 44 | That's because every function in Python needs a return statement (specified by the underlying protocols). 45 | -------------------------------------------------------------------------------- /third_party/img_to_ascii_2.py: -------------------------------------------------------------------------------- 1 | from PIL import Image 2 | 3 | ASCII_CHARS = [ '#', '?', '%', '.', 'S', '+', '.', '*', ':', ',', '@'] 4 | 5 | def scale_image(image, new_width=100): 6 | """Resizes an image preserving the aspect ratio. 7 | """ 8 | (original_width, original_height) = image.size 9 | aspect_ratio = original_height/float(original_width) 10 | new_height = int(aspect_ratio * new_width) 11 | 12 | new_image = image.resize((new_width, new_height)) 13 | return new_image 14 | 15 | def convert_to_grayscale(image): 16 | return image.convert('L') 17 | 18 | def map_pixels_to_ascii_chars(image, range_width=25): 19 | """Maps each pixel to an ascii char based on the range 20 | in which it lies. 21 | 22 | 0-255 is divided into 11 ranges of 25 pixels each. 23 | """ 24 | 25 | pixels_in_image = list(image.getdata()) 26 | pixels_to_chars = [ASCII_CHARS[pixel_value//range_width] for pixel_value in 27 | pixels_in_image] 28 | 29 | return "".join(pixels_to_chars) 30 | 31 | def convert_image_to_ascii(image, new_width=100): 32 | image = scale_image(image) 33 | image = convert_to_grayscale(image) 34 | 35 | pixels_to_chars = map_pixels_to_ascii_chars(image) 36 | len_pixels_to_chars = len(pixels_to_chars) 37 | 38 | image_ascii = [pixels_to_chars[index: index + new_width] for index in 39 | range(0, len_pixels_to_chars, new_width)] 40 | 41 | return "\n".join(image_ascii) 42 | 43 | def handle_image_conversion(image_filepath): 44 | image = None 45 | try: 46 | image = Image.open(image_filepath) 47 | except Exception as e: 48 | print("Unable to open image file {image_filepath}.".format(image_filepath=image_filepath)) 49 | print(e) 50 | return 51 | 52 | image_ascii = convert_image_to_ascii(image) 53 | print(image_ascii) 54 | 55 | if __name__=='__main__': 56 | import sys 57 | 58 | image_file_path = sys.argv[1] 59 | handle_image_conversion(image_file_path) 60 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/context_manager.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Context Manager} 4 | 5 | In this section you will find a collection of self-implemented context manager. 6 | 7 | 8 | \subsection{Open Multiple Files} 9 | 10 | Ever wanted to save a certain text to multiple files? 11 | Well, with this context manager you can open multiple files at once and write a specified text to them. 12 | 13 | \lstinputlisting[caption=multi\_open\_files.py]{../standard_lib/multi_open_files.py} 14 | 15 | If you run the snippet, it will create ten text files all containing the same text. 16 | 17 | 18 | \subsection{Temporal SQLite Table} 19 | 20 | If you are working with \lstinline{sqlite}, you may find this context manager helpful. 21 | It creates a temporal table you can interact with. 22 | 23 | \lstinputlisting[caption=temptable\_contextmanager.py]{../standard_lib/temptable_contextmanager.py} 24 | 25 | When leaving the \lstinline{with}-statement, the table is deleted. 26 | That said, you can run the snippet as often as you like, the output remains the same. 27 | 28 | \begin{lstlisting}[caption=Output of temptable\_contextmanager.py] 29 | $ python temptable_contextmanager.py 30 | (1, 1) 31 | (1, 2) 32 | (2, 1) 33 | (5,) 34 | \end{lstlisting} 35 | 36 | 37 | \subsection{Timing Context Manager} 38 | 39 | Sometimes you simply want to find out, how long a certain code block needs to be executed. 40 | To do that you have different options: Use a third-party package like \lstinline{boxx} or create your own timing context manager based on the standard library. 41 | The following Listing shows you, how you can create your own timing context manager. 42 | In specific, this approach makes use of a generator function (you can implement this context manager based on a class as well). 43 | 44 | \lstinputlisting[caption=timing\_context\_manager.py]{../standard_lib/timing_context_manager.py} 45 | 46 | As you can see, the code is pretty straightforward. 47 | An example output is shown below. 48 | 49 | \begin{lstlisting}[caption=Output of timing\_context\_manager.py] 50 | $ python timing_context_manager.py 51 | Time List Comprehension: 0.5184009075164795 52 | \end{lstlisting} 53 | -------------------------------------------------------------------------------- /standard_lib/reduce_memory_consumption.py: -------------------------------------------------------------------------------- 1 | """Reference: https://medium.com/@alexmaisiura/python-how-to-reduce-memory-consumption-by-half-by-adding-just-one-line-of-code-56be6443d524""" 2 | import sys 3 | import tracemalloc 4 | 5 | 6 | def main(): 7 | tracemalloc.start() 8 | 9 | data = [] 10 | for p in range(100000): 11 | data.append(DataItem("Alex", 42, "middle of nowhere")) 12 | 13 | snapshot = tracemalloc.take_snapshot() 14 | top_stats = snapshot.statistics("lineno") 15 | total = sum(stat.size for stat in top_stats) 16 | print("[With __slots__] Total allocated size: %.1f MB" % (total / (1024 * 1024))) 17 | 18 | tracemalloc.stop() 19 | 20 | tracemalloc.start() 21 | 22 | data = [] 23 | for p in range(100000): 24 | data.append(DataItemWithoutSlots("Alex", 42, "middle of nowhere")) 25 | 26 | snapshot = tracemalloc.take_snapshot() 27 | top_stats = snapshot.statistics("lineno") 28 | total = sum(stat.size for stat in top_stats) 29 | print("[Without __slots__] Total allocated size: %.1f MB" % (total / (1024 * 1024))) 30 | 31 | tracemalloc.stop() 32 | 33 | 34 | class DataItem: 35 | __slots__ = ["name", "age", "address"] 36 | 37 | def __init__(self, name, age, address): 38 | self.name = name 39 | self.age = age 40 | self.address = address 41 | 42 | 43 | class DataItemWithoutSlots: 44 | def __init__(self, name, age, address): 45 | self.name = name 46 | self.age = age 47 | self.address = address 48 | 49 | 50 | def get_size(obj, seen=None): 51 | # From https://goshippo.com/blog/measure-real-size-any-python-object/ 52 | # Recursively finds size of objects 53 | size = sys.getsizeof(obj) 54 | if seen is None: 55 | seen = set() 56 | obj_id = id(obj) 57 | if obj_id in seen: 58 | return 0 59 | 60 | # Important mark as seen *before* entering recursion to gracefully handle 61 | # self-referential objects 62 | seen.add(obj_id) 63 | if isinstance(obj, dict): 64 | size += sum([get_size(v, seen) for v in obj.values()]) 65 | size += sum([get_size(k, seen) for k in obj.keys()]) 66 | elif hasattr(obj, "__dict__"): 67 | size += get_size(obj.__dict__, seen) 68 | elif hasattr(obj, "__iter__") and not isinstance(obj, (str, bytes, bytearray)): 69 | size += sum([get_size(i, seen) for i in obj]) 70 | 71 | return size 72 | 73 | 74 | if __name__ == "__main__": 75 | main() 76 | -------------------------------------------------------------------------------- /ebook/chapters/third_party_chapters/timing.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Timing} 4 | 5 | The \textit{Timing} section contains snippets related to timing. 6 | This means, that you'll find ways to time the execution time of certain code parts and much more. 7 | To achieve that, \lstinline{boxx} is mainly used. 8 | 9 | 10 | \subsection{Timing Tool: boxx} 11 | 12 | The third party \lstinline{boxx} module provides you a pretty neat way to time your code executions via \lstinline{with}-blocks. 13 | 14 | \lstinputlisting[caption=timing\_tool.py]{../third_party/timing_tool.py} 15 | 16 | \begin{lstlisting}[caption=Output of timing\_tool.py] 17 | $ python timing_tool.py 18 | "timeit" spend time: 0.01047492 19 | "sleep" spend time: 0.10027 20 | \end{lstlisting} 21 | 22 | 23 | \subsection{f-strings VS str} 24 | 25 | Let's say you want to convert an integer to a string. 26 | There exist different ways to do that. 27 | Working on an open source project I came across an option I've never thought of: using f-strings. 28 | But which method is faster? 29 | Using the builtin \lstinline{str} function or f-strings? 30 | You can use \lstinline{boxx} to time it easily. 31 | 32 | \lstinputlisting[caption=f-strings\_vs\_str.py]{../third_party/f-strings_vs_str.py} 33 | 34 | \begin{lstlisting}[caption=Output of f-strings\_vs\_str.py] 35 | $ python f-strings_vs_str.py 36 | "f-strings" spend time: 0.09714508 37 | "str" spend time: 0.1627851 38 | \end{lstlisting} 39 | 40 | \textbf{Note:} Even though f-strings are faster in this situation, keep in mind, that the builtin \lstinline{str} function is preferred to be used. 41 | Clean code is usually more important than efficiency. 42 | 43 | 44 | \subsection{range VS repeat} 45 | 46 | \lstinline{itertools.repeat} is faster than \lstinline{range} for looping a fixed number of times when you don't need the loop variable. 47 | An example is given below. 48 | 49 | \lstinputlisting[caption=range\_vs\_repeat.py]{../third_party/range_vs_repeat.py} 50 | 51 | As you can see, \lstinline{itertools.repeat} is slightly faster. 52 | 53 | \begin{lstlisting}[caption=Output of range\_vs\_repeat.py] 54 | $ python range_vs_repeat.py 55 | "Range" spend time: 0.01480103 56 | "Repeat" spend time: 0.01320004 57 | \end{lstlisting} 58 | 59 | If you want to avoid passing the \lstinline{None} argument, you can create a new function called \lstinline{times}, which hides the \lstinline{None} argument. 60 | 61 | \lstinputlisting[caption=partial\_function.py]{../standard_lib/partial_function.py} 62 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/dealing_builtins.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Dealing with builtins} 4 | 5 | Sometimes it can be helpful to manipulate builtins or to find a better way to deal with them. 6 | This section will show you ways to interact with builtins you may not thought of. 7 | 8 | 9 | \subsection{Change Behaviour} 10 | 11 | In some situations it can be helpful to extend the functionality of a certain builtin. 12 | Keep in mind, that this can be dangerous if it's not documented and is done in a global scope! 13 | 14 | \lstinputlisting[caption=builtins\_manipulation.py]{../standard_lib/builtins_manipulation.py} 15 | 16 | The Listing first assigns the builtin \lstinline{print} function to a variable called \lstinline{_print}. 17 | After that, a custom \mbox{\lstinline{print}} function is defined shadowing the builtin one. 18 | The functionality is extended by displaying the number of arguments before printing the actual output. 19 | You can find the output in the following Listing. 20 | 21 | \begin{lstlisting}[caption=Output of builtins\_manipulation.py] 22 | $ python builtins_manipulation.py 23 | One Two Three 24 | Number of arguments: 3 25 | \end{lstlisting} 26 | 27 | 28 | \subsection{Capture Output} 29 | 30 | It may be helpful to capture and redirect the output of certain functions. 31 | For instance you don't want to send the output of the builtin \lstinline{help} function to stdout but want to redirect it to a file. 32 | The following Listing shows you three ways how to capture and redirect the output of functions. 33 | 34 | \lstinputlisting[caption=capture\_output.py]{../standard_lib/capture_output.py} 35 | 36 | The first option saves the output to a \lstinline{StringIO} object. 37 | The value can be accessed using \lstinline{.getvalue()}. 38 | 39 | The second option can be used to save the output to a specified file. 40 | In this case we save the output to \lstinline{help.txt}. 41 | 42 | Last but not least we are redirecting the output to stderr. 43 | 44 | 45 | \subsection{max() Key Function} 46 | 47 | As many other built-in functions, the \lstinline{max()} function allows you to specify a key function. 48 | This allows you to (for instance) have a list of strings of integers and select the largest number of them. 49 | Without specifying \lstinline{key}, the wrong element is being selected. 50 | 51 | \lstinputlisting[caption=max\_int\_in\_list\_of\_str.py]{../standard_lib/max_int_in_list_of_str.py} 52 | 53 | \begin{lstlisting}[caption=Output of max\_int\_in\_list\_of\_str.py] 54 | $ python max_int_in_list_of_str.py 55 | 9 56 | 222 57 | \end{lstlisting} 58 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/decorators.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.py 2 | 3 | \section{Decorators} 4 | 5 | A collection of quite useful decorators you can use in your projects. 6 | 7 | 8 | \subsection{Deprecation Decorator} 9 | 10 | Let's assume you want to remove a certain function or method. 11 | Instead of just removing it, you want to signal your users that the function or method is deprecated, so they know, it will be removed in a future release. 12 | You can write your custom deprecation decorator printing a \lstinline{DeprecationWarning} whenever the deprecated function or method is invoked. 13 | Listing~\ref{lst:deprecateddecorator} shows you a sample implementation of the decorator as well as possible usages. 14 | 15 | \lstinputlisting[caption=deprecated\_decorator.py,label=lst:deprecateddecorator]{../standard_lib/deprecated_decorator.py} 16 | 17 | Everytime the deprecated function or method is called, a \lstinline{DeprecationWarning} is printed. 18 | 19 | \begin{lstlisting}[caption=Output of deprecated\_decorator.py] 20 | $ python deprecated_decorator.py 21 | /deprecated_decorator.py:10: DeprecationWarning: Call to deprecated function some_old_function. 22 | f"Call to deprecated function {func.__name__}.", category=DeprecationWarning 23 | /deprecated_decorator.py:10: DeprecationWarning: Call to deprecated function some_old_method. 24 | f"Call to deprecated function {func.__name__}.", category=DeprecationWarning 25 | \end{lstlisting} 26 | 27 | 28 | \subsection{Keep Function Metadata} 29 | 30 | The original functions metadata are lost when using a decorator. 31 | You can keep them using the \lstinline{@wraps} decorator provided by the \lstinline{functools} module. 32 | 33 | \lstinputlisting[caption=keep\_metadata\_on\_decorator\_usage.py]{../standard_lib/keep_metadata_on_decorator_usage.py} 34 | 35 | The output shows the result we would expect from a decorated function. 36 | 37 | \begin{lstlisting}[caption=Output of keep\_metadata\_on\_decorator\_usage.py] 38 | $ python keep_metadata_on_decorator_usage.py 39 |

Hello World

40 | get_text 41 | Returns some text 42 | \end{lstlisting} 43 | 44 | 45 | \subsection{Trace Decorator} 46 | 47 | This snippet contains a custom trace decorator revealing a functions flow. 48 | 49 | \lstinputlisting[caption=trace\_decorator.py]{../standard_lib/trace_decorator.py} 50 | 51 | The output is shown in the following Listing. 52 | 53 | \begin{lstlisting}[caption=Output of trace\_decorator.py] 54 | $ python trace_decorator.py 55 | -------------------- 56 | TRACE: calling greet() with ('Florian', 'Nice to see you!'), {} 57 | TRACE: greet() returned 'Florian, Nice to see you!' 58 | -------------------- 59 | Florian, Nice to see you! 60 | \end{lstlisting} 61 | -------------------------------------------------------------------------------- /ebook/chapters/third_party_chapters/image_animation.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Image and Animation} 4 | 5 | The section \textit{Image and Animation} contains recipes for image and video manipulation as well as animation creation. 6 | 7 | \subsection{Create A GIF} 8 | 9 | You can create a GIF by using the following three third party packages: 10 | 11 | \begin{itemize} 12 | \item \lstinline{animatplot} 13 | \item \lstinline{matplotlib} 14 | \item \lstinline{numpy} 15 | \end{itemize} 16 | 17 | \lstinputlisting[caption=animated\_graphics.py]{../third_party/animated_graphics.py} 18 | 19 | \textbf{Note:} Make sure, that an \lstinline{images} directory exists as it's not created automatically. 20 | 21 | 22 | \subsection{Change Image Background} 23 | 24 | To be able to use this recipe you not only need to have \lstinline{numpy} installed but also have \lstinline{OpenCV} installed on your system. 25 | There is an unofficial pre-built OpenCV packages for Python availabl on PyPI. 26 | You can install it via \lstinline{pip}. 27 | 28 | \begin{lstlisting}[caption=Install the unofficial pre-build OpenCV Python package from PyPI] 29 | $ python -m pip install opencv-python 30 | \end{lstlisting} 31 | 32 | \lstinputlisting[caption=change\_bg.py]{../third_party/change_bg.py} 33 | 34 | \textbf{Note:} \lstinline{np.array([114, 89, 47])} represents the background image you want to replace by the one on the next line. 35 | 36 | 37 | \subsection{Manipulate Images Using Imageio} 38 | 39 | \lstinline{imageio} is a Python library that provides an easy interface to read and write a wide range of image data, including animated images, volumetric data, and scientific formats. 40 | The following Listing shows you how to read an image from an url, turn it into a grey one and finally save it after blurring. 41 | 42 | \lstinputlisting[caption=manipulate\_images.py]{../third_party/manipulate_images.py} 43 | 44 | 45 | \subsection{Resize Images} 46 | 47 | With \lstinline{OpenCV} it's possible to manipulate images. 48 | This includes resizing images as well. 49 | This recipe shows you how all \lstinline{.jpg} images in the current working directory can be resized. 50 | 51 | \lstinputlisting[caption=resize\_images.py]{../third_party/resize_images.py} 52 | 53 | 54 | \subsection{Hide Image Inside Another} 55 | 56 | Making use of the steganography technique (hiding information), you can hide a whole image in another one. 57 | This can be achieved by using the Python Imaging Library (PIL) or the active developed fork \textit{Pillow}. 58 | The recipe contained by the \lstinline{steganography.py} file provides you a CLI, which you can use to hide an image inside another. 59 | 60 | 61 | \subsection{Create Own Images} 62 | 63 | Using four different packages, you can create your own (random generated) image. 64 | The following Listing reveals you the necessary source code. 65 | 66 | \lstinputlisting[caption=tensorflow\_image.py]{../third_party/tensorflow_image.py} 67 | -------------------------------------------------------------------------------- /third_party/refactoring_code/testutils.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import shutil 3 | import sys 4 | import logging 5 | 6 | logging.basicConfig(format="%(levelname)s:%(funcName)s:%(message)s", level=logging.INFO) 7 | try: 8 | import unittest2 as unittest 9 | except ImportError: 10 | import unittest 11 | 12 | import rope.base.project 13 | from rope.contrib import generate 14 | 15 | 16 | def sample_project(root=None, foldername=None, **kwds): 17 | if root is None: 18 | root = "sample_project" 19 | if foldername: 20 | root = foldername 21 | # HACK: Using ``/dev/shm/`` for faster tests 22 | if os.name == "posix": 23 | if os.path.isdir("/dev/shm") and os.access("/dev/shm", os.W_OK): 24 | root = "/dev/shm/" + root 25 | elif os.path.isdir("/tmp") and os.access("/tmp", os.W_OK): 26 | root = "/tmp/" + root 27 | logging.debug("Using %s as root of the project.", root) 28 | # Using these prefs for faster tests 29 | prefs = { 30 | "save_objectdb": False, 31 | "save_history": False, 32 | "validate_objectdb": False, 33 | "automatic_soa": False, 34 | "ignored_resources": [".ropeproject", "*.pyc"], 35 | "import_dynload_stdmods": False, 36 | } 37 | prefs.update(kwds) 38 | remove_recursively(root) 39 | project = rope.base.project.Project(root, **prefs) 40 | return project 41 | 42 | 43 | create_module = generate.create_module 44 | create_package = generate.create_package 45 | 46 | 47 | def remove_project(project): 48 | project.close() 49 | remove_recursively(project.address) 50 | 51 | 52 | def remove_recursively(path): 53 | import time 54 | 55 | # windows sometimes raises exceptions instead of removing files 56 | if os.name == "nt" or sys.platform == "cygwin": 57 | for i in range(12): 58 | try: 59 | _remove_recursively(path) 60 | except OSError as e: 61 | if e.errno not in (13, 16, 32): 62 | raise 63 | time.sleep(0.3) 64 | else: 65 | break 66 | else: 67 | _remove_recursively(path) 68 | 69 | 70 | def _remove_recursively(path): 71 | if not os.path.exists(path): 72 | return 73 | if os.path.isfile(path): 74 | os.remove(path) 75 | else: 76 | shutil.rmtree(path) 77 | 78 | 79 | def only_for(version): 80 | """Should be used as a decorator for a unittest.TestCase test method""" 81 | return unittest.skipIf( 82 | sys.version < version, 83 | "This test requires at least {0} version of Python.".format(version), 84 | ) 85 | 86 | 87 | def only_for_versions_lower(version): 88 | """Should be used as a decorator for a unittest.TestCase test method""" 89 | return unittest.skipIf( 90 | sys.version > version, 91 | "This test requires version of Python lower than {0}".format(version), 92 | ) 93 | 94 | 95 | def skipNotPOSIX(): 96 | return unittest.skipIf(os.name != "posix", "This test works only on POSIX") 97 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/lists.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Lists} 4 | 5 | Provide useful tips for dealing with lists. 6 | 7 | 8 | \subsection{Flatten a List} 9 | 10 | Sometimes you have a nested list and just want to flatten it. 11 | Here's a small snippet revealing how to achieve right that. 12 | 13 | \lstinputlisting[caption=flatten.py]{../standard_lib/flatten.py} 14 | 15 | \begin{lstlisting}[caption=Output of flatten.py] 16 | $ python flatten.py 17 | [1, 2, 3, 4, 5, 6] 18 | \end{lstlisting} 19 | 20 | 21 | \subsection{Priority Queue} 22 | 23 | Image you have a sports tournament (e.g. table tennis or basketball). 24 | Now you want to get the player or team with the most points. 25 | You could store all the information in a list, but if new items are added, you need to resort it. 26 | This can take a significant time amount if the data set keeps growing. 27 | You can make use of a \lstinline{Heap} data structure to implement your own priority queue to auto-sort the data for you. 28 | 29 | \lstinputlisting[caption=priority\_queue.py]{../standard_lib/priority_queue.py} 30 | 31 | As the snippet provides a sample usage, the following Listing shows you the output. 32 | 33 | \begin{lstlisting}[caption=Output of priority\_queue.py] 34 | $ python priority_queue.py 35 | hello 36 | None 37 | world 38 | None 39 | \end{lstlisting} 40 | 41 | 42 | \subsection{Remove Duplicates} 43 | 44 | Remove duplicates from a list but keeping the order by using \lstinline{OrderedDict}. 45 | 46 | \lstinputlisting[caption=remove\_duplicates\_list.py]{../standard_lib/remove_duplicates_list.py} 47 | 48 | \begin{lstlisting}[caption=Output of remove\_duplicates\_list.py] 49 | $ python remove_duplicates_list.py 50 | List with duplicates: ['foo', 'Alice', 'bar', 'foo', 'Bob'] 51 | Without duplicates: ['foo', 'Alice', 'bar', 'Bob'] 52 | \end{lstlisting} 53 | 54 | 55 | \subsection{Unpacking Lists Using * Operator} 56 | 57 | Assuming you have a list with more elements than you have variables to store the values in. 58 | You can use the \lstinline{*} operator when unpacking a list to store a partial list in one variable. 59 | 60 | \lstinputlisting[caption=list\_unpacking.py]{../standard_lib/list_unpacking.py} 61 | 62 | \begin{lstlisting}[caption=Output of list\_unpacking.py] 63 | $ python list_unpacking.py 64 | a = 1 65 | b = [2, 3, 4] 66 | c = 5 67 | \end{lstlisting} 68 | 69 | 70 | \subsection{Remove Elements Not Matching Pattern} 71 | 72 | Let's assume you have a list of certain data and want to remove all elements in it, which are not matching a certain pattern. 73 | You could make use of the built-in \lstinline{list.remove()} method, which removes the first match of and then shifts all subsequent data one position to the left. 74 | So, if you want to remove all, you need to loop over it. 75 | However, this approach gives quadratic behaviour. 76 | The following Listing shows you a much better way to achieve this: 77 | 78 | \lstinputlisting[caption=remove\_elements\_list.py]{../standard_lib/remove_elements_list.py} 79 | 80 | This approach is not only faster, but creates a new, distinct list and then replaces the old contents all at once. 81 | -------------------------------------------------------------------------------- /third_party/README.md: -------------------------------------------------------------------------------- 1 | # Third Party # 2 | ## Description ## 3 | A collection of useful snippets using third party packages. 4 | 5 | ## Available Snippets ## 6 | | Snippet Name | Description | 7 | |--------------|-------------| 8 | | animated_graphics | Gives an inside in the usage of animatplot - animated graphics build on matplotlib | 9 | | attention_message | Display a message on the screen in different colors or blinking | 10 | | auto_login_website | Using Selenium to auto-login into a website | 11 | | change_bg | Use OpenCV to change image background | 12 | | cli_help_strings | Beautiful rendered cli help strings | 13 | | clint_cli_tool | Tool helping you to create awesome CLIs | 14 | | colored_python | Print colored text to stdout using termcolor | 15 | | complex_list_ops | Speed up complex list ops using `blist` | 16 | | cooked_input_example | Getting, cleaning, converting, and validating command line input | 17 | | count_python_bytes | Get the bytes size of all Python files in a certain directory and it's subdirectories | 18 | | display_tables | Using prettytable to display tables well-formatted in the console | 19 | | fancy_cli_header | Turns strings into ASCII text with arts fonts (fancy CLI header) | 20 | | folium_snippet | Folium is used to create interactive maps | 21 | | formatted_json | Dump json-objects in a formatted way | 22 | | f-strings_vs_str | Compare the time of f-strings and str | 23 | | img_to_ascii(_2) | Create ascii arts based on images | 24 | | inspect_docker | Control docker from within Python | 25 | | interactive_cli | Example of PyInquirer to create interactive CLIs | 26 | | is_holiday | Checks whether a given date is a holiday in a certain country | 27 | | manipulate_images | Small example on how to manipulate images | 28 | | mathematicians | Small project crawling the most popular mathematicians | 29 | | medium | Get a list of interesting users to follow based on your profile | 30 | | mocking_requests | Mocking the entire `requests` library and check for correct response handling | 31 | | mypy_example | Example on how to use Pythons type hints and mypy | 32 | | numpy_array_operations | Reveiling the behaviour of simply array operations using NumPy | 33 | | parse_complex_excel_sheets | Parse complex Excel Sheets using `pandas` and `xlrd` | 34 | | port_scanner_nmap | Port Scanner using nmap | 35 | | pytest_rename_class_backwards_compatibility | Rename a critical class and test for backwards compatibility | 36 | | range_vs_repeat | `itertools.repeat()` is faster than `range()` for looping a fixed number of times when you don't need the loop variable | 37 | | reduce_pandas_df_memory | Reduce pandas df memory usage by converting `object` to `category` | 38 | | refactoring_code | Refactoring Python code using rope | 39 | | resize_images | Resize all images in the current directory using OpenCV | 40 | | show_progress | Shows the progress using progressbar | 41 | | simple_debugger | Use PySnoopers's decorator to debug functions | 42 | | simple_progressbar | Create a simple progressbar using tqdm.tqdm() | 43 | | steganography | Hide an image inside another | 44 | | tensorflow_image | Generate an image using tensorflow | 45 | | test_asyncio | Example on how to use asyncio | 46 | | test_gevent | Demonstrates the usage of gevent | 47 | | test_tornado | Reveales the usage of tornado | 48 | | text_analysis | Analyse text using TextBlob | 49 | | timing_tool | Illustrates the usage of boxx.timeit() reveilling the time a certain code block takes to run | 50 | | world_bank_data | Using official world bank data to demonstrate the usage of `zipfile` and `csv` | 51 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/string_operations.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{String Operations} 4 | 5 | In the following section you will find useful snippets when dealing with strings. 6 | 7 | 8 | \subsection{Check for String Pattern} 9 | 10 | Assuming you have a list of strings and you want to check them for multiple pattern. 11 | The following snippet solves this issue by using a fairly simple list comprehension. 12 | 13 | \lstinputlisting[caption=check\_pattern.py]{../standard_lib/check_pattern.py} 14 | 15 | The output after running the snippet is something like that: 16 | 17 | \begin{lstlisting}[caption=Output of check\_pattern.py] 18 | $ python check_pattern.py 19 | ['hello', 'hello world', 'xxx world'] 20 | \end{lstlisting} 21 | 22 | 23 | \subsection{Compare Strings} 24 | 25 | If you want to compare two strings and want to know \glqq how equal they are\grqq, you can make use of the \linebreak \lstinline{SequenceMatcher}. 26 | 27 | \lstinputlisting[caption=compare\_strings.py]{../standard_lib/compare_strings.py} 28 | 29 | It will split the strings into matching blocks and return how many characters of the blocks are matching. 30 | Furthermore, you get a float value representing the overall matching. 31 | The output is shown in the following Listing. 32 | 33 | \begin{lstlisting}[caption=Output of compare\_strings.py] 34 | $ python compare_strings.py 35 | 0.895 36 | a[0] and b[0] match for 11 elements 37 | a[11] and b[15] match for 6 elements 38 | a[17] and b[21] match for 0 elements 39 | \end{lstlisting} 40 | 41 | 42 | \subsection{Fill Strings} 43 | 44 | You can use the \lstinline{zfill} string method to fill a string with zeros if the provided maximum length isn't already reached. 45 | 46 | \lstinputlisting[caption=fill\_zeros.py]{../standard_lib/fill_zeros.py} 47 | 48 | The outpur is shown in the following Listing. 49 | 50 | \begin{lstlisting}[caption=Output of fill\_zeros.py] 51 | $ python fill_zeros.py 52 | 0001 53 | 0011 54 | 0222 55 | 0aaa 56 | 1234 57 | \end{lstlisting} 58 | 59 | 60 | \subsection{Parse Query String} 61 | 62 | You can use the builtin \lstinline{urllib} module to parse query strings. 63 | 64 | \lstinputlisting[caption=parse\_query\_string.py]{../standard_lib/parse_query_string.py} 65 | 66 | The provided snippets will output the following: 67 | 68 | \begin{lstlisting}[caption=Output of parse\_query\_string.py] 69 | $ python parse_query_string.py 70 | {'host': ['171.25.2.7'], 'port': ['8080', '8081']} 71 | \end{lstlisting} 72 | 73 | 74 | \subsection{Print Numbers Human-Friendly} 75 | 76 | You can print numbers in a human-friendly way using the builtin \lstinline{format} function. 77 | 78 | \lstinputlisting[caption=print\_human\_friendly\_numbers.py]{../standard_lib/print_human_friendly_numbers.py} 79 | 80 | \begin{lstlisting}[caption=Output of print\_human\_friendly\_numbers.py] 81 | $ python print_human_friendly_numbers.py 82 | 123,123,123 83 | \end{lstlisting} 84 | 85 | 86 | \subsection{Split Strings Preserving Substrings} 87 | 88 | In some cases you're splitting a string into the words it contains, but you want to keep substrings. 89 | Fortunately, Python has a module called \lstinline{shlex} providing a \lstinline{split} function keeping substrings as one. 90 | 91 | \lstinputlisting[caption=split\_preserving\_sub-strings.py]{../standard_lib/split_preserving_sub-strings.py} 92 | 93 | \begin{lstlisting}[caption=Output of split\_preserving\_sub-strings.py] 94 | $ python split_preserving_sub-strings.py 95 | Split string output: ['This', 'is', '"Berlin', 'Cafe"'] 96 | Shlex string outpur: ['This', 'is', 'Berlin Cafe'] 97 | \end{lstlisting} 98 | 99 | 100 | \subsection{capitalize() vs. title()} 101 | 102 | The following Listings shows the difference between \lstinline{capitalize} and \lstinline{title} 103 | 104 | \lstinputlisting[caption=string\_capitalize.py]{../standard_lib/string_capitalize.py} 105 | 106 | \begin{lstlisting}[caption=Output of string\_capitalize.py] 107 | $ python string_capitalize.py 108 | capitalize(): I love coffee 109 | title(): I Love Coffee 110 | \end{lstlisting} 111 | -------------------------------------------------------------------------------- /third_party/mathematicians.py: -------------------------------------------------------------------------------- 1 | from requests import get 2 | from requests.exceptions import RequestException 3 | from contextlib import closing 4 | from bs4 import BeautifulSoup 5 | 6 | 7 | def simple_get(url): 8 | """ 9 | Attempts to get the content at `url` by making an HTTP GET request. 10 | If the content-type of response is some kind of HTML/XML, return the 11 | text content, otherwise return None 12 | """ 13 | try: 14 | with closing(get(url, stream=True)) as resp: 15 | if is_good_response(resp): 16 | return resp.content 17 | else: 18 | return None 19 | 20 | except RequestException as e: 21 | log_error("Error during requests to {0} : {1}".format(url, str(e))) 22 | return None 23 | 24 | 25 | def is_good_response(resp): 26 | """ 27 | Returns true if the response seems to be HTML, false otherwise 28 | """ 29 | content_type = resp.headers["Content-Type"].lower() 30 | return ( 31 | resp.status_code == 200 32 | and content_type is not None 33 | and content_type.find("html") > -1 34 | ) 35 | 36 | 37 | def log_error(e): 38 | """ 39 | It is always a good idea to log errors. 40 | This function just prints them, but you can 41 | make it do anything. 42 | """ 43 | print(e) 44 | 45 | 46 | def get_names(): 47 | """ 48 | Downloads the page where the list of mathematicians is found 49 | and returns a list of strings, one per mathematician 50 | """ 51 | url = "http://www.fabpedigree.com/james/mathmen.htm" 52 | response = simple_get(url) 53 | 54 | if response is not None: 55 | html = BeautifulSoup(response, "html.parser") 56 | names = set() 57 | for li in html.select("li"): 58 | for name in li.text.split("\n"): 59 | if len(name) > 0: 60 | names.add(name.strip()) 61 | 62 | return list(names) 63 | 64 | # Raise an exception if we failed to get any data from the url 65 | raise Exception("Error retrieving contents at {}".format(url)) 66 | 67 | 68 | def get_hits_on_name(name): 69 | """ 70 | Accepts a `name` of a mathematician and returns the number 71 | of hits that mathematician's wikipedia page received in the 72 | last 60 days, as an `int` 73 | """ 74 | # url_root is a template string that used to buld a URL. 75 | url_root = "https://xtools.wmflabs.org/articleinfo/en.wikipedia.org/{}" 76 | response = simple_get(url_root.format(name)) 77 | 78 | if response is not None: 79 | html = BeautifulSoup(response, "html.parser") 80 | 81 | hit_link = [a for a in html.select("a") if a["href"].find("latest-60") > -1] 82 | 83 | if len(hit_link) > 0: 84 | # Strip commas: 85 | link_text = hit_link[0].text.replace(",", "") 86 | try: 87 | # Convert to integer 88 | return int(link_text) 89 | except ValueError: 90 | log_error("couldn't parse {} as an `int`".format(link_text)) 91 | 92 | log_error("No pageviews found for {}".format(name)) 93 | 94 | return None 95 | 96 | 97 | if __name__ == "__main__": 98 | print("Getting the list of names....") 99 | names = get_names() 100 | print("... done.\n") 101 | 102 | results = [] 103 | 104 | print("Getting stats for each name....") 105 | 106 | for name in names: 107 | try: 108 | hits = get_hits_on_name(name) 109 | if hits is None: 110 | hits = -1 111 | results.append((hits, name)) 112 | except Exception: 113 | results.append((-1, name)) 114 | log_error("error encountered while processing " "{}, skipping".format(name)) 115 | 116 | print("... done.\n") 117 | 118 | results.sort() 119 | results.reverse() 120 | 121 | if len(results) > 5: 122 | top_marks = results[:5] 123 | else: 124 | top_marks = results 125 | 126 | print("\nThe most popular mathematicians are:\n") 127 | for (mark, mathematician) in top_marks: 128 | print("{} with {} page views".format(mathematician, mark)) 129 | 130 | no_results = len([res for res in results if res[0] == -1]) 131 | print( 132 | "\nBut we did not find results for " 133 | "{} mathematicians on the list".format(no_results) 134 | ) 135 | -------------------------------------------------------------------------------- /third_party/img_to_ascii.py: -------------------------------------------------------------------------------- 1 | 2 | # Python code to convert an image to ASCII image. 3 | import sys, random, argparse 4 | import numpy as np 5 | import math 6 | 7 | from PIL import Image 8 | 9 | # gray scale level values from: 10 | # http://paulbourke.net/dataformats/asciiart/ 11 | 12 | # 70 levels of gray 13 | gscale1 = "$@B%8&WM#*oahkbdpqwmZO0QLCJUYXzcvunxrjft/\|()1{}[]?-_+~<>i!lI;:,\"^`'. " 14 | 15 | # 10 levels of gray 16 | gscale2 = '@%#*+=-:. ' 17 | 18 | def getAverageL(image): 19 | 20 | """ 21 | Given PIL Image, return average value of grayscale value 22 | """ 23 | # get image as numpy array 24 | im = np.array(image) 25 | 26 | # get shape 27 | w,h = im.shape 28 | 29 | # get average 30 | return np.average(im.reshape(w*h)) 31 | 32 | def covertImageToAscii(fileName, cols, scale, moreLevels): 33 | """ 34 | Given Image and dims (rows, cols) returns an m*n list of Images 35 | """ 36 | # declare globals 37 | global gscale1, gscale2 38 | 39 | # open image and convert to grayscale 40 | image = Image.open(fileName).convert('L') 41 | 42 | # store dimensions 43 | W, H = image.size[0], image.size[1] 44 | print("input image dims: %d x %d" % (W, H)) 45 | 46 | # compute width of tile 47 | w = W/cols 48 | 49 | # compute tile height based on aspect ratio and scale 50 | h = w/scale 51 | 52 | # compute number of rows 53 | rows = int(H/h) 54 | 55 | print("cols: %d, rows: %d" % (cols, rows)) 56 | print("tile dims: %d x %d" % (w, h)) 57 | 58 | # check if image size is too small 59 | if cols > W or rows > H: 60 | print("Image too small for specified cols!") 61 | exit(0) 62 | 63 | # ascii image is a list of character strings 64 | aimg = [] 65 | # generate list of dimensions 66 | for j in range(rows): 67 | y1 = int(j*h) 68 | y2 = int((j+1)*h) 69 | 70 | # correct last tile 71 | if j == rows-1: 72 | y2 = H 73 | 74 | # append an empty string 75 | aimg.append("") 76 | 77 | for i in range(cols): 78 | 79 | # crop image to tile 80 | x1 = int(i*w) 81 | x2 = int((i+1)*w) 82 | 83 | # correct last tile 84 | if i == cols-1: 85 | x2 = W 86 | 87 | # crop image to extract tile 88 | img = image.crop((x1, y1, x2, y2)) 89 | 90 | # get average luminance 91 | avg = int(getAverageL(img)) 92 | 93 | # look up ascii char 94 | if moreLevels: 95 | gsval = gscale1[int((avg*69)/255)] 96 | else: 97 | gsval = gscale2[int((avg*9)/255)] 98 | 99 | # append ascii char to string 100 | aimg[j] += gsval 101 | 102 | # return txt image 103 | return aimg 104 | 105 | # main() function 106 | def main(): 107 | # create parser 108 | descStr = "This program converts an image into ASCII art." 109 | parser = argparse.ArgumentParser(description=descStr) 110 | # add expected arguments 111 | parser.add_argument('--file', dest='imgFile', required=True) 112 | parser.add_argument('--scale', dest='scale', required=False) 113 | parser.add_argument('--out', dest='outFile', required=False) 114 | parser.add_argument('--cols', dest='cols', required=False) 115 | parser.add_argument('--morelevels',dest='moreLevels',action='store_true') 116 | 117 | # parse args 118 | args = parser.parse_args() 119 | 120 | imgFile = args.imgFile 121 | 122 | # set output file 123 | outFile = 'out.txt' 124 | if args.outFile: 125 | outFile = args.outFile 126 | 127 | # set scale default as 0.43 which suits 128 | # a Courier font 129 | scale = 0.43 130 | if args.scale: 131 | scale = float(args.scale) 132 | 133 | # set cols 134 | cols = 80 135 | if args.cols: 136 | cols = int(args.cols) 137 | 138 | print('generating ASCII art...') 139 | # convert image to ascii txt 140 | aimg = covertImageToAscii(imgFile, cols, scale, args.moreLevels) 141 | 142 | # open file 143 | f = open(outFile, 'w') 144 | 145 | # write to file 146 | for row in aimg: 147 | f.write(row + '\n') 148 | 149 | # cleanup 150 | f.close() 151 | print("ASCII art written to %s" % outFile) 152 | 153 | # call main 154 | if __name__ == '__main__': 155 | main() 156 | -------------------------------------------------------------------------------- /third_party/steganography.py: -------------------------------------------------------------------------------- 1 | import click 2 | from PIL import Image 3 | 4 | 5 | class Steganography: 6 | @staticmethod 7 | def __int_to_bin(rgb): 8 | """Convert an integer tuple to a binary (string) tuple. 9 | :param rgb: An integer tuple (e.g. (220, 110, 96)) 10 | :return: A string tuple (e.g. ("00101010", "11101011", "00010110")) 11 | """ 12 | r, g, b = rgb 13 | return "{0:08b}".format(r), "{0:08b}".format(g), "{0:08b}".format(b) 14 | 15 | @staticmethod 16 | def __bin_to_int(rgb): 17 | """Convert a binary (string) tuple to an integer tuple. 18 | :param rgb: A string tuple (e.g. ("00101010", "11101011", "00010110")) 19 | :return: Return an int tuple (e.g. (220, 110, 96)) 20 | """ 21 | r, g, b = rgb 22 | return int(r, 2), int(g, 2), int(b, 2) 23 | 24 | @staticmethod 25 | def __merge_rgb(rgb1, rgb2): 26 | """Merge two RGB tuples. 27 | :param rgb1: A string tuple (e.g. ("00101010", "11101011", "00010110")) 28 | :param rgb2: Another string tuple 29 | (e.g. ("00101010", "11101011", "00010110")) 30 | :return: An integer tuple with the two RGB values merged. 31 | """ 32 | r1, g1, b1 = rgb1 33 | r2, g2, b2 = rgb2 34 | rgb = (r1[:4] + r2[:4], g1[:4] + g2[:4], b1[:4] + b2[:4]) 35 | return rgb 36 | 37 | @staticmethod 38 | def merge(img1, img2): 39 | """Merge two images. The second one will be merged into the first one. 40 | :param img1: First image 41 | :param img2: Second image 42 | :return: A new merged image. 43 | """ 44 | 45 | # Check the images dimensions 46 | if img2.size[0] > img1.size[0] or img2.size[1] > img1.size[1]: 47 | raise ValueError("Image 2 should not be larger than Image 1!") 48 | 49 | # Get the pixel map of the two images 50 | pixel_map1 = img1.load() 51 | pixel_map2 = img2.load() 52 | 53 | # Create a new image that will be outputted 54 | new_image = Image.new(img1.mode, img1.size) 55 | pixels_new = new_image.load() 56 | 57 | for i in range(img1.size[0]): 58 | for j in range(img1.size[1]): 59 | rgb1 = Steganography.__int_to_bin(pixel_map1[i, j]) 60 | 61 | # Use a black pixel as default 62 | rgb2 = Steganography.__int_to_bin((0, 0, 0)) 63 | 64 | # Check if the pixel map position is valid for the second image 65 | if i < img2.size[0] and j < img2.size[1]: 66 | rgb2 = Steganography.__int_to_bin(pixel_map2[i, j]) 67 | 68 | # Merge the two pixels and convert it to a integer tuple 69 | rgb = Steganography.__merge_rgb(rgb1, rgb2) 70 | 71 | pixels_new[i, j] = Steganography.__bin_to_int(rgb) 72 | 73 | return new_image 74 | 75 | @staticmethod 76 | def unmerge(img): 77 | """Unmerge an image. 78 | :param img: The input image. 79 | :return: The unmerged/extracted image. 80 | """ 81 | 82 | # Load the pixel map 83 | pixel_map = img.load() 84 | 85 | # Create the new image and load the pixel map 86 | new_image = Image.new(img.mode, img.size) 87 | pixels_new = new_image.load() 88 | 89 | # Tuple used to store the image original size 90 | original_size = img.size 91 | 92 | for i in range(img.size[0]): 93 | for j in range(img.size[1]): 94 | # Get the RGB (as a string tuple) from the current pixel 95 | r, g, b = Steganography.__int_to_bin(pixel_map[i, j]) 96 | 97 | # Extract the last 4 bits (corresponding to the hidden image) 98 | # Concatenate 4 zero bits because we are working with 8 bit 99 | rgb = (r[4:] + "0000", g[4:] + "0000", b[4:] + "0000") 100 | 101 | # Convert it to an integer tuple 102 | pixels_new[i, j] = Steganography.__bin_to_int(rgb) 103 | 104 | # If this is a 'valid' position, store it 105 | # as the last valid position 106 | if pixels_new[i, j] != (0, 0, 0): 107 | original_size = (i + 1, j + 1) 108 | 109 | # Crop the image based on the 'valid' pixels 110 | new_image = new_image.crop((0, 0, original_size[0], original_size[1])) 111 | 112 | return new_image 113 | 114 | 115 | @click.group() 116 | def cli(): 117 | pass 118 | 119 | 120 | @cli.command() 121 | @click.option( 122 | "--img1", required=True, type=str, help="Image that will hide another image" 123 | ) 124 | @click.option("--img2", required=True, type=str, help="Image that will be hidden") 125 | @click.option("--output", required=True, type=str, help="Output image") 126 | def merge(img1, img2, output): 127 | merged_image = Steganography.merge(Image.open(img1), Image.open(img2)) 128 | merged_image.save(output) 129 | 130 | 131 | @cli.command() 132 | @click.option("--img", required=True, type=str, help="Image that will be hidden") 133 | @click.option("--output", required=True, type=str, help="Output image") 134 | def unmerge(img, output): 135 | unmerged_image = Steganography.unmerge(Image.open(img)) 136 | unmerged_image.save(output) 137 | 138 | 139 | if __name__ == "__main__": 140 | cli() 141 | -------------------------------------------------------------------------------- /standard_lib/README.md: -------------------------------------------------------------------------------- 1 | # Standard Library # 2 | ## Description ## 3 | A collection of useful snippets using only the standard library. 4 | 5 | ## Available Snippets ## 6 | | Snippet Name | Description | 7 | |--------------|-------------| 8 | | abc_class | Example on how to use abc | 9 | | async_sleep_async | Async snippet demonstrating the usage of asyncio.sleep() | 10 | | async_sleep_sync | Async snippet demonstrating the usage of time.sleep() | 11 | | binary_search | Demonstrating the efficiency of the binary search algorithm | 12 | | bit_flipper | A poor man's encryption | 13 | | bitmask | Class that represents a bit mask | 14 | | builtins_manipulation | Illustrates the easy overriding/manipulation of built-in functions | 15 | | capture_output | Capture the output of a function generally directing to stdout | 16 | | chained_assignments | Assign a value to multiple variables using chained assignments | 17 | | chained_comparison | Chain comparisons to have more math-like comparisons | 18 | | check_pattern | Check for multiple string patterns | 19 | | combine_iterables | Combine two iterables using `zip()` | 20 | | compare_strings | Use `difflib.SequenceMatcher` to compare strings | 21 | | count_thresholds | Count the number of elements between certain thresholds | 22 | | crazy_dict_expression | The craziest dictionary expression ever seen | 23 | | deprecated_decorator | Prints a DeprecationWarning when using a function/method marked as deprecated | 24 | | dict_based_on_lists | Create a dict based on two lists | 25 | | disassemble_bytecode | Shows the bytecode representation of f-strings and str() conversion of integers | 26 | | drawing_turtle | Drawing a dragon symbol using Python's built-in turtle | 27 | | empty_set_literal | Create an empty set using a literal | 28 | | emulate_switch_case | Emulate switch-case-statements as they don't exist in Python | 29 | | file_matching_regex | Find matching files using `re` and `fnmatch` | 30 | | fill_zeros | Fill strings using leading zeros. | 31 | | flatten | Flatten a nested iterable | 32 | | for_loop_index | Utilize `enumerate()` to get the index of the current iteration | 33 | | function_arguments | Example on how to (not) use function arguments | 34 | | get_password_input | Request user input without showing, what he or she is typing | 35 | | hash_file | Hash a file using built-in packages | 36 | | hex_decode | Get bytes-like string from hex | 37 | | human_readable_bytecode | Print the bytecode representations in a human readble format | 38 | | iterable_unpacking | Unpack an iterable and assigns its elements to multiple variables | 39 | | keep_metadata_on_decorator_usage | Keep functions metadata after decoration | 40 | | list_unpacking | Unpacking a list using the `*` operator | 41 | | max_int_in_list_of_str | Get the largest number from a list of strings | 42 | | merge_arbitrary_number_of_dicts | The most pythonic way to merge an arbitrary number of dicts | 43 | | MicroWebServer | A lightweight micro web server | 44 | | multi_dict_with_init | Provide a multi dict with default value using `defaultdict` | 45 | | multi_open_files | Open and manage multiple files at once using `ExitStack` | 46 | | open_browser_tab | Opens a new tab in a specified browser (and more) | 47 | | overwrite_dictionary | PEP 448 - overwriting dictionary of default values | 48 | | parse_query_string | Parse a query string | 49 | | partial_function | Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords | 50 | | pass_multiple_dicts | Unpack multiple dicts and pass them as keyword args to a function | 51 | | pathlib_relpath | `pathlib.Path` implementation of `os.relpath` (should never be used) | 52 | | port_scanner | Simple port scanner implementation using built-in packages | 53 | | print_human_friendly_numbers | Use `format` to print large numbers human friendly | 54 | | priority_queue | Implement a priority queue using `heapq` | 55 | | provide_default_config_values | Chain multiple dicts to provide default config and provide config value overwriting | 56 | | read_files_using_iterator | Use iterators to efficiently load huge files | 57 | | reduce_memory_consumption | Reduce the memory consumption for an instance of a class using __slots__ | 58 | | reduce_memory_consumption_iterator | Shows how to reduce memory consumption by using itertools.repeat | 59 | | regular_expression_debug | Display debug information about the compiled regular expression with re.DEBUG flag | 60 | | remove_duplicates_list | Remove duplicates from list and keep the order | 61 | | remove_elements_list | Remove elements not matching certain pattern - the fast way! | 62 | | reverse_iterables | Reverse an iterable using `reversed()` | 63 | | save_dict_update_without_loosing_original | Update dict value without loosing original value | 64 | | scopes_namespaces | Reveils the differences between `global`, `local`, and `nonlocal` | 65 | | set_union_intersection | Use \| and & for set union and intersection | 66 | | slice_generators | Slice generators using `itertools.islice` | 67 | | sort_complex_tuples | Sort complex tuples by one of their keys | 68 | | split_preserving_sub-strings | Python standard lib's `shlex` splitting strings preserving sub-strings | 69 | | string_capitalize | Difference between title() and capitalize() | 70 | | temptable_contextmanager | Create a contextmanager providing a temptable for SQLite | 71 | | timing_context_manager | A simple timing context manager based on a generator function | 72 | | trace_decorator | Trace decorator tracing function calls | 73 | | tree_clone | Python clone of the UNIX tree command - but better! | 74 | | unicode_source_code | Example on how awesome unicode in source code is | 75 | | update_dict_using_tuples | Update a dict using a list of tuples representing key-value-pairs | 76 | | uuid1_example | Generate RFC compliant UUID | 77 | | variable_swapping | Swap the values of two variables in one line of code | 78 | | zip_safe | Illustrates how zip is stopping if one iterable is exhausted without a warning and how to prevent it | 79 | -------------------------------------------------------------------------------- /ebook/chapters/third_party_chapters/commandline.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Command-Line} 4 | 5 | In this section you will find useful snippets when dealing with command-lines. 6 | 7 | \subsection{Attention Please!} 8 | 9 | Sometimes you need to create an eye catcher on the command-line. 10 | Therefore, you can use \lstinline{termcolor} to create one. 11 | In the following code snippet you will find the code necessary to create a blinking white message on a red background. 12 | 13 | \lstinputlisting[caption=attention\_message.py]{../third_party/attention_message.py} 14 | 15 | 16 | \subsection{Colored Python} 17 | 18 | As indicated in the previous recipe, you can use \lstinline{termcolor} to print colored strings on the command-line. 19 | 20 | \lstinputlisting[caption=colored\_python.py]{../third_party/colored_python.py} 21 | 22 | 23 | \subsection{Generate CLI Help Strings} 24 | 25 | Making use of \lstinline{docopt} you can generate cli help strings based on Python docstrings. 26 | 27 | \lstinputlisting[caption=cli\_help\_strings.py]{../third_party/cli_help_strings.py} 28 | 29 | \begin{lstlisting}[caption=Output of cli\_help\_string.py] 30 | $ python cli_help_strings.py --help 31 | CLI HELP STRINGS 32 | Usage: 33 | cli_help_strings.py 34 | cli_help_strings.py 35 | cli_help_strings.py -h|--help 36 | cli_help_strings.py -v|--version 37 | Options: 38 | Optional name argument. 39 | -h --help Show this screen. 40 | -v --version Show version. 41 | \end{lstlisting} 42 | 43 | 44 | \subsection{Parse And Print Colored Arguments} 45 | 46 | \lstinline{clint} provides you the functionalities to parse command-line arguments and to print them in a colored way. 47 | 48 | \lstinputlisting[caption=clint\_cli\_tool.py]{../third_party/clint_cli_tool.py} 49 | 50 | \begin{lstlisting}[caption=Output of clint\_cli\_tool.py] 51 | $ python clint_cli_tool.py 52 | >>> Arguments passed in: [] 53 | >>> Flags detected: 54 | >>> Files detected: [] 55 | >>> NOT Files detected: 56 | >>> Grouped Arguments: {'_': } 57 | \end{lstlisting} 58 | 59 | \textbf{Note:} The keywords between \lstinline{>>>} and the colon are blue. 60 | 61 | 62 | \subsection{Print Tables} 63 | 64 | You can use \lstinline{prettytable} to print a table on the command-line. 65 | 66 | \lstinputlisting[caption=display\_tables.py]{../third_party/display_tables.py} 67 | 68 | \begin{lstlisting}[caption=Output of display\_tables.py] 69 | $ python display_tables.py 70 | +------+-------+ 71 | | food | price | 72 | +------+-------+ 73 | | ham | $2 | 74 | | eggs | $1 | 75 | | spam | $4 | 76 | +------+-------+ 77 | \end{lstlisting} 78 | 79 | 80 | \subsection{Fancy CLI Header} 81 | 82 | Ever wondered how other projects create those fancy cli headers? 83 | You can use \lstinline{pyfiglet} to achieve exactly that! 84 | 85 | \lstinputlisting[caption=fancy\_cli\_header.py]{../third_party/fancy_cli_header.py} 86 | 87 | The following output is not included as image. 88 | It's copied and pasted directly from the command-line. 89 | So if you recognize a non-fancy header, make sure to try it yourself to get a better impression of what you can achieve with this recipe. 90 | 91 | \begin{minipage}{\textwidth} 92 | \begin{lstlisting}[caption=Output of fancy\_cli\_header.py] 93 | __ __ __ 94 | / /____ _ __/ /_ / /_____ 95 | / __/ _ \| |/_/ __/ / __/ __ \ 96 | / /_/ __/> ", "", 1)) 14 | 15 | 16 | # Returns the User ID of a Medium Username 17 | def get_user_id(username): 18 | 19 | print("Retrieving user ID...") 20 | 21 | url = MEDIUM + "/@" + username + "?format=json" 22 | response = requests.get(url) 23 | response_dict = clean_json_response(response) 24 | return response_dict["payload"]["user"]["userId"] 25 | 26 | 27 | # Returns the list of Usernames from a user's Followings list 28 | def get_list_of_followings(user_id): 29 | 30 | print("Retrieving users from Followings...") 31 | 32 | next_id = False 33 | followings = [] 34 | 35 | while True: 36 | 37 | if next_id: 38 | # If this is not the first page of the followings list 39 | url = ( 40 | MEDIUM + "/_/api/users/" + user_id + "/following?limit=8&to=" + next_id 41 | ) 42 | else: 43 | # If this is the first page of the followings list 44 | url = MEDIUM + "/_/api/users/" + user_id + "/following" 45 | 46 | response = requests.get(url) 47 | response_dict = clean_json_response(response) 48 | 49 | for user in response_dict["payload"]["value"]: 50 | followings.append(user["username"]) 51 | 52 | try: 53 | # If the "to" key is missing, we've reached the end of the list and an exception is thrown 54 | next_id = response_dict["payload"]["paging"]["next"]["to"] 55 | except: 56 | break 57 | 58 | return followings 59 | 60 | 61 | # Returns the list of IDs of the latest posts of a list of users 62 | def get_list_of_latest_posts_ids(usernames): 63 | 64 | print("Retrieving the latest posts...") 65 | 66 | post_ids = [] 67 | 68 | for username in usernames: 69 | url = MEDIUM + "/@" + username + "/latest?format=json" 70 | response = requests.get(url) 71 | response_dict = clean_json_response(response) 72 | 73 | try: 74 | posts = response_dict["payload"]["references"]["Post"] 75 | except: 76 | posts = [] 77 | 78 | if posts: 79 | for key in posts.keys(): 80 | post_ids.append(posts[key]["id"]) 81 | 82 | return post_ids 83 | 84 | 85 | # Returns the list of post responses of a list of posts that are no older than 1 month 86 | def get_post_responses(posts): 87 | 88 | print("Retrieving the post responses...") 89 | 90 | responses = [] 91 | 92 | for post in posts: 93 | url = MEDIUM + "/_/api/posts/" + post + "/responses" 94 | response = requests.get(url) 95 | response_dict = clean_json_response(response) 96 | responses += response_dict["payload"]["value"] 97 | sleep( 98 | 0.5 99 | ) # This is the most intensive operation for the Medium servers, we'll help them out 100 | 101 | return responses 102 | 103 | 104 | # Checks if a response was created in the last 30 days 105 | def check_if_recent(response): 106 | 107 | limit_date = datetime.now() - timedelta(days=30) 108 | creation_epoch_time = response["createdAt"] / 1000 109 | creation_date = datetime.fromtimestamp(creation_epoch_time) 110 | 111 | if creation_date >= limit_date: 112 | return True 113 | 114 | 115 | # Checks if a response is over a certain number of recommends 116 | def check_if_high_recommends(response, recommend_min): 117 | if response["virtuals"]["recommends"] >= recommend_min: 118 | return True 119 | 120 | 121 | # Returns the list of User IDs of a list of responses that have over a certain number of recommends 122 | def get_user_ids_from_responses(responses, recommend_min): 123 | 124 | print("Retrieving user IDs from the responses...") 125 | 126 | user_ids = [] 127 | 128 | for response in responses: 129 | recent = check_if_recent(response) 130 | high_recommends = check_if_high_recommends(response, recommend_min) 131 | if recent and high_recommends: 132 | user_ids.append(response["creatorId"]) 133 | 134 | return user_ids 135 | 136 | 137 | # Returns the list of usernames of a list of User IDs 138 | def get_usernames(user_ids): 139 | 140 | print("Retrieving usernames of interesting users...") 141 | 142 | usernames = [] 143 | 144 | for user_id in user_ids: 145 | url = MEDIUM + "/_/api/users/" + user_id 146 | response = requests.get(url) 147 | response_dict = clean_json_response(response) 148 | usernames.append(response_dict["payload"]["value"]["username"]) 149 | 150 | return usernames 151 | 152 | 153 | # Adds list of interesting users to the interesting_users.csv and adds a timestamp 154 | def list_to_csv(interesting_users_list): 155 | with open("interesting_users.csv", "a") as file: 156 | writer = csv.writer(file) 157 | 158 | now = datetime.now().strftime("%Y-%m-%d %H:%M:%S") 159 | interesting_users_list.insert(0, now) 160 | 161 | writer.writerow(interesting_users_list) 162 | 163 | 164 | # Returns a list of usernames in your network that might be interesting to interact with 165 | def get_interesting_users(username, recommend_min): 166 | 167 | print("Looking for interesting users for %s..." % username) 168 | 169 | user_id = get_user_id(username) 170 | 171 | usernames = get_list_of_followings(user_id) 172 | 173 | posts = get_list_of_latest_posts_ids(usernames) 174 | 175 | responses = get_post_responses(posts) 176 | 177 | users = get_user_ids_from_responses(responses, recommend_min) 178 | 179 | return get_usernames(users) 180 | 181 | 182 | @click.command() 183 | @click.option("-n", "--name", default="DahlitzF", help="Medium username") 184 | @click.option( 185 | "-r", 186 | "--min-recommendations", 187 | default=10, 188 | help="Minimum number of recommendations per response", 189 | ) 190 | def main(name, min_recommendations): 191 | interesting_users = get_interesting_users(name, min_recommendations) 192 | print(interesting_users) 193 | list_to_csv(interesting_users) 194 | 195 | 196 | if __name__ == "__main__": 197 | main() 198 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/dicts.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Dictionaries} 4 | 5 | When dealing with dictionaries the following snippets might be helpful - or just fascinating. 6 | 7 | 8 | \subsection{Crazy Dictionary Expression} 9 | 10 | The following snippet shows you a crazy dictionary expression. 11 | Maybe the most craziest dictionary expression you've seen so far. 12 | 13 | \lstinputlisting[caption=crazy\_dict\_expression.py]{../standard_lib/crazy_dict_expression.py} 14 | 15 | If you're not familiar with dictionary keys, the following output might confuse you. 16 | 17 | \begin{lstlisting}[caption=Output of crazy\_dict\_expression.py,label=lst:crazydictoutput] 18 | $ python crazy_dict_expression.py 19 | {True: 'maybe'} 20 | \end{lstlisting} 21 | 22 | Dictionary keys are compared by their hash values. 23 | As \lstinline{True}, \lstinline{1} and \lstinline{1.0} have the same hash values, the keys value gets overwritten. 24 | \lstinline{True} is inserted as no key with the same hash value exists in the dictionary so far. 25 | Inserting \lstinline{1} will lead to an overwriting of the value of \lstinline{True} as both share the same hash value. 26 | This results in the following dictionary: 27 | 28 | \begin{lstlisting} 29 | {True: "no"} 30 | \end{lstlisting} 31 | 32 | Last but not least \lstinline{1.0} with its corresponding value is inserted. 33 | The behaviour is similar to the insertion of \lstinline{1} resulting in the final output shown in Listing~\ref{lst:crazydictoutput}. 34 | 35 | 36 | \subsection{Emulate switch-case} 37 | 38 | A \lstinline{switch-case} statement doesn't exist in Python, but you can easily emulate it using a dictionary. 39 | By using \mbox{\lstinline{.get()}} you can provide a default return value if a \lstinline{KeyError} is raised. 40 | 41 | \lstinputlisting[caption=emulate\_switch\_case.py]{../standard_lib/emulate_switch_case.py} 42 | 43 | 44 | \subsection{Merge Abitrary Number of Dicts} 45 | 46 | You can use the following snippet to merge an arbitrary number of dictionaries using the \lstinline{**} operator. 47 | 48 | \lstinputlisting[caption=merge\_arbitrary\_number\_of\_dicts.py]{../standard_lib/merge_arbitrary_number_of_dicts.py} 49 | 50 | Note that later dictionaries are overwriting existing key-value-pairs. 51 | 52 | \begin{lstlisting}[caption=Output of merge\_arbitrary\_number\_of\_dicts.py] 53 | $ python merge_arbitrary_number_of_dicts.py 54 | Result of merging dict 1-4: {'a': 2, 'b': 3, 'c': 4, 3: 9, 'd': 8, 'e': 1} 55 | \end{lstlisting} 56 | 57 | 58 | \subsection{MultiDict with Default Values} 59 | 60 | Sometimes you simply want to store several values for a single key. 61 | \lstinline{MultiDict} is the perfect data structure for that. 62 | However, a \lstinline{MultiDict} implementation is not provided by the Python standard library. 63 | You can implement your own by using lists as dict values. 64 | However, when inserting a value to a key, which doesn't exist so far, a \lstinline{KeyError} is raised. 65 | Using a \lstinline{defaultdict} solves the problem as a new empty list is added if the key not already exists. 66 | 67 | \lstinputlisting[caption=multidict\_with\_default\_init.py]{../standard_lib/multidict_with_default_init.py} 68 | 69 | The output shows what we expect. 70 | 71 | \begin{lstlisting}[caption=Output of multidict\_with\_default\_init.py] 72 | $ python multidict_with_default_init.py 73 | defaultdict(, {1: ['ichi', 'one', 'uno', 'un']}) 74 | \end{lstlisting} 75 | 76 | 77 | \subsection{Overwrite Dictionary Values} 78 | 79 | Loving Python~3.5 PEP~448 for overwriting a dictionary of default values, you can use the following snippet to do exactly that! 80 | It makes use of the very same operator used when merging dictionaries: \lstinline{**}. 81 | 82 | \lstinputlisting[caption=overwrite\_dictionary.py]{../standard_lib/overwrite_dictionary.py} 83 | 84 | You may use it when you provide a dictionary containing default configuration settings and want to overwrite them with custom settings. 85 | 86 | \begin{lstlisting}[caption=Output of overwrite\_dictionary.py] 87 | $ python overwrite_dictionary.py 88 | {'lenny': 'yellow', 'carl': 'black'} 89 | \end{lstlisting} 90 | 91 | 92 | \subsection{Pass Multiple Dicts as Argument} 93 | 94 | If you want to pass multiple dicts as argument for a certain function, you can make use of the \lstinline{**} operator for auto-unpacking. 95 | However, there are two ways to achieve that depending on whether you want dictionary keys to be overwritten or not. 96 | 97 | \lstinputlisting[caption=pass\_multiple\_dicts.py]{../standard_lib/pass_multiple_dicts.py} 98 | 99 | Using a \lstinline{ChainMap} doesn't overwrite earlier specified key-value-pairs. 100 | The second option overwrites key-value-pairs. 101 | 102 | \begin{lstlisting}[caption=Output of pass\_multiple\_dicts.py] 103 | $ python pass_multiple_dicts.py 104 | 5 6 8 9 105 | 5 7 8 9 106 | \end{lstlisting} 107 | 108 | 109 | \subsection{Default Configuration} 110 | 111 | The following snippetshows you, how you can provide a default config, overwrite it with a config file and overwrite the result with command-line arguments. 112 | All this can be achieved by using \lstinline{ChainMap}. 113 | 114 | \lstinputlisting[caption=provide\_default\_config\_values.py]{../standard_lib/provide_default_config_values.py} 115 | 116 | 117 | \subsection{Keep Original After Updates} 118 | 119 | You may want to update a dictionaries values without loosing the original values. 120 | This can be achieved by using a \lstinline{ChainMap}. 121 | Therefore, we chain an empty dictionary and the one containing the original values. 122 | Now we are only operating with the \lstinline{ChainMap}. 123 | 124 | \lstinputlisting[caption=save\_dict\_update\_without\_loosing\_original.py]{../standard_lib/save_dict_update_without_loosing_original.py} 125 | 126 | If we insert new values, they are added to the \lstinline{changes} dict. 127 | If we try to retrieve values, which aren't part of the \lstinline{changes} dict, we fall back and use the once from the original dict. 128 | 129 | \begin{lstlisting}[caption=Output of save\_dict\_update\_without\_loosing\_original.py] 130 | $ python save_dict_update_without_loosing_original.py 131 | Before update: 127 132 | Updated population: 128 133 | Original population: 127 134 | Changes: dict_keys(['japan']) 135 | Did not changed: {'italy', 'uk'} 136 | \end{lstlisting} 137 | 138 | 139 | \subsection{Update Dict Using Tuples} 140 | 141 | You can update a dictionaries values by using tuples. 142 | 143 | \lstinputlisting[caption=update\_dict\_using\_tuples.py]{../standard_lib/update_dict_using_tuples.py} 144 | 145 | \begin{lstlisting}[caption=Output of update\_dict\_using\_tuples.py] 146 | $ python update_dict_using_tuples.py 147 | {1: '1', 2: '2', 3: '3'} 148 | \end{lstlisting} 149 | 150 | 151 | \subsection{Create A Dict Based On Lists} 152 | 153 | Let's assume you have two lists and you want to create a dictionary out of them. 154 | The first list might contain the letters 1-26 and the second list might contain the characters a-z. 155 | So you want to map a character to each numbers to finally end up with a dictionary representing the alphabet. 156 | This recipe shows you how to achieve that in a very simple manner. 157 | 158 | \lstinputlisting[caption=dict\_based\_on\_lists.py]{../standard_lib/dict_based_on_lists.py} 159 | 160 | As we are using \lstinline{json.dumps} with the argument \lstinline{sortkeys=True}, the final dictionary is sorted and looks like this: 161 | 162 | \begin{lstlisting}[caption=Output of dict\_based\_on\_lists.py] 163 | $ python dict_based_on_lists.py 164 | { 165 | "1": "a", 166 | "2": "b", 167 | "3": "c" 168 | } 169 | \end{lstlisting} 170 | -------------------------------------------------------------------------------- /standard_lib/MicroWebServer.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import threading 3 | import os.path 4 | import re 5 | 6 | HTTPVER = "HTTP/1.1" 7 | SERVER = "PythonMicroWebServer/0.1" 8 | LINEBREAK = "\r\n" 9 | 10 | 11 | class WebServer(object): 12 | def __init__(self, hostname="", port=80, document_root="./"): 13 | self._connectionHandler = WebServer.ConnectionHandler( 14 | hostname, port, document_root 15 | ) 16 | 17 | def server_forever(self): 18 | self._connectionHandler.start() 19 | 20 | def clean_exit(self): 21 | self._connectionHandler.stop() 22 | 23 | def route(self, path, **options): 24 | def decorator(func): 25 | self._connectionHandler.routes.append((path, func)) 26 | 27 | return decorator 28 | 29 | class ConnectionHandler(threading.Thread): 30 | def __init__(self, hostname, port, document_root): 31 | self._hostname = hostname 32 | self._port = port 33 | self._document_root = document_root 34 | 35 | self.routes = [] 36 | 37 | self.running = False 38 | 39 | threading.Thread.__init__(self) 40 | 41 | def stop(self): 42 | self.running = False 43 | 44 | def run(self): 45 | self.running = True 46 | 47 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 48 | sock.bind((self._hostname, self._port)) 49 | sock.listen(1) 50 | 51 | while self.running: 52 | client, address = sock.accept() 53 | 54 | raw_data = client.recv(1024) 55 | 56 | if not raw_data: 57 | continue 58 | 59 | http_request = WebServer.HTTPRequest.parse(raw_data) 60 | 61 | print( 62 | "%s [%s] requested %s%s" 63 | % ( 64 | address[0], 65 | http_request.header["User-Agent"], 66 | http_request.host, 67 | http_request.uri, 68 | ) 69 | ) 70 | 71 | file = http_request.uri 72 | 73 | if file in [x[0] for x in self.routes]: 74 | for x in self.routes: 75 | if x[0] == file: 76 | x[1](client) 77 | break 78 | continue 79 | 80 | if os.path.isfile(self._document_root + file): 81 | with open(self._document_root + file, "rb") as f: 82 | http_response = WebServer.HTTPResponse( 83 | f.read(), 84 | content_type=WebServer.HTTPResponse.file_extension_to_content_type( 85 | file.split(".")[-1] 86 | ), 87 | ) 88 | f.close() 89 | 90 | elif os.path.isfile(self._document_root + "error404.html"): 91 | with open(self._document_root + "error404.html", "rb") as f: 92 | http_response = WebServer.HTTPResponse( 93 | f.read(), code="404 NOT FOUND" 94 | ) 95 | f.close() 96 | 97 | else: 98 | http_response = WebServer.HTTPResponse( 99 | "

404 Not Found

404 Error not found...

", 100 | code="404 NOT FOUND", 101 | ) 102 | 103 | client.send(http_response.raw) 104 | client.close() 105 | 106 | sock.close() 107 | 108 | class HTTPRequest(object): 109 | def __init__(self, http_ver, host, request_type, uri, header): 110 | self.http_ver = http_ver 111 | self.host = host 112 | self.request_type = request_type 113 | self.uri = uri 114 | self.header = header 115 | 116 | @staticmethod 117 | def parse(data): 118 | data = data.decode("utf-8") 119 | lines = data.splitlines() 120 | 121 | request_type, uri, http_ver = lines[0].split() 122 | 123 | header = {} 124 | 125 | for h in lines: 126 | h = h.split(": ") 127 | 128 | if len(h) < 2: 129 | continue 130 | 131 | field = h[0] 132 | value = h[1] 133 | 134 | header[field] = value 135 | 136 | host = header["Host"] 137 | 138 | if "?" in uri: 139 | uri = uri.split("?")[0] 140 | 141 | print(uri) 142 | 143 | if uri.endswith("/"): 144 | uri += "index.html" 145 | 146 | return WebServer.HTTPRequest(http_ver, host, request_type, uri, header) 147 | 148 | class HTTPResponse(object): 149 | def __init__(self, body, content_type="text/html", code="200 OK"): 150 | response = HTTPVER + " " + code + LINEBREAK 151 | response += "SERVER: " + SERVER + LINEBREAK 152 | response += "Connection: close" + LINEBREAK 153 | response += "Content-Type: " + content_type + LINEBREAK 154 | response += LINEBREAK 155 | 156 | response = response.encode("utf-8") 157 | self.header = response 158 | 159 | if type(body) == str: 160 | response += body.encode("utf-8") 161 | else: 162 | response += body 163 | 164 | self.body = body 165 | self.content_type = content_type 166 | self.code = code 167 | 168 | self.raw = response 169 | 170 | @staticmethod 171 | def file_extension_to_content_type(file_extension): 172 | if file_extension in ["html", "htm"]: 173 | return "text/html" 174 | if file_extension in ["js"]: 175 | return "text/javascript" 176 | if file_extension in ["css"]: 177 | return "text/css" 178 | 179 | if file_extension in ["jpeg", "jpg", "jpe"]: 180 | return "image/jpeg" 181 | if file_extension in ["png"]: 182 | return "image/png" 183 | if file_extension in ["ico"]: 184 | return "image/x-ico" 185 | if file_extension in ["gif"]: 186 | return "image/gif" 187 | if file_extension in ["cod"]: 188 | return "image/cis-cod" 189 | if file_extension in ["ras"]: 190 | return "image/cmu-raster" 191 | if file_extension in ["fif"]: 192 | return "image/fif" 193 | if file_extension in ["ief"]: 194 | return "image/ief" 195 | if file_extension in ["tiff", "tif"]: 196 | return "image/tiff" 197 | if file_extension in ["mcf"]: 198 | return "image/vasa" 199 | if file_extension in ["wbmp"]: 200 | return "image/vnd.wap.wbmp" 201 | if file_extension in ["fh4", "fh5", "fhc"]: 202 | return "image/x-freehand" 203 | if file_extension in ["pnm"]: 204 | return "image/x-portable-anymap" 205 | if file_extension in ["pbm"]: 206 | return "image/x-portable-bitmap" 207 | if file_extension in ["pgm"]: 208 | return "image/x-portable-graymap" 209 | if file_extension in ["ppm"]: 210 | return "image/x-portable-pixmap" 211 | if file_extension in ["rgb"]: 212 | return "image/x-rgb" 213 | if file_extension in ["xwd"]: 214 | return "image/x-windowdump" 215 | if file_extension in ["xbm"]: 216 | return "image/x-bitmap" 217 | if file_extension in ["xpm"]: 218 | return "image/x-pixmap" 219 | 220 | if file_extension in ["wmv"]: 221 | return "video/x-ms-wmv" 222 | if file_extension in ["avi"]: 223 | return "video/x-msvideo" 224 | if file_extension in ["mpeg", "mpg", "mpe"]: 225 | return "video/mpeg" 226 | if file_extension in ["qt", "mov"]: 227 | return "video/quicktime" 228 | if file_extension in ["viv", "vivo"]: 229 | return "video/vnd.vivo" 230 | if file_extension in ["movie"]: 231 | return "video/x-sgi-movie" 232 | 233 | return "application/octet-stream" 234 | 235 | 236 | if __name__ == "__main__": 237 | webserver = WebServer(document_root="./web/") 238 | 239 | @webserver.route("/*") 240 | def data(client): 241 | http_response = WebServer.HTTPResponse("

hallo pual :o

") 242 | client.send(http_response.raw) 243 | client.close() 244 | 245 | webserver.server_forever() 246 | input("Press any key to stop...") 247 | webserver.clean_exit() 248 | -------------------------------------------------------------------------------- /ebook/chapters/third_party_chapters/non_categorized.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Non-Categorized} 4 | 5 | All recipes, which do not fall into one of the underlying categories, are listed here. 6 | 7 | \subsection{Auto Login On Website} 8 | 9 | Using \lstinline{selenium} allows you to automatically open a new browser window and login into a certain website, e.g. GitHub. 10 | 11 | \lstinputlisting[caption=auto\_login\_website.py]{../third_party/auto_login_website.py} 12 | 13 | 14 | \subsection{Count Python Bytes In A Directory} 15 | 16 | \lstinline{PyFilesystem2} is Python's file system abstraction layer. 17 | You can use a file system object to analyse your files. 18 | The folowing recipe shows you how you can get the number of Python source code bytes in your directory. 19 | 20 | \lstinputlisting[caption=count\_python\_bytes.py]{../third_party/count_python_bytes.py} 21 | 22 | 23 | \subsection{Create World Maps} 24 | 25 | \lstinline{folium} builds on the data wrangling strengths of the Python ecosystem and the mapping strengths of the Leaflet.js library. 26 | Manipulate your data in Python, then visualize it in a Leaflet map via \lstinline{folium}. 27 | This recipe creates a world map with the USA in the centre of the map. 28 | 29 | \lstinputlisting[caption=folium\_snippet.py]{../third_party/folium_snippet.py} 30 | 31 | 32 | \subsection{Print Formatted JSON} 33 | 34 | Printing JSON object formatted is as easy as: 35 | 36 | \lstinputlisting[caption=formatted\_json.py]{../third_party/formatted_json.py} 37 | 38 | \begin{lstlisting}[caption=Output of formatted\_json.py] 39 | $ python formatted_json.py 40 | { 41 | "userId": 1, 42 | "id": 1, 43 | "title": "delectus aut autem", 44 | "completed": false 45 | } 46 | \end{lstlisting} 47 | 48 | \textbf{Note:} The snippet is part of the third party part as it makes use of the \lstinline{requests} library. 49 | The \lstinline{json} module is part of Pythons standard library. 50 | 51 | 52 | \subsection{Inspect Docker} 53 | 54 | You can inspect running Docker containers and existing images using the \lstinline{docker} module. 55 | 56 | \lstinputlisting[caption=inspect\_docker.py]{../third_party/inspect_docker.py} 57 | 58 | 59 | \subsection{Is Holiday} 60 | 61 | The \lstinline{holidays} module provides you an elegant and easy way to check, whether a given date is a holiday in the specified region. 62 | 63 | \lstinputlisting[caption=is\_holiday.py]{../third_party/is_holiday.py} 64 | 65 | 66 | \subsection{Web Scraping} 67 | 68 | The recipe in the \lstinline{mathematicians.py} file shows you how you can scrape information from the internet. 69 | It makes use of the following third party packages: 70 | 71 | \begin{itemize} 72 | \item BeautifulSoup 73 | \item requests 74 | \end{itemize} 75 | 76 | The recipe returns you the most popular mathematicians. 77 | As it's to long, please have a look at it in the repository. 78 | 79 | 80 | \subsection{Interacting With The Medium API} 81 | 82 | Another recipe, which is to long but worth to mention, is the one contianed by the \lstinline{medium.py} file. 83 | Running the file gives you the ability to interact with the Medium API via the command-line. 84 | 85 | 86 | \subsection{Mocking Requests} 87 | 88 | When writing tests for your application, you may come across the situation, where you have to mock requests. 89 | The following Listing shows you how you can do this by using the standard libraries \lstinline{unittest.mock} module. 90 | 91 | \lstinputlisting[caption=mocking\_requests.py]{../third_party/mocking_requests.py} 92 | 93 | 94 | \subsection{Mypy Example} 95 | 96 | The following Listing reveals the usage of \lstinline{mypy} as a static type checker. 97 | Running the snippet will throw a \lstinline{TypeError} as expected. 98 | 99 | \lstinputlisting[caption=mypy\_example.py]{../third_party/mypy_example.py} 100 | 101 | 102 | \subsection{NumPy Array Operations} 103 | 104 | Running the following code snippet reveals you some of the existing \lstinline{numpy} array operations. 105 | 106 | \lstinputlisting[caption=numpy\_array\_operations.py]{../third_party/numpy_array_operations.py} 107 | 108 | \begin{lstlisting}[caption=Output of numpy\_array\_operations.py] 109 | $ python numpy_array_operations.py 110 | [ 2 4 6 8 10] 111 | [ 1 4 9 16 25] 112 | \end{lstlisting} 113 | 114 | 115 | \subsection{Parse Complex Excel Sheets} 116 | 117 | You can parse more complex Excel Sheets by using \lstinline{pandas} and installed a \lstinline{pandas} extension called \lstinline{xlrd}. 118 | 119 | \lstinputlisting[caption=parse\_complex\_excel\_sheets.py]{../third_party/parse_complex_excel_sheets.py} 120 | 121 | 122 | \subsection{Nmap For Python} 123 | 124 | Creating your own port scanner becomes easy when using \lstinline{nmap}. 125 | Luckily, a Python package exists providing access to nmap: \lstinline{python-nmap}. 126 | \textbf{Hint:} Make sure you have Nmap installed on your operating system as \lstinline{python-nmap} only provides access to the Nmap API. 127 | 128 | \lstinputlisting[caption=port\_scanner\_nmap.py]{../third_party/port_scanner_nmap.py} 129 | 130 | 131 | \subsection{Test Renamed Class} 132 | 133 | Renaming a class can happen over time. 134 | If you want to test, wether your code is backwards compatible, you can make use of the following snippet. 135 | 136 | \lstinputlisting[caption=pytest\_rename\_class\_backwards\_compatibility.py]{../third_party/pytest_rename_class_backwards_compatibility.py} 137 | 138 | 139 | \subsection{Reduce Pandas Dataframe Memory} 140 | 141 | When dealing with large data sets, it can be an advantage to change column types from \lstinline{object} to \lstinline{category} as shown in the next Listing. 142 | 143 | \lstinputlisting[caption=reduce\_pandas\_df\_memory.py]{../third_party/reduce_pandas_df_memory.py} 144 | 145 | 146 | \subsection{Async Libraries} 147 | 148 | Before \lstinline{async}/\lstinline{await} became so popular and part of the standard library, you had several options to use asynchronous techniques in your project. 149 | The three main ones were \lstinline{asynio}, \lstinline{gevent} and \lstinline{tornado}. 150 | You can find an example for each in the \textit{third\_party} directory of the repository. 151 | The file names are as follows: 152 | 153 | \begin{itemize} 154 | \item \lstinline{test_asyncio.py} 155 | \item \lstinline{test_gevent.py} 156 | \item \lstinline{test_tornado.py} 157 | \end{itemize} 158 | 159 | 160 | \subsection{Unzip World Bank Data} 161 | 162 | The following recipe shows you how you can download a \lstinline{.zip} file from an online source, extract and work with the data. 163 | Therefore, except the \lstinline{requests} library only standard library modules are used. 164 | 165 | \lstinputlisting[caption=world\_bank\_data.py]{../third_party/world_bank_data.py} 166 | 167 | 168 | \subsection{Simple Debugger} 169 | 170 | When writing Python code you may come across situations, where you want to find out, what's going on. 171 | Sometimes you may make use of an actual debugger, but most of the times inserting some \lstinline{print}-statements seems just fine. 172 | The \lstinline{PySnooper} package is probably the best reason, why you should never use \lstinline{print}-statements to debug your code again. 173 | The following Listing shows you the simple usage of its \lstinline{snoop} decorator. 174 | 175 | \lstinputlisting[caption=simple\_debugger.py]{../third_party/simple_debugger.py} 176 | 177 | The output is too long to be covered here. 178 | Feel free to run the snippet from your terminal and get a sense of how amazing and helpful this little package can be. 179 | 180 | 181 | \subsection{Text Analysis} 182 | 183 | If you want a simple way to perform text analysis, you can use \lstinline{TextBlob}. 184 | The following snippet shows you a sample usage. 185 | Make sure to read the docs of the package as it provides functionality for translations and further analysis, too. 186 | 187 | \lstinputlisting[caption=text\_analysis.py]{../third_party/text_analysis.py} 188 | 189 | \textbf{Note:} Make sure to download the needed data before hand, otherwise an exception is thrown telling you to download it. 190 | 191 | \begin{lstlisting}[caption=Download data using NLTK] 192 | >>> import nltk 193 | >>> nltk.download('averaged_perceptron_tagger') 194 | \end{lstlisting} 195 | 196 | 197 | \subsection{Complex List Ops} 198 | 199 | In your day to day work you will very likely come across performance issues in Python. 200 | Sometimes it's a good idea to choose a different data structure than you have used to speed up certain operations. 201 | However, you may come across situations where this is even not enough. 202 | Instead of switching to another language like C, you may want to check out the \lstinline{blist} package first. 203 | \glqq The \lstinline{blist} is a drop-in replacement for the Python list that provides better performance when modifying large lists.\grqq 204 | Here is a small snippet illustrating the huge impact of using a \lstinline{blist} instead of a \lstinline{list}: 205 | 206 | \lstinputlisting[caption=complex\_list\_ops.py]{../third_party/complex_list_ops.py} 207 | 208 | \begin{lstlisting}[caption=Output of complex\_list\_ops.py] 209 | $ python complex_list_ops.py 210 | "Builtin" spend time: 24.61005 211 | "Blist" spend time: 4.675813 212 | \end{lstlisting} 213 | 214 | \glqq The blist package also provides \lstinline{sortedlist}, \lstinline{sortedset}, [...] \lstinline{btuple} types\grqq , and much more. 215 | -------------------------------------------------------------------------------- /ebook/chapters/standard_lib_chapters/non_categorized.tex: -------------------------------------------------------------------------------- 1 | % !TeX root = ../../python-snippets.tex 2 | 3 | \section{Non-Categorized} 4 | 5 | This section includes all those snippets not belonging to any of the underlying categories. 6 | 7 | 8 | \subsection{Turtle} 9 | 10 | The \lstinline{turtle} module provides turtle graphics primitives, in both object-oriented and procedure-oriented ways. 11 | Because it uses \lstinline{tkinter} for the underlying graphics, it needs a version of Python installed with Tk support. 12 | The following Listing shows you a sample implementation of the \lstinline{turtle} module drawing a dragon. 13 | 14 | \lstinputlisting[caption=drawing\_turtle.py]{../standard_lib/drawing_turtle.py} 15 | 16 | 17 | \subsection{Function Parameters} 18 | 19 | There's not much to say about the following snippet. 20 | It shows the different usages of positional and keyword arguments. 21 | The last call shows you how \textit{not} to call a function with positional and keyword arguments. 22 | So don't be confused if a \lstinline{TypeError} is raised. 23 | 24 | \lstinputlisting[caption=function\_arguments.py]{../standard_lib/function_arguments.py} 25 | 26 | 27 | \subsection{Password Input} 28 | 29 | Python has a module called \lstinline{getpass}, which lets you take user input without printing the typed characters. 30 | 31 | \lstinputlisting[caption=get\_password\_input.py]{../standard_lib/get_password_input.py} 32 | 33 | 34 | \subsection{Hex Decode} 35 | 36 | You can decode hex-code in Python as follows. 37 | 38 | \lstinputlisting[caption=hex\_decode.py]{../standard_lib/hex_decode.py} 39 | 40 | \begin{lstlisting}[caption=Output of hex\_decode.py] 41 | $ python hex_decode.py 42 | b'Merry Christmas!' 43 | \end{lstlisting} 44 | 45 | 46 | \subsection{MicroWebServer} 47 | 48 | The implementation of a micro web server is stored in the \lstinline{MicroWebServer.py} file. 49 | As this file is much longer than usual snippets and it's not meant to discussed it here, I'm referring the source code repo to see and test the program on your own. 50 | 51 | 52 | \subsection{Open Browser Tab} 53 | 54 | You can control a browser through the \lstinline{webbrowser} module. 55 | If you want to open a new browser tab, you can simply run the code of the following Listing. 56 | 57 | \textbf{Note:} Only browsers, which are part of the \lstinline{PATH} variable, can be found 58 | 59 | \lstinputlisting[caption=open\_browser\_tab.py]{../standard_lib/open_browser_tab.py} 60 | 61 | 62 | \subsection{Port Scanner} 63 | 64 | In the \lstinline{port_scanner.py} file you find an implementation of a very basic port scanner. 65 | As the file is to large, it's not displayed here. 66 | Feel free to use the port scanner. 67 | 68 | 69 | \subsection{Reduce Memory Consumption - Customizing \_\_slots\_\_} 70 | 71 | Every class has a \lstinline{__slots__} attribute. 72 | This attribute is quite big by default. 73 | By specifying a custom \lstinline{__slots__} attribute, you can reduce the memory consumption. 74 | As the snippet is quite large, I just refer to file in the repo: \lstinline{reduce_memory_consumption.py}. 75 | However, I want to show you the output of the snippet: 76 | 77 | \begin{lstlisting}[caption=Output of reduce\_memory\_consumption.py] 78 | $ python reduce_memory_consumption.py 79 | [With __slots__] Total allocated size: 6.9 MB 80 | [Without __slots__] Total allocated size: 16.8 MB 81 | \end{lstlisting} 82 | 83 | 84 | \subsection{Reduce Memory Consumption - Using Iterator} 85 | 86 | You can reduce the memory consumption by using iterators whenever possible. 87 | 88 | \lstinputlisting[caption=reduce\_memory\_consumption\_iterator.py]{../standard_lib/reduce_memory_consumption_iterator.py} 89 | 90 | Just have a look at the resulting output, it speaks for itself. 91 | 92 | \begin{lstlisting}[caption=Output of reduce\_memory\_consumption\_iterator.py] 93 | $ python reduce_memory_consumption_iterator.py 94 | Using itertools.repeat: 56 bytes 95 | Using list with 100.000.000 elements: 762.9395141601562 MB 96 | \end{lstlisting} 97 | 98 | 99 | \subsection{RegEx Parse Tree} 100 | 101 | Print the ReqEx parse tree using \lstinline{re.DEBUG}. 102 | 103 | \lstinputlisting[caption=regular\_expression\_debug.py]{../standard_lib/regular_expression_debug.py} 104 | 105 | 106 | \subsection{Scopes} 107 | 108 | The snippet \lstinline{scopes_namespaces.py} contains an example to demonstrate the different scopes and namespaces available in Python. 109 | As it's only for demonstarting purposes and as the file is quite huge, it's not displayed here. 110 | 111 | 112 | \subsection{Set Union and Intersection} 113 | 114 | Python provides set union and intersection using the \lstinline{&} and \lstinline{|} operators. 115 | 116 | \lstinputlisting[caption=set\_union\_intersection.py]{../standard_lib/set_union_intersection.py} 117 | 118 | \begin{lstlisting}[caption=set\_union\_intersection.py] 119 | $ python set_union_intersection.py 120 | {1, 2} & {2, 3} = {2} 121 | {1, 2} | {2, 3} = {1, 2, 3} 122 | \end{lstlisting} 123 | 124 | 125 | \subsection{Sort Complex Tuples} 126 | 127 | If you have a list of more complex tuples, you may want to sort them by a certain key. 128 | You can provide a key function for the builtin \lstinline{sorted} function. 129 | 130 | \lstinputlisting[caption=sort\_complex\_tuples.py]{../standard_lib/sort_complex_tuples.py} 131 | 132 | \begin{lstlisting}[caption=Output of sort\_complex\_tuples.py] 133 | $ python sort_complex_tuples.py 134 | [('Dave', 'B', 10), ('Jane', 'B', 12), ('John', 'A', 15)] 135 | \end{lstlisting} 136 | 137 | 138 | \subsection{Unicode in Source Code} 139 | 140 | Python allows you to use unicode in your source, what you shouldn't do. 141 | Nevertheless, this snippet shows you a sample unicode usage in source code. 142 | 143 | \lstinputlisting[caption=unicode\_source\_code.py]{../standard_lib/unicode_source_code.py} 144 | 145 | However, this only works in the REPL or in iPython, but not in files. 146 | 147 | 148 | \subsection{UUID} 149 | 150 | The \lstinline{uuid} module provides methods to generade UUIDs. 151 | In this snippet you can find a sample implementation of UUID1. 152 | 153 | \lstinputlisting[caption=uuid1\_example.py]{../standard_lib/uuid1_example.py} 154 | 155 | The output may look like this: 156 | 157 | \begin{lstlisting}[caption=Output of uuid1\_example.py] 158 | $ python uuid1_example.py 159 | 3120c650-4e5e-11e9-be8f-dca904927157 160 | 3120ca1a-4e5e-11e9-be8f-dca904927157 161 | 3120cb0a-4e5e-11e9-be8f-dca904927157 162 | 3120cbbe-4e5e-11e9-be8f-dca904927157 163 | 3120cc72-4e5e-11e9-be8f-dca904927157 164 | \end{lstlisting} 165 | 166 | 167 | \subsection{Zip Safe} 168 | 169 | This snippet illustrates how zip is stopping if one iterable is exhausted without a warning and how to prevent it. 170 | 171 | \lstinputlisting[caption=zip\_safe.py]{../standard_lib/zip_safe.py} 172 | 173 | \begin{lstlisting}[caption=Output of zip\_safe.py] 174 | $ python zip_safe.py 175 | [(1, 'One'), (2, 'Two')] 176 | [(1, 'One'), (2, 'Two'), (3, None)] 177 | \end{lstlisting} 178 | 179 | 180 | \subsection{Tree Clone} 181 | 182 | If you have ever worked with unix systems, you might know the \lstinline{tree} command. 183 | The following Listing shows you a pure Python implementation of this command. 184 | 185 | \lstinputlisting[caption=tree\_clone.py]{../standard_lib/tree_clone.py} 186 | 187 | A sampl output might look like this: 188 | 189 | \begin{lstlisting}[caption=Output of tree\_clone.py,escapechar=@] 190 | $ python tree_clone.py 191 | /Users/florian/workspace/python/test-directory 192 | @\pmboxdrawuni{2514}@---- app 193 | @\pmboxdrawuni{2514}@---- __init__.py 194 | @\pmboxdrawuni{2514}@---- __main__.py 195 | @\pmboxdrawuni{2514}@---- main.py 196 | @\pmboxdrawuni{2514}@---- test.py 197 | @\pmboxdrawuni{2514}@---- utils.py 198 | \end{lstlisting} 199 | 200 | 201 | \subsection{pathlib implementation of os.relpath} 202 | 203 | The following Listing is a \lstinline{pathlib.Path} implementation of \lstinline{os.relpath}, which should \textit{never} be used in production. 204 | It's only meant to show a nice hacky implementation of \lstinline{os.relpath} as its not provided by default by \lstinline{pathlib.Path}. 205 | 206 | \lstinputlisting[caption=pathlib\_relpath.py]{../standard_lib/pathlib_relpath.py} 207 | 208 | The snippet assumes that you have at least two directories in your current working directory: \lstinline{foo} and \lstinline{baz}. 209 | The output is shown in the following Listing: 210 | 211 | \begin{lstlisting}[caption=Output of pathlib\_relpath.py] 212 | $ python pathlib_relpath.py 213 | . 214 | \end{lstlisting} 215 | 216 | 217 | \subsection{Slicing Generators} 218 | 219 | Generators are a pretty nice thing. 220 | For instance you can write a function returning the fibonacci numbers using \lstinline{yield} turning it into a generator, which is way more efficient than creating a function calculating a finite number of results and returning the resulting list. 221 | The problem: You're not able to slice the returned generator. 222 | Or at least you can't using the \lstinline{[]}-operator. 223 | Making use of the awesome \lstinline{itertools} module, you can! 224 | 225 | \lstinputlisting[caption=slice\_generators.py]{../standard_lib/slice_generators.py} 226 | 227 | 228 | \subsection{Count Elements Between Thresholds} 229 | 230 | If you want to get the number of elements between certain thresholds, you can make use of the following Listing. 231 | 232 | \lstinputlisting[caption=count\_thresholds.py]{../standard_lib/count_thresholds.py} 233 | 234 | The essential point is, that you can sum booleans as they are a subclass of integers. 235 | 236 | \begin{lstlisting}[caption=Output of count\_thresholds.py] 237 | $ python count_thresholds.py 238 | 3 239 | \end{lstlisting} 240 | 241 | 242 | \subsection{Empty Set Literal} 243 | 244 | People, who are new to Python, meeting the built-in set type for the first time often get confused when trying to create an empty set by using the \lstinline|{}| literal. 245 | This literal is already reserved for creating empty dictionaries. 246 | The normal way to create a new empty set in Python is to use the \lstinline{set()} constructor. 247 | However, if you like, you can use a literal, too! 248 | 249 | \lstinputlisting[caption=empty\_set\_literal.py]{../standard_lib/empty_set_literal.py} 250 | 251 | What happens here is that an empty list (using the list literal \lstinline{[]}) is created and unpacked using the \lstinline{*}-operator. 252 | The elements from the unpacked empty list are passed to the curly braces. 253 | Even though no elements at all are passed to the set literal, Python accepts it as valid set creation syntax. 254 | 255 | \begin{lstlisting}[caption=Output of empty\_set\_literal.py] 256 | $ python empty_set_literal.py 257 | set() 258 | \end{lstlisting} 259 | --------------------------------------------------------------------------------