├── .git-commit-template.txt └── README.md /.git-commit-template.txt: -------------------------------------------------------------------------------- 1 | Add tricks 2 | 3 | Add following topics to main README file: 4 | - Topic 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python 3 tricks 2 | 3 | Disclaimer: This list is meant as a comprehension of cool tips and tricks I found on the internet. If you would like to contribute, or notice any mistakes or typos, please contact me or upload a pull request. If you think any material here can be considered personal property let me know and I will take it down. 4 | 5 | ### 1. Looping over a range of numbers 6 | 7 | Using `range()` is better than using a list (ex. `[1, 2, 3]`), because the list takes up memory space, whereas the `range()` function generates values on demand, thus taking a fixed amount of memory whatever the size of the elements is: 8 | 9 | ```python 10 | for i in range(10): 11 | print(i**2) 12 | ``` 13 | 14 | takes the same memory space as: 15 | 16 | ```python 17 | for i in range(100000): 18 | print(i**2) 19 | ``` 20 | 21 | **Note:** This function used to create a list in python2, and `xrange()` used to do what `range()` currently does, but it got changed, so `range()` in python3 is `xrange()` in python2. 22 | 23 | ### 2. Looping backwards 24 | 25 | Use the function `reversed()`: 26 | 27 | ```python 28 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 29 | 30 | for lang in reversed(langs): 31 | print(lang) 32 | 33 | # prints rust, kotlin, c++, java, python, c 34 | ``` 35 | 36 | ### 3. Looping over a collection and indices 37 | 38 | Use the function `enumerate()`: 39 | 40 | ```python 41 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 42 | 43 | for i,lang in enumerate(langs): 44 | print(f'{i} --> {lang}') 45 | 46 | # prints 47 | # 0 --> c 48 | # 1 --> python 49 | # 2 --> java 50 | # 3 --> c++ 51 | # 4 --> kotlin 52 | # 5 --> rust 53 | ``` 54 | 55 | ### 4. Looping over two collections 56 | 57 | Use the function `zip()`. It returns tuples of the elements until one of the iterables is exahusted: 58 | 59 | ```python 60 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 61 | numbers = [1, 2, 3] 62 | 63 | for number, lang in zip(numbers, langs): 64 | print(f'{number} --> {lang}') 65 | 66 | # prints 67 | # 1 --> c 68 | # 2 --> python 69 | # 3 --> java 70 | ``` 71 | 72 | **Note:** It takes any number of iterables and "zips" them into tuples. 73 | 74 | **Note 2:** It's important to note that it generates tuples on demand, so it reuses memory space (it used to create a third list in python2, and `izip()` used to do what `zip()` does now in python3). 75 | 76 | ### 5. Looping in sorted order 77 | 78 | Use the function `sorted()` or the method `sort()` of iterables. 79 | 80 | By default, it sorts the iterable in ascending order: 81 | 82 | ```python 83 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 84 | 85 | for lang in sorted(langs): 86 | print(lang) 87 | 88 | # or 89 | 90 | langs.sort() 91 | print(*langs) 92 | 93 | # prints c c++ java kotlin python rust 94 | ``` 95 | 96 | The second method sorts the iterable IN-PLACE, whilst the first returns a different iterable. 97 | 98 | Both the functions can take 2 extra parameters which can specify a comparison function and if the iterable should be reversed: 99 | 100 | ```python 101 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 102 | 103 | for lang in sorted(langs, key=len, reverse=True): 104 | print(lang) 105 | 106 | # or 107 | 108 | langs.sort(key=len, reverse=True) 109 | print(*langs) 110 | 111 | # prints kotlin python java rust c++ c 112 | # notice that kotlin and python might be interchanged because they have the same size, same java and rust 113 | ``` 114 | 115 | ### 6. Partial functions 116 | 117 | A partial function is a function who has some parameters "frozen", in the sense that they are preset. The other parameters must be given when the partial function is called: 118 | 119 | ```python 120 | from functools import partial 121 | 122 | def func(x, y, z): 123 | return x + 2*y + 3*z 124 | 125 | my_func = partial(func, 2, 3) # assign (preset) 2 to x, 3 to y 126 | my_func(3) # equivalent to func(2, 3, 3) 127 | # prints 17 128 | my_func(4) # equivalent to func(2, 3, 4) 129 | # prints 20 130 | ``` 131 | 132 | ### 7. Fastest way to format multiple strings 133 | 134 | Going from fastest to slowest, they are: 135 | 136 | ```python 137 | f'{s} {t}' # fastest 138 | s + ' ' + t 139 | ' '.join((s, t)) 140 | '%s %s' % (s, t) 141 | '{} {}'.format(s, t) 142 | Template('$s $t').substitute(s=s, t=t) # slowest 143 | ``` 144 | 145 | **Note:** f-strings were added in Python 3.6. 146 | 147 | ### 8. Iterating until a sentinel value 148 | 149 | This method has 2 forms: 150 | 151 | 1. `iter(iterable)` - this form simply returns an iterator from the iterable. You can call `next()` on the iterator and iterate through the iterable. 152 | 153 | ```python 154 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 155 | 156 | ir = iter(langs) 157 | print(next(ir)) # prints c 158 | print(next(ir)) # prints python 159 | ``` 160 | 161 | 2. `iter(callable, sentinel)` - this form executes the function `callable` until it returns `sentinel` value. 162 | 163 | ```python 164 | def func(langs = []): 165 | langs.append('c') 166 | return len(langs) 167 | 168 | ir = iter(func, 5) 169 | next(ir) # prints 1 170 | next(ir) # prints 2 171 | next(ir) # prints 3 172 | next(ir) # prints 4 173 | next(ir) # raise StopIteration 174 | ``` 175 | 176 | Read 80 characters from file `f` into `line` and append to `text` until `f.read()` returns `''`: 177 | 178 | ```python 179 | text = list() 180 | 181 | for line in iter(partial(f.read, 80), ''): 182 | text.append(line) 183 | ``` 184 | 185 | ### 9. For else in Python 186 | 187 | Search a certain value in an iterable and do something if it is not there: 188 | 189 | ```python 190 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 191 | 192 | for lang in langs: 193 | if lang == 'scala': 194 | print('We found Scala!') 195 | break 196 | else: 197 | print('Scala is not in the list...') 198 | ``` 199 | 200 | **Note:** Careful if you ever come back to this kind of code. Don't indent the `else` statement by accident!!! 201 | 202 | ### 10. Looping over dictionary keys 203 | 204 | ```python 205 | d = { 206 | 'foo': 'c', 207 | 'bar': 'java', 208 | 'baz': 'rust' 209 | } 210 | 211 | # cannot mutate dictionary here 212 | for k in d: 213 | print(k) 214 | 215 | # free to mutate the keys and values 216 | for k in list(d.keys()): 217 | if k == 'foo': 218 | del d[k] 219 | ``` 220 | 221 | **Note:** `d.keys()` used to make a list copy of the keys, so there was no problem iterating and mutating the original dictionary at the same time. In modern Python3, `d.keys()` returns an iterable and can no longer be used to iterate and mutate a dictionary at the same time. To go around this, just wrap the method into a list as in the example. 222 | 223 | **Note 2:** There is an 'alternative' to this, but it has worse performance and memory usage: 224 | 225 | ```python 226 | d = { 227 | 'foo': 'c', 228 | 'bar': 'java', 229 | 'baz': 'rust' 230 | } 231 | 232 | # Don't do this, performance is bad as it copies every element in a dictionary and can be really bad for really big dictionaries 233 | s = {k: v for k, v in d.items() if k != 'foo'} 234 | ``` 235 | 236 | ### 11. Looping over dictionary keys and values 237 | 238 | ```python 239 | d = { 240 | 'foo': 'c', 241 | 'bar': 'java', 242 | 'baz': 'rust' 243 | } 244 | 245 | for k, v in d.items(): 246 | print(f'{k} --> {v}') 247 | ``` 248 | 249 | The `items()` method returns and iterator, so it uses the same amount of memory no matter how big the dictionary is. 250 | 251 | **Note:** In python2, the `items()` method used to return a list of tuples, and the `iteritems()` used to do what `items()` does now in python3. 252 | 253 | ### 12. Construct a dictionary from 2 iterables 254 | 255 | Use the `zip()` method to pack 2 iterables into a zip object, then use the `dict()` method to make that into a dictionary. 256 | 257 | ```python 258 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 259 | colors = ['blue', 'green', 'red'] 260 | 261 | d = dict(zip(langs, colors)) 262 | print(d) # prints {'c': 'blue', 'python': 'green', 'java': 'red'} 263 | 264 | s = dict(enumerate(colors)) 265 | print(s) # prints {1: 'blue', 2: 'green', 3: 'red'} 266 | ``` 267 | 268 | ### 13. Populating a dictionary with default data (Counting with dictionary) 269 | 270 | Use the `defaultdict()` method imported from `collections`. When a key is not in the dictionary, it creates a new key that has the default value. 271 | 272 | ```python 273 | from collections import defaultdict 274 | 275 | colors = ['red', 'green', 'red', 'blue', 'green', 'red'] 276 | 277 | d = defaultdict(int) 278 | for color in colors: 279 | d[color] += 1 280 | 281 | print(dict(d)) # prints {'blue': 1, 'green': 2, 'red': 3} 282 | ``` 283 | 284 | **Note:** This is a faster approach than `setdefault()` on most cases and faster than `get()` in all cases. Also, `defaultdict()` seems to work faster on native types like `int` or `string` and slower on `dict` or `list`. That being said, there are times when you cannot use `defaultdict()` and have to use either `setdefault()` or `get()`, for example when the default value of a certain key depends on the key itself, so `defaultdict()` cannot be used from the beginning to have a default value for every new key. 285 | 286 | ```python 287 | colors = ['red', 'green', 'red', 'blue', 'green', 'red'] 288 | 289 | d = {} 290 | for color in colors: 291 | d[color] = d.setdefault(color, 2 if color == 'red' else 0) + 1 292 | 293 | print(d) # prints {'blue': 1, 'green': 2, 'red': 5} 294 | ``` 295 | 296 | **Note 2:** A case where `get()` accomplishes nicely what `setdefault()` and `defaultdict()` would do in a more complicated manner is when you have to return a default value from a dictionary if the key is not in it. 297 | 298 | ```python 299 | d = { 300 | 1: 'Alice', 301 | 2: 'Bob', 302 | 3: 'Carla' 303 | } 304 | 305 | def hello(id): 306 | return f'Hi, {d.get(id, "random person")}' 307 | 308 | print(hello(1)) # prints Hi, Alice 309 | print(hello(4)) # prints Hi, random person 310 | ``` 311 | 312 | ### 14. Creating a list with n elements 313 | 314 | Say you want to create a list with 100 elements of 0. You can just do: 315 | 316 | ```python 317 | lst = [0] * 100 318 | 319 | print(lst) 320 | ``` 321 | 322 | ### 15. Ternary operator in Python 323 | 324 | This: 325 | 326 | ```c 327 | #include 328 | 329 | int main() { 330 | int x; 331 | int y = 1; 332 | 333 | x = (y == 1 ? 1 : 0); 334 | 335 | printf("%d", x); 336 | 337 | return 0; 338 | } 339 | ``` 340 | 341 | can be written like this in python: 342 | 343 | ```python 344 | y = 1 345 | 346 | x = (1 if y == 1 else 0) 347 | 348 | print(x) 349 | ``` 350 | 351 | ### 16. Grouping data with dictionaries 352 | 353 | Say you want to group the items in a list based on some comparison function, for example `len()`: 354 | 355 | ```python 356 | from collections import defaultdict 357 | 358 | names = ['julia', 'mark', 'thomas', 'rachel', 'alex', 'maria'] 359 | 360 | d = defaultdict(list) 361 | for name in names: 362 | key = len(name) 363 | d[key].append(name) 364 | 365 | print(dict(d)) # prints {5: ['julia', 'maria'], 4: ['mark', 'alex'], 6: ['thomas', 'rachel']} 366 | ``` 367 | 368 | All you have to do to group based on some other function is change the `key` to something else. 369 | 370 | ### 17. Unpacking sequences 371 | 372 | ```python 373 | p = 'alex', 'blue', 20, 'c' # same as p = ('alex', 'blue', 20, 'c') 374 | 375 | name, color, age, lang = p 376 | 377 | print(p) # prints a tuple - ('alex', 'blue', 20, 'c') 378 | print(name, color, age, lang) # prints alex blue 20 c 379 | ``` 380 | 381 | **Note:** In the same manner, swapping 2 variables in python might be the most elegant way out of all the languages: 382 | 383 | ```python 384 | x, y = 1, 2 385 | 386 | # swap x and y 387 | x, y = y, x 388 | 389 | print(x, y) # prints 2 1 390 | ``` 391 | 392 | ### 18. Concatenating strings 393 | 394 | Use the `join()` method to concatenate strings from an iterable. 395 | 396 | ```python 397 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 398 | 399 | # join the strings from langs, having ', ' as separator 400 | text = ', '.join(langs) 401 | 402 | print(text) # prints c, python, java, c++, kotlin, rust 403 | ``` 404 | 405 | ### 19. Atomicity of builtin data types 406 | 407 | Most (!not all) of the builtin data types methods are implemented using C function calls, so that makes it atomic. 408 | 409 | For a better explanaton check [here](https://webcache.googleusercontent.com/search?q=cache:9ATPT7NPHg0J:effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm+&cd=4&hl=en&ct=clnk&gl=in). 410 | 411 | Also, dictionaries' `popitem()` is atomic, while `pop()` may not, based on the key type (if the key is not a builtin data type, Python has to call that object's `__hash__()` implementation), so better use `popitem()` where atomicity is needed. 412 | 413 | ```python 414 | d = { 415 | 'foo': 'c', 416 | 'bar': 'java', 417 | 'baz': 'rust' 418 | } 419 | 420 | while d: 421 | key, value = d.popitem() 422 | print(f'{key} --> {value}') 423 | 424 | # prints 425 | # foo --> c 426 | # bar --> java 427 | # baz --> rust 428 | 429 | # d is empty at the end 430 | ``` 431 | 432 | **Note:** If unsure, don't hesitate to use mutexes! 433 | 434 | ### 20. Linking and overriding dictionaries with defaults 435 | 436 | When you have a dictionary that has some default values and you want to override it with another dictionary, use `ChainMap()`. `ChainMap()` has the advantage that it doesn't copy anything, it just "links" the dictionaries, using the initial memory (this also means that any change in the initial dictionary will be reflected in the `ChainMap()` as well). 437 | 438 | ```python 439 | from collections import ChainMap 440 | 441 | defaults = { 442 | 'bar': 'c', 443 | 'foo': 'java' 444 | } 445 | 446 | overwritten = { 447 | 'foo': 'rust', 448 | 'barn': 'c++' 449 | } 450 | 451 | d = ChainMap(overwritten, defaults) 452 | 453 | print(dict(d)) # prints {'foo': 'rust', 'barn': 'c++', 'bar': 'c'} 454 | ``` 455 | 456 | **Note:** Don't use `copy()` and then `update()`, it is really bad performance-wise and can be replaced in 99% of the cases by a `ChainMap()`. 457 | 458 | ```python 459 | d1 = { 460 | 'bar': 'c', 461 | 'foo': 'java' 462 | } 463 | 464 | d2 = { 465 | 'foo': 'rust', 466 | 'barn': 'c++' 467 | } 468 | 469 | # Don't do this!! 470 | d = d1.copy() 471 | d.update(d2) 472 | ``` 473 | 474 | **Note 2:** For a better example when this is useful, see [this](https://docs.python.org/3/library/collections.html#collections.ChainMap). 475 | 476 | ### 21. Ordered dictionary 477 | 478 | A dictionary is not guaranteed to preserve the order of insertion. It actually optimizes keys for faster lookup. However there is one way to have a dictionary preserve insertion order, using `OrderedDict()` from `collections`. 479 | 480 | ```python 481 | from collections import OrderedDict 482 | 483 | d = OrderedDict() 484 | d['bar'] = 'c' 485 | d['foo'] = 'java' 486 | d['baz'] = 'rust' 487 | 488 | print(dict(d)) # prints {'bar': 'c', 'foo': 'java', 'baz': 'rust'} 489 | ``` 490 | 491 | **Note:** Since Python 3.7, regular `dict`s have guaranteed ordering. More [here](https://docs.python.org/3/library/stdtypes.html#dict). Note however that they don't **_completely_** replace `OrderedDict`s, since they have extra features: 492 | 493 | ```python 494 | from collections import OrderedDict 495 | 496 | a = {1: 1, 2: 2} 497 | b = {2: 2, 1: 1} 498 | 499 | c = OrderedDict(a) 500 | d = OrderedDict(b) 501 | 502 | print(a == b) # returns True 503 | print(c == d) # returns False since OrderedDicts are order-sensitive, and regular dicts are not 504 | ``` 505 | 506 | Also, `OrderedDict`s have methods to change order of elements, while regular `dict`s don't. 507 | 508 | ### 22. Using deque instead of a list when updating 509 | 510 | Deques (double ended queues) are really fast in python3. They are implemented using doubly-linked lists, so inserting and removing at the end or at the beginning is O(1) complexity. Lists are implemented as normal arrays, so they have to sometimes `realloc()` to accomodate for the number of elements (only sometimes because by default it `realloc()`s more memory at the time than necessary'), so that makes them have O(n) complexity when inserting or removing at the beginning because they have to copy the rest of the elements. 511 | 512 | Generally, updating a sequence is MUCH faster when using a `deque()` as opposed to using a `list()` (though keep in mind that accessing a random element in a `deque()` is expensive, whereas accessing a random element in a `list()` is O(1)). 513 | 514 | ```python 515 | from collections import deque 516 | 517 | # Wrong! 518 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 519 | 520 | del langs[0] 521 | langs.pop(0) 522 | langs.insert(0, 'scala') 523 | 524 | # Right! 525 | langs = deque(['c', 'python', 'java', 'c++', 'kotlin', 'rust']) 526 | 527 | del langs[0] 528 | langs.popleft(0) 529 | langs.appendleft(0, 'scala') 530 | ``` 531 | 532 | ### 23. Temporary contexts 533 | 534 | Usually there is the case that code like this is written in other languages: 535 | 536 | ```python 537 | from decimal import getcontext, setcontext 538 | 539 | old_context = getcontext().copy() 540 | getcontext().prec = 50 541 | print(Decimal(355) / Decimal(113)) 542 | setcontext(old_context) 543 | ``` 544 | 545 | This can easily be replaced with contexts: 546 | 547 | ```python 548 | from decimal import localcontext, Context 549 | 550 | with localcontext(Context(prec=50)): 551 | print(Decimal(355) / Decimal(113)) 552 | ``` 553 | 554 | Other examples: 555 | 556 | 1. Writing or reading from file 557 | 558 | ```python 559 | f = open('data.txt') 560 | try: 561 | data = f.read() 562 | # do something with data 563 | finally: 564 | f.close() 565 | ``` 566 | 567 | can be replaced with: 568 | 569 | ```python 570 | with open('data.txt') as f: 571 | data = f.read() 572 | # do something with data 573 | ``` 574 | 575 | 2. Deleting a file (getting rid of the try-except-pass idiom): 576 | 577 | ```python 578 | try: 579 | os.remove('sometempfile.tmp') 580 | except OSError: 581 | pass 582 | ``` 583 | 584 | can be replaced with: 585 | 586 | ```python 587 | from contextlib import suppress 588 | 589 | with suppress(FileNotFoundError): 590 | os.remove('sometempfile.tmp') 591 | ``` 592 | 593 | **Note:** `suppress()` is a reentrant context manager. More info [here](https://docs.python.org/3/library/contextlib.html#reentrant-context-managers). 594 | 595 | 3. Using a lock 596 | 597 | ```python 598 | lock = threading.Lock() 599 | 600 | lock.acquire() 601 | try: 602 | # critical section 603 | finally: 604 | lock.release() 605 | ``` 606 | 607 | can be replaced with: 608 | 609 | ```python 610 | lock = threading.Lock() 611 | 612 | with lock: 613 | # critical section 614 | ``` 615 | 616 | **Note:** For reentrant lock context manager, see [threading.RLock](https://docs.python.org/3/library/threading.html#threading.RLock). 617 | 618 | 4. Redirecting output from stdout to file 619 | 620 | ```python 621 | with open('help.txt', 'w') as f: 622 | sldstdout = sys.stdout 623 | sys.stdout = f 624 | try: 625 | help(pow) 626 | finally: 627 | sys.stdout = oldstdout 628 | ``` 629 | 630 | can be replaced with: 631 | 632 | ```python 633 | with open('help.txt', 'w') as f: 634 | with redirect_stdout(f): 635 | help(pow) 636 | ``` 637 | 638 | **Note:** `redirect_stdout()` is also a reentrant context manager. 639 | 640 | More on context managers [here](https://docs.python.org/3/library/contextlib.html). 641 | 642 | ### 24. Using the cache for optimized function calls 643 | 644 | For example, looking up a webpage numerous times is expensive, and usually the result is the same. So use the `lru_cache()` decorator: 645 | 646 | ```python 647 | from functools import lru_cache 648 | 649 | @lru_cache 650 | def web_lookup(url): 651 | return urllib.urlopen(url).read() 652 | ``` 653 | 654 | More can be found [here](https://docs.python.org/3/library/functools.html#functools.lru_cache). 655 | 656 | ### 25. Test a sequence or generator for truthness 657 | 658 | Using the `any()` function, you can check if at least one value in the iterable is `True`. It applies the `bool()` function to every element. 659 | 660 | ```python 661 | false_lst = [0, False, '', 0.0, [], {}, None] # all of these return False when using bool() on them 662 | 663 | print(any(false_lst)) # prints False 664 | 665 | 666 | true_lst = [1, True, 'x', 3.14, ['x'], {'a': 'b'}] # all of these return True when using bool() on them 667 | 668 | print(any(true_lst)) # prints True 669 | 670 | 671 | falst_lst.append(-1) # any integer different from 0 is considered True 672 | 673 | print(any(false_lst)) # prints True 674 | ``` 675 | 676 | **Note:** This function shortcircuits, meaning the first time it finds `True` it returns; it does **NOT** check for the rest of the values to be `True`. 677 | 678 | **Note 2:** It is really useful with generators: 679 | 680 | ```python 681 | print(any(range(1000000)) # prints True after 2 values evaluated, as range() is a generator 682 | 683 | print(any([range(1000000)])) # prints True after the whole list of 1000000 elements has been initialized, as range() has to populate the list first 684 | ``` 685 | 686 | There is another function, `all()`, that does what it says: it tests for all the elements in the sequence to be `True`, and works much in the same way as `any()`. 687 | 688 | ### 26. Use namedtuples instead of tuples 689 | 690 | Aside from the fact that `namedtuple()`s are more verbose, they also offer better usage, as they can be treated as regular tuples, classes or even dictionaries. 691 | 692 | For example, having a point: 693 | 694 | ```python 695 | pt1 = (2, 3) 696 | 697 | print(pt1[0], pt1[1]) # prints 2 3 698 | ``` 699 | 700 | can be replaced with the better alternative `namedtuple()`: 701 | 702 | ```python 703 | from collections import namedtuple 704 | 705 | Point = namedtuple('Point', 'x y') # a tuple named 'Point' with attributes 'x' and 'y' 706 | # alternatively this means the exact same thing 707 | # Point = namedtuple('Point', ['x', 'y']) 708 | 709 | pt1 = Point(2, 3) 710 | print(pt1) # prints Point(x=2, y=3) 711 | print(pt1.x, pt1.y) # prints 2 3 712 | print(pt1[0], pt1[1]) # prints 2 3 713 | print(dict(pt1._asdict())) # prints {'x': 2, 'y': 3} 714 | print(pt1._replace(x=50)) # prints Point(x=50, y=3) 715 | # Note however that _replace() returns a modified copy. The original is still a tuple, so it cannot be modified 716 | ``` 717 | 718 | Another common example: 719 | 720 | ```python 721 | from collections import namedtuple 722 | 723 | Person = namedtuple('Person', 'age color lang') 724 | 725 | person = Person(31, 'blue', 'c') 726 | print(person) # prints Person(age=31, color='blue', lang='c') 727 | ``` 728 | 729 | **Note:** When the values from a `namedtuple()` are invalid (e.g. having one of the fields named `class` or having the same field twice), it throws a `ValueError`. To avoid this you can possibly provide a third parameter named `rename`. If set to `True`, it will rename the field that is incorrect. 730 | 731 | ```python 732 | from collections import namedtuple 733 | 734 | Person = namedtuple('Person', 'age color age', rename=True) 735 | print(Person(31, 'blue', 'whatever')) # prints Person(age=31, color='blue', _2='whatever') 736 | ``` 737 | 738 | **Note 2:** Since Python 3.8, `_asdict()` method returns a regular dictionary, as regular `dict`s now have guaranteed ordering based on insertion (since Python 3.7). 739 | 740 | ### 27. Profiling code with cProfile 741 | 742 | Since Python 3.8, cProfile can be used as a context manager, making it extremely easy to profile code. 743 | 744 | ```python 745 | import cProfile 746 | 747 | with cProfile.Profile() as profiler: 748 | # code to be profiled 749 | 750 | profiler.print_stats() 751 | ``` 752 | 753 | ### 28. Pretty print stuff 754 | 755 | One way to format the output is to use the `pprint` module. 756 | 757 | ```python 758 | from pprint import pprint 759 | 760 | d = { 761 | 'b': [*range(5)], 762 | 'c': [] 763 | 'a': "Here is a long string".split(" "), 764 | } 765 | 766 | pprint(d, indent=2, width=20, compact=True) 767 | 768 | # prints 769 | # { 'a': [ 'Here', 770 | # 'is', 'a', 771 | # 'long', 772 | # 'string'], 773 | # 'b': [ 0, 1, 2, 3, 774 | # 4], 775 | # 'c': []} 776 | ``` 777 | 778 | **Note**: Since Python 3.8, the parameter `sort_dicts` was added (`True` by default): 779 | 780 | ```python 781 | from pprint import pprint 782 | 783 | d = { 784 | 'b': [*range(5)], 785 | 'c': [] 786 | 'a': "Here is a long string".split(" "), 787 | } 788 | 789 | pprint(d, indent=2, width=20, compact=True, sort_dicts=False) 790 | 791 | # prints 792 | # { 'b': [ 0, 1, 2, 3, 793 | # 4], 794 | # 'c': [], 795 | # 'a': [ 'Here', 796 | # 'is', 'a', 797 | # 'long', 798 | # 'string']} 799 | ``` 800 | 801 | More info [here](https://docs.python.org/3/library/pprint.html#module-pprint). 802 | 803 | ### 29. The "is" operator vs "==" operator 804 | 805 | The `is` operator checks if 2 objects point to the same memory address. The equality operator `==` checks if 2 objects are equal. 806 | 807 | ```python 808 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 809 | copy = langs # now copy and langs point to the same memory object 810 | 811 | print(copy == langs) # prints True 812 | print(copy is langs) # prints True 813 | 814 | other_copy = list(langs) # other_copy has a copy of langs, but point to different memory objects 815 | print(other_copy == langs) # prints True 816 | print(other_copy is langs) # prints False 817 | ``` 818 | 819 | ### 30. List slices 820 | 821 | You can use slices to replace elements, delete elements or make a copy of a list. 822 | 823 | 1. Delete items: 824 | 825 | ```python 826 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 827 | 828 | del langs[0:3] 829 | print(langs) # prints ['c++', 'kotlin', 'rust'] 830 | ``` 831 | 832 | 2. Replace elements of a list without creating a new list object 833 | 834 | ```python 835 | langs = ['c', 'python', 'java', 'c++', 'kotlin', 'rust'] 836 | copy = langs 837 | 838 | print(copy is langs) # prints True 839 | 840 | langs[:] = [41, 42, 43] 841 | print(copy is langs) # prints True 842 | print(copy) # prints [41, 42, 43] 843 | 844 | langs = [1, 2, 3] 845 | print(copy is langs) # prints False, langs points to new list (new memory object) 846 | ``` 847 | 848 | 3. Make a (shallow) copy of a list 849 | 850 | ```python 851 | langs = ['c', ['python', 'java'], 'c++', 'kotlin', 'rust'] 852 | copy = langs[:] 853 | 854 | copy[1][0] = 'some other lang' 855 | print(langs) # prints ['c', ['some other lang', 'java'], 'c++', 'kotlin', 'rust'] 856 | ``` 857 | 858 | **Note:** If you need a deep copy consider using the function `deepcopy()` from the module `copy`. 859 | 860 | ### 31. Deep and shallow copies 861 | 862 | There are 2 types of copies in Python. One is the shallow copy, that works very similar to how assigning to pointers works in C (they only reference the object they point to, changing one also changes the other), and the other is the deep copy, which makes a perfect copy of the object. 863 | 864 | ```python 865 | import copy 866 | 867 | list1 = [1, 2, [3, 4], 5] 868 | 869 | list2 = copy.copy(list1) 870 | 871 | list2[2][1] = 6 872 | list2[0] = 7 873 | 874 | # shallow copy, list2 holds references to objects in list1, changing one also changes the other 875 | print(list1) # prints [1, 2, [3, 6], 5] 876 | print(list2) # prints [7, 2, [3, 6], 5] 877 | 878 | list3 = copy.deepcopy(list1) 879 | 880 | list3[2][1] = 8 881 | list3[0] = 9 882 | 883 | # deep copy, list3 is a perfect copy of list1 with no references to it, changing one doesn't change the other 884 | print(list1) # prints [1, 2, [3, 6], 5] 885 | print(list3) # prints [9, 2, [3, 8], 5] 886 | ``` 887 | 888 | More about deep and shallow copies [here](https://www.geeksforgeeks.org/copy-python-deep-copy-shallow-copy/). 889 | 890 | ### 32. Python's built-in http server 891 | 892 | Python has a built-in http server; it can be super useful if you want to preview a website without going the hurdle of starting an apache or nginx server. 893 | 894 | This serves the website in the current directory at address `localhost:8000`: 895 | 896 | ```bash 897 | python3 -m http.server 898 | ``` 899 | 900 | ### 33. Type annotations 901 | 902 | Python 3.5 supports type annotations, which can ensure better readability. **Note however that they are only there for the programmer to acknowledge, Python does not care and won't change anything based on them**. 903 | 904 | ```python 905 | def func(s1: int, s2: int = 42) -> int: 906 | return s1 + s2 907 | ``` 908 | 909 | They can be changed to anything you want: 910 | 911 | ```python 912 | def func2(page: 'web page', request: 'web request') -> 'web response': 913 | # return response 914 | ``` 915 | 916 | **Note:** Passing 2 strings to `func()` is perfectly valid, as Python does **_NOT_** care at all about these annotations (in this case the function would return the 2 strings concatenated). 917 | 918 | **Note 2:** You can use stuff like [Mypy](http://mypy-lang.org) to enforce this kind of behaviour, so Python becomes statically-typed! 919 | 920 | More info about type annotations can be found in [PEP 484](https://www.python.org/dev/peps/pep-0484/). 921 | 922 | **Note 3:** Since Python 3.6, [PEP 526](https://www.python.org/dev/peps/pep-0526/), more support for type annotations was added. Again, **Python will always be a dynamically-typed language**, but tools can be used to ensure static typing. 923 | 924 | ### 34. Counter for iterables 925 | 926 | This is an easy method to find the most common elements in an iterable: 927 | 928 | ```python 929 | import collections 930 | 931 | count = collections.Counter('some random string') 932 | 933 | print(c.most_common()) 934 | # prints [('s', 2), ('o', 2), ('m', 2), (' ', 2), ('r', 2), ('n', 2), ('e', 1), ('a', 1), ('d', 1), ('t', 1), ('i', 1), ('g', 1)] 935 | print(c.most_common(3)) 936 | # prints [('s', 2), ('o', 2), ('m', 2)] 937 | ``` 938 | 939 | More info can be found [in the Python docs for the Counter class](https://docs.python.org/3/library/collections.html#collections.Counter). 940 | 941 | ### 35. Permutations of an iterable 942 | 943 | Get permutations of an iterable: 944 | 945 | ```python 946 | import itertools 947 | 948 | lst = list(itertools.permutations('abc', 2)) 949 | 950 | print(lst) 951 | # prints [('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'c'), ('c', 'a'), ('c', 'b')] 952 | ``` 953 | 954 | The function takes an iterable and another optional argument specifying the length of one permutation. 955 | 956 | ### 36. Getting the representation of an object as a string 957 | 958 | Python has 2 methods to transform an object into a string (similar to other languages `toString()` methods); those are `str()` and `repr()`. 959 | 960 | ```python 961 | import datetime 962 | 963 | now = datetime.date.today() 964 | 965 | print(str(now)) # prints '2020-02-12' 966 | print(repr(now)) # prints 'datetime.date(2020, 2, 12)' 967 | ``` 968 | 969 | The function `str()` is made for clarity, while the function `repr()` is made to be unambiguos about what the object represents. 970 | 971 | The python console uses `repr()`. 972 | 973 | ### 37. Python disassembler 974 | 975 | Python has a built-in disassembler. It is very rudimentary, but it can help debug some code. 976 | 977 | ```python 978 | import dis 979 | 980 | def func(text): 981 | return 'This is some text \'' + str(text) + '\'.' 982 | 983 | dis.dis(func) 984 | # 4 0 LOAD_CONST 1 ("This is some text '") 985 | # 2 LOAD_GLOBAL 0 (str) 986 | # 4 LOAD_FAST 0 (text) 987 | # 6 CALL_FUNCTION 1 988 | # 8 BINARY_ADD 989 | # 10 LOAD_CONST 2 ("'.") 990 | # 12 BINARY_ADD 991 | # 14 RETURN_VALUE 992 | ``` 993 | 994 | More info [in the docs](https://docs.python.org/3/library/dis.html). 995 | 996 | ### 38. Lambda functions 997 | 998 | Lambda functions, as in other functional programming languages, are anonymous functions that don't have a name. They are useful for small code that doesn't require more than a line or two, and they are generally passed as arguments to other functions. 999 | 1000 | One such example that applies to all functional programming languages is the `map()` function. It takes a callable as the first argument (read function, lambda function, something that can be called), and an iterable as the second argument, and applies the function to each of the elements of the iterable, returning a new iterable. 1001 | 1002 | ```python 1003 | obj = map(lambda string: string.lower(), ['StRiNg', 'ANOTHER string']) 1004 | 1005 | print(list(obj)) 1006 | # prints ['string', 'another string'] 1007 | ``` 1008 | 1009 | This code does the exact same thing: 1010 | 1011 | ```python 1012 | def stringlower(string): 1013 | return string.lower() 1014 | 1015 | lst = ['StRiNg', 'ANOTHER string'] 1016 | obj = [] 1017 | 1018 | for item in lst: 1019 | obj.append(stringlower(item)) 1020 | 1021 | print(obj) 1022 | # prints ['string', 'another string'] 1023 | ``` 1024 | 1025 | Another example: 1026 | 1027 | ```python 1028 | power_func = lambda x, y: x ** y 1029 | 1030 | print(power_func(2, 3)) # prints 8 1031 | 1032 | x = (lambda a, b: a - b)(5, 4) 1033 | print(x) # prints 1 1034 | ``` 1035 | 1036 | ### 39. Ip addresses in Python 1037 | 1038 | Python has an interesting module to work with Ip addresses: 1039 | 1040 | ```python 1041 | import ipaddress 1042 | 1043 | address = ipaddress.ip_address('192.168.100.14') 1044 | print(repr(address)) 1045 | # prints IPv4Address('192.168.100.14') 1046 | 1047 | # you can even have arithmetic operations done on this address 1048 | print(address + 3) 1049 | # prints 192.168.100.17 1050 | ``` 1051 | 1052 | More info [here](https://docs.python.org/3/library/ipaddress.html). 1053 | 1054 | ### 40. Subclasses and subinstances 1055 | 1056 | In Python, you can check if a class is a subclass of some other class: 1057 | 1058 | ```python 1059 | class BaseClass(): pass 1060 | class SubClass(BaseClass): pass 1061 | 1062 | print(issubclass(SubClass, BaseClass)) # prints True 1063 | print(issubclass(SubClass, object)) # prints True 1064 | ``` 1065 | 1066 | You can also check if some instance is an instance of the specified class or another sublass of that class: 1067 | 1068 | ```python 1069 | class BaseClass(): pass 1070 | class SubClass(BaseClass): pass 1071 | 1072 | obj = SubClass() 1073 | 1074 | print(isinstance(obj, BaseClass)) # prints True 1075 | ``` 1076 | 1077 | ### 41. Asterisk (\*) and slash (\\) in function definition (positional- and keyword-only function parameters) 1078 | 1079 | In Python 3, you can add an asterisk and a slash to a function definition with special meaning. Asterisk marks keyword-only parameters (that means parameters that can be given to the function just by keyword, not by position), while slash marks positional-only parameters (meaning parameters cannot be given by keyword, but by position only). 1080 | 1081 | ```python 1082 | def func(positional_only_argument, /, positional_and_keyword_argument, *, keyword_only_argument): 1083 | return positional_only_argument + positional_and_keyword_argument + keyword_only_argument 1084 | 1085 | print(func(1, 2, 3)) 1086 | # Type error, third parameter should be keyword 1087 | 1088 | print(func(positional_only_argument = 1, 2, 3)) 1089 | # Type error, first parameter is positional only 1090 | 1091 | print(func(1, 2, keyword_only_argument = 3)) 1092 | # fine, prints 6 1093 | 1094 | print(func(1, positional_and_keyword_argument = 2, keyword_only_argument = 3)) 1095 | # fine, prints 6 1096 | ``` 1097 | 1098 | Info and rationale about these 2 types of parameters can be found in [PEP 3102 - keyword-only parameters](https://www.python.org/dev/peps/pep-3102/) and in [PEP 570 - positional-only parameters](https://www.python.org/dev/peps/pep-0570/). 1099 | 1100 | **Note:** Until Python 3.8, positional-only arguments could only be used in library functions. Starting from Python 3.8, they can be used in programmer constructions too. 1101 | 1102 | ### 42. Python interactive shell 1103 | 1104 | Say you wrote some Python code like this: 1105 | 1106 | ```python 1107 | def min(a, b): 1108 | return a if a < b else b 1109 | ``` 1110 | 1111 | You can launch it in an interactive shell with `python -i main.py`, which is similar to calling only `python` in the command line, with the key difference that the python shell contains your function in the global scope as well. Go ahead, try it! 1112 | 1113 | ### 43. Python debugger 1114 | 1115 | Python has a debugger, similar to gdb. One way to use it is to simply add `import pdb; pdb.set_trace()` in your program wherever u want the debugger to stop program execution. 1116 | 1117 | In Python 3.7, the debugger can also be called on a script like this: `python -m pdb script.py`, and it stops when the module loads, just before executing the first line of the script. 1118 | 1119 | ```python 1120 | def add(a, b): 1121 | return a + b 1122 | 1123 | import pdb 1124 | pdb.set_trace() 1125 | # code execution will stop here, and the program will enter the debugger 1126 | 1127 | print(add(1, 2)) 1128 | ``` 1129 | 1130 | For more information on how to operate the python debugger, visit [this](https://docs.python.org/3/library/pdb.html#debugger-commands). 1131 | 1132 | **Note:** Since Python 3.7, instead of `import pdb; pdb.set_trace()`, you can simply add a `breakpoint()` function call whenever you want the program to stop execution. 1133 | 1134 | ### 44. The walrus operator (:=) 1135 | 1136 | Python 3.8 introduced assignment expressions through the use of a new operator, called the walrus operator (if you look sideways, the operator looks like a walrus). 1137 | 1138 | Assignment expressions allow you to assign and return a value in the same expression, similar to how things work in a language like C. 1139 | 1140 | ```python 1141 | while (x := int(input("What is your age?"))) > 18: 1142 | print("You are a grown-up!") 1143 | else: 1144 | print("You are a kid!") 1145 | ``` 1146 | 1147 | It can be useful, for example in list comprehensions: 1148 | 1149 | ```python 1150 | lst = [y for x in 'abcd' if (y := f(x)) is not None] 1151 | # instead of having to compute f(x) twice 1152 | lst = [f(x) for x in 'abcd' if f(x) is not None] 1153 | ``` 1154 | 1155 | Arguably, the operator is a little confusing, and most of the times not needed and can be replaced with more expressive syntax. There are good arguments to why this operator is not needed in Python [here](https://www.reddit.com/r/Python/comments/8ex72p/pep_572_assignment_expressions/). 1156 | 1157 | Nonetheless, Python 3.8 adopted assignment expressions through the use of the walrus operator :=. 1158 | 1159 | For more info on the walrus operator and assignment expressions, see [PEP 572](https://www.python.org/dev/peps/pep-0572/). 1160 | 1161 | ### 45. Formatted strings (f-strings) 1162 | 1163 | Formatted string literals (or f-strings) are a construct added in Python 3.6 and have since become very popular due to the speed (see tip 7) and simplicity. 1164 | 1165 | Some examples: 1166 | 1167 | ```python 1168 | number = 3.1415 1169 | width = 10 1170 | precision = 3 1171 | print(f'This is {number:{width}.{precision}}') 1172 | # prints 1173 | # This is 3.14 1174 | ``` 1175 | 1176 | There are three conversion fields; `r`, `s` and `a`. What they do is call the functions `repr()`, `str()` and `ascii()` respectively on the formatted parameter. 1177 | 1178 | ```python 1179 | name = 'Alex' 1180 | print(f'My name is {name!r}') 1181 | # prints 1182 | # My name is 'Alex' 1183 | ``` 1184 | 1185 | Since Python 3.8, there is a new specifier (=), that expands to the representation of the expression, making it useful for debugging and self-documenting. 1186 | 1187 | ```python 1188 | import datetime 1189 | 1190 | name = 'Alex' 1191 | print(f'{name=}') # prints name='Alex' 1192 | 1193 | now = datetime.date.today() 1194 | print(f'{now=}') # prints now=datetime.date(2020, 2, 14) 1195 | # f-string specifiers still work 1196 | print(f'{now=!s}') # prints now=2020-02-14 1197 | number = 3.1415 1198 | 1199 | # Careful when adding format specifiers 1200 | print(f'{number + 1=:10.2f}') # prints number + 1= 4.14 1201 | print(f'{number + 1=:10.2}') # prints number + 1= 4.1 1202 | ``` 1203 | 1204 | More info about f-strings [in the docs](https://docs.python.org/3/reference/lexical_analysis.html#f-strings). 1205 | 1206 | **Note:** Formatted strings have a 'formatting' option similar to how `printf()` works in other languages. Python's implementation of formatted print is [a little more advanced though](https://docs.python.org/3/library/string.html#format-specification-mini-language). 1207 | 1208 | ### 46. Decorators and the functools.wraps helper function 1209 | 1210 | Say you have a function: 1211 | 1212 | ```python 1213 | def sum(a, b): 1214 | """This function adds 2 numbers and returns the result.""" 1215 | return a + b 1216 | ``` 1217 | 1218 | But now we want to log this function call. Of course, adding this code in the implementation of the function is bad, since we're polluting the function code. Even more so, what if we want to log another 10 function calls? 1219 | 1220 | For this purpose, we can easily use a decorator. 1221 | 1222 | ```python 1223 | def log(func): 1224 | 1225 | def wrapper(*args, **kwargs): 1226 | """Wrapper function.""" 1227 | # do some logging 1228 | return func(*args, **kwargs) 1229 | 1230 | return wrapper 1231 | ``` 1232 | 1233 | Now it is easy to use the decorator on whatever function we want to log. 1234 | 1235 | ```python 1236 | @log 1237 | def sum(a, b): 1238 | """This function adds 2 numbers and returns the result.""" 1239 | return a + b 1240 | 1241 | print(sum(4, 5)) # this function call will be logged 1242 | ``` 1243 | 1244 | However, one problem arises when decorating a function like this. If we now try to get the doc or the function name, we notice that we get the information of the wrapper function, rather than that of our initial function: 1245 | 1246 | ```python 1247 | print(sum.__doc__) # prints "Wrapper function" 1248 | print(sum.__name__) # prints "wrapper" 1249 | ``` 1250 | 1251 | This is not ideal, considering that debuggers and other introspection tools use this. To fix this, we can use functools.wraps. 1252 | 1253 | ```python 1254 | import functools 1255 | 1256 | def log(func): 1257 | 1258 | @functools.wraps(func) 1259 | def wrapper(*args, **kwargs): 1260 | """Wrapper function.""" 1261 | # do some logging 1262 | return func(*args, **kwargs) 1263 | 1264 | return wrapper 1265 | 1266 | @log 1267 | def sum(a, b): 1268 | """This function adds 2 numbers and returns the result.""" 1269 | return a + b 1270 | 1271 | print(sum.__doc__) # prints "This function adds 2 numbers and returns the result." 1272 | print(sum.__name__) # prints "sum" 1273 | ``` 1274 | 1275 | ### 47. Static function variables 1276 | 1277 | Python does not have a built-in method to have a static variable in a function like C or other languages do through the use of the `static` keyword. 1278 | 1279 | Instead, we can use the fact that functions are first-class objects in Python and we can assign variables to them. 1280 | 1281 | ```python 1282 | def func(): 1283 | try: 1284 | func.number_of_times_called += 1 1285 | except: 1286 | func.number_of_times_called = 1 1287 | # some really interesting code 1288 | ``` 1289 | 1290 | This is better than having a global variable pollute the global namespace, and is better than having a decorator that does that (because the decorator runs when the python module is loaded even if the function might never be called, so the decorator will still do some work and initialize some value; instead here the code runs only when the function is called, if ever). 1291 | --------------------------------------------------------------------------------