├── .gitignore ├── A_Array_of_Sequences_part_2_1 └── README.md ├── A_Array_of_Sequences_part_2_2 └── README.md ├── A_Array_of_Sequences_part_2_3 ├── README.md ├── eg_2_17.py └── eg_2_19.py ├── A_Pythonic_Card_Deck ├── README.md └── eg_1_1.py ├── Dictionaries_and_Sets_part_3_1 ├── README.md └── eg_3_6.py ├── LICENSE ├── README.md ├── what_is_the_Descriptor └── README.md ├── what_is_the_Monkey_Patch ├── README.md ├── demo1.py ├── demo2.py ├── demo3.py ├── demo4.py └── demo5.py ├── what_is_the_closures ├── README.md ├── demo1.py ├── demo2.py └── demo3.py ├── what_is_the_hashable └── README.md └── what_is_the_python_GIL ├── README.md ├── process_demo.py └── thread_demo.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | .idea/ 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Distribution / packaging 11 | .Python 12 | env/ 13 | build/ 14 | develop-eggs/ 15 | dist/ 16 | downloads/ 17 | eggs/ 18 | .eggs/ 19 | lib/ 20 | lib64/ 21 | parts/ 22 | sdist/ 23 | var/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *,cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # IPython Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # dotenv 80 | .env 81 | 82 | # virtualenv 83 | venv/ 84 | ENV/ 85 | 86 | # Spyder project settings 87 | .spyderproject 88 | 89 | # Rope project settings 90 | .ropeproject 91 | -------------------------------------------------------------------------------- /A_Array_of_Sequences_part_2_1/README.md: -------------------------------------------------------------------------------- 1 | # An Array of Sequences - Part 2-1 2 | 3 | [Youtube Tutorial - An Array of Sequences - Part 2-1](https://youtu.be/FD4kmQt0-RY) 4 | 5 | 基本上，sequences 分為兩種， 6 | 7 | ***container sequences*** 8 | 9 | ```text 10 | list, tuple, and collections.deque can hold items of different types. 11 | ``` 12 | 13 | ***flat sequences*** 14 | 15 | ```text 16 | str, bytes, bytearray, memoryview, and array.array hold items of one type. 17 | ``` 18 | 19 | 另外一種區分的方法是藉由 mutability 20 | 21 | ***Mutable sequences*** 22 | 23 | ```text 24 | list, sets, dictionary, bytearray, array.array, collections.deque, and memoryview. 25 | ``` 26 | 27 | ***Immutable sequences*** 28 | 29 | ```text 30 | tuple, str, int(float), and bytes. 31 | ``` 32 | 33 | ***List Comprehensions ( listcomps ) and Generator Expressions ( genexps )*** 34 | 35 | 先來講 **List Comprehensions** 36 | 37 | 還記得在 [之前](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Pythonic_Card_Deck/README.md) 提到的這段 code 嗎 :question: 38 | 39 | ```python 40 | >>> [str(n) for n in range(2, 11)] 41 | ['2', '3', '4', '5', '6', '7', '8', '9', '10'] 42 | ``` 43 | 44 | 上面這就是很典型的 listcomps，這段程式碼，相等於 45 | 46 | ```python 47 | >>> seqs=[] 48 | >>> for n in range(2,11): 49 | ... seqs.append(str(n)) 50 | >>> seqs 51 | ['2', '3', '4', '5', '6', '7', '8', '9', '10'] 52 | ``` 53 | 54 | 應該非常的明顯，前者可讀性好很多，而且 code 也少了很多行， 接著再看一個例子， 55 | 56 | ```python 57 | >>> colors = ['black', 'white'] 58 | >>> sizes = ['S', 'M', 'L'] 59 | >>> tshirts = [(color, size) for color in colors for size in sizes] # 第一種寫法 60 | >>> tshirts = [(color, size) for color in colors # 第二種寫法 ( 也可以換行 ) 61 | ... for size in sizes] 62 | >>> tshirts 63 | [('black', 'S'), ('black', 'M'), ('black', 'L'), ('white', 'S'), ('white', 'M'), ('white', 'L')] 64 | >>> 65 | >>> ## 上方這段程式碼，相等於下方 66 | >>> 67 | >>> tshirts = [] 68 | >>> for color in colors: 69 | ... for size in sizes: 70 | ... tshirts.append((color, size)) 71 | ... 72 | >>> tshirts 73 | [('black', 'S'), ('black', 'M'), ('black', 'L'), ('white', 'S'), ('white', 'M'), ('white', 'L')] 74 | ``` 75 | 76 | 相信大家看完就會了解 [之前](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Pythonic_Card_Deck/README.md) 此部分的 code， 77 | 78 | ```python 79 | self._cards = [Card(rank, suit) for suit in self.suits 80 | for rank in self.ranks] 81 | ``` 82 | 83 | 接著我們來看 **Generator Expressions ( genexps )** 84 | 85 | ```text 86 | To initialize tuples, arrays, and other types of sequences, you could also start from a 87 | listcomp,but a genexp saves memory because it yields items one by one using the iterator 88 | protocol instead of building a whole list just to feed another constructor. 89 | ``` 90 | 91 | 使用 genexp 可以節省記憶體。 92 | 93 | ```python 94 | >>> genexps = (str(n) for n in range(2, 11)) # <1> 95 | >>> genexps # <2> 96 | at 0x00000213C7F2C468> 97 | >>> tuple(genexps) 98 | ('2', '3', '4', '5', '6', '7', '8', '9', '10') 99 | ``` 100 | 101 | **1** 的部份和 listcomps 寫法唯一不一樣的地方就是， 102 | 103 | listcomps 是使用 `[` `]` 這個符號， 而 genexps 則是使用 `(` `)` 這個符號。 104 | 105 | **2** 的部分是要讓大家了解，它是一個 generator object 。 106 | 107 | 剛剛說 genexp 可以節省記憶體，讓我們透過數據說話，如下， 108 | 109 | ```python 110 | >>> from sys import getsizeof # Return the size of an object in bytes. 111 | >>> list_comp = [x for x in range(1000000)] 112 | >>> gen_exp = (x for x in range(1000000)) 113 | >>> getsizeof(list_comp) 114 | 8697464 115 | >>> getsizeof(gen_exp) 116 | 88 117 | ``` 118 | 119 | 接下來我們來談談 `tuples` 120 | 121 | ```text 122 | Tuples do duuble duty : they can be used as immutable lists and also as record with no field names. 123 | ``` 124 | 125 | 先來看 immutable lists 126 | 127 | ```python 128 | >>> tuples = (1,3) 129 | >>> tuples[0] 130 | 1 131 | >>> tuples[0] = 2 132 | Traceback (most recent call last): 133 | File "", line 1, in 134 | TypeError: 'tuple' object does not support item assignment 135 | ``` 136 | 137 | 再來看 Tuples as record with no field names. 138 | 139 | ( 咦? 是不是有似曾相識的感覺，沒錯，我們在 [這裡](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Pythonic_Card_Deck/README.md) 有提到 ) 140 | 141 | namedtuple 就如同一筆 record，可參考 [namedtuple_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/namedtuple_tutorial.py) 142 | 143 | ```python 144 | >>> name_ids = [('name1',1),('name2',2),('name3',3)] # Tuples as record with no field names. 145 | >>> for name, _ in name_ids: # <1> Unpacking 146 | ... print(name) 147 | ... 148 | name1 149 | name2 150 | name3 151 | ``` 152 | 153 | **1** 的這部分我只希望顯示 name ( 不管 id )，所以我將 id 的部份使用了一個 `_` ( dummy variable )， 154 | 155 | 關於 `_` 額外補充一下，如果你有使用到多國語系，就會使用到 [gettext module](https://docs.python.org/3/library/gettext.html) ， 156 | 157 | 這時候 `_` 可能就不是一個很適合的 dummy variable。 158 | 159 | Unpacking 是什麼 ? 160 | 161 | ```python 162 | >>> name_pk = ('name1',1) 163 | >>> name , pk = name_pk # tuple unpacking 164 | >>> name 165 | 'name1' 166 | >>> pk 167 | 1 168 | ``` 169 | 170 | tuple unpacking 這個特性，讓我們想要交換兩個變數時，可以直接這樣寫 171 | 172 | ( 大家如果有摸過 C 語言就知道，要用一個 temp 的變數 ) 173 | 174 | ```python 175 | b, a = a, b # without using a temporary variable 176 | ``` 177 | 178 | 超簡單快速 179 | 180 | ***using * to grab excess items*** 181 | 182 | 這個是 python3 才有的功能，可參考 [new-syntax](https://docs.python.org/3.0/whatsnew/3.0.html#new-syntax)， 183 | 184 | 官方說明為 Extended Iterable Unpacking，直接看範例，會更快了解 185 | 186 | ```python 187 | >>> a, b, *rest = range(5) 188 | >>> a, b, rest 189 | (0, 1, [2, 3, 4]) 190 | >>> a, *body, c, d = range(5) 191 | >>> a, body, c, d 192 | (0, [1, 2], 3, 4) 193 | >>> *head, b, c, d = range(5) 194 | >>> head, b, c, d 195 | ([0, 1], 2, 3, 4) 196 | ``` 197 | 198 | ***Slice*** 199 | 200 | `seq[start,stop,step]` 201 | 202 | ```python 203 | >>> s = 'abcdefg' 204 | >>> s[::2] 205 | 'aceg' 206 | >>> s[::-1] 207 | 'gfedcba' 208 | >>> s[2::2] 209 | 'ceg' 210 | ``` 211 | 212 | 這其實在 [這裡](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Pythonic_Card_Deck/README.md) 也有提到，也就是 `deck[12::13]`。 213 | 214 | 當執行 `seq[start,stop,step]` 時，其實 python 是去 call `seq.__getitem__( slice(start, stop, step ))` 215 | 216 | ```python 217 | >>> slice 218 | 219 | >>> dir(slice) 220 | [..., 'indices', 'start', 'step', 'stop'] 221 | ``` 222 | 223 | indices 它讓 slice 變的強大，它可以幫我們 normalized，保持你適合的長度，看下面的例子 224 | 225 | ```python 226 | >>> s = 'abcdefg' 227 | >>> len(s) 228 | 7 229 | >>> s[0:7:2] 230 | 'aceg' 231 | >>> s[:100:2] 232 | 'aceg' 233 | >>> s[2:7:1] 234 | 'cdefg' 235 | >>> s[-5:] 236 | 'cdefg' 237 | ``` 238 | 239 | 就是因為 indices 的原因，讓 `s[0:7:2]` 和 `s[:100:2]` 以及 `s[2:7:1]` 和 `s[-5:]` 結果相同。 240 | 241 | Assigning to Slice 242 | 243 | ```python 244 | >>> seqs = list(range(10)) 245 | >>> seqs 246 | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] 247 | >>> seqs[2:5] = [20, 30] 248 | >>> seqs 249 | [0, 1, 20, 30, 5, 6, 7, 8, 9] 250 | >>> del seqs[5:7] 251 | >>> seqs 252 | [0, 1, 20, 30, 5, 8, 9] 253 | >>> seqs[2:5] = 100 254 | Traceback (most recent call last): 255 | File "", line 1, in 256 | TypeError: can only assign an iterable 257 | >>> seqs[2:5] = [100] # must be an iterable object 258 | >>> seqs 259 | [0, 1, 100, 8, 9] 260 | ``` 261 | 262 | ## 執行環境 263 | 264 | * Python 3.6.4 265 | 266 | ## Reference 267 | 268 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 269 | 270 | ## Donation 271 | 272 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 273 | 274 | ![alt tag](https://i.imgur.com/LRct9xa.png) 275 | 276 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 277 | -------------------------------------------------------------------------------- /A_Array_of_Sequences_part_2_2/README.md: -------------------------------------------------------------------------------- 1 | # An Array of Sequences - Part 2-2 2 | 3 | [Youtube Tutorial - An Array of Sequences - Part 2-2](https://youtu.be/g1_XjMMB60s) 4 | 5 | ***A += Assignment Puzzler*** 6 | 7 | ```python 8 | >>> t = (1, 2, [30, 40]) 9 | >>> t[2] += [50, 60] 10 | ``` 11 | 12 | What happens next :question: Choose the best answer: 13 | 14 | ```text 15 | a. t becomes (1, 2, [30, 40, 50, 60]) 16 | b. TypeError is raised with the message 'tuple' object does not support item assignment. 17 | c. Neither 18 | d. Both a and b. 19 | ``` 20 | 21 | 選 **a** 的人很明顯是對 tuple 不夠了解 ( 請前往 [上一篇](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_1) 文章複習 )，tuple 是 immutable sequences， 22 | 23 | 但你可能會反駁我說，可是它是在 list 裡面耶:triumph: 關於這個問題，我等等和大家解答。 24 | 25 | 選 **b** 的人，代表你了解 tuple，恭喜你，但還是錯的:smirk: 26 | 27 | 讓我們用 Python console 執行看看 28 | 29 | ```python 30 | >>> t = (1, 2, [30, 40]) 31 | >>> t[2] += [50, 60] 32 | Traceback (most recent call last): 33 | File "", line 1, in 34 | TypeError: 'tuple' object does not support item assignment 35 | >>> t 36 | (1, 2, [30, 40, 50, 60]) 37 | ``` 38 | 39 | 所以答案是 **d** ( 選 **a** 或 **b** 的都各對一半 :laughing: ) 40 | 41 | 可以透過 [Python Tutor](http://www.pythontutor.com/) 這個網站幫我們視覺化 Python 執行的細節。 42 | 43 | 如果你不懂如何操作，請點選我文章最前面的 [教學影片](https://youtu.be/g1_XjMMB60s)， 44 | 45 | 我們還可以透過 Python btyecode，可參考官方文件 [dis module](https://docs.python.org/3/library/dis.html) 46 | 47 | ```python 48 | >>> dis.dis('s[a] += b') 49 | 1 0 LOAD_NAME 0 (s) # <1> 50 | 2 LOAD_NAME 1 (a) # <2> 51 | 4 DUP_TOP_TWO # <3> 52 | 6 BINARY_SUBSCR # <4> 53 | 8 LOAD_NAME 2 (b) # <5> 54 | 10 INPLACE_ADD # <6> 55 | 12 ROT_THREE # <7> 56 | 14 STORE_SUBSCR # <8> 57 | 16 LOAD_CONST 0 (None) 58 | 18 RETURN_VALUE # <9> 59 | ``` 60 | 61 | 上面的左上角的 1 代表是行數，先來看一下幾個定義， 62 | 63 | ```text 64 | the top of the stack - ( TOS ) 65 | the second top-most stack item ( TOS1) 66 | the third top-most stack item (TOS2 ) 67 | ``` 68 | 69 | 大家可以對照我做的這張圖，我理解 [dis module](https://docs.python.org/3/library/dis.html) 的是這樣， 70 | 71 | 如果有錯，在請糾正:smile: 72 | 73 | ![tag](https://i.imgur.com/XwNBbgl.png) 74 | 75 | 可搭配下面的說明 76 | 77 | **1** 的部份，Pushes the value associated with co_names[namei] onto the stack. 78 | 79 | **3** 的部份，Duplicates the two references on top of the stack, leaving them in the same order. 80 | 81 | **4** 的部份，Implements TOS = TOS1[TOS]. ( TOS = s[a] ) 82 | 83 | **6** 的部份，Implements TOS = TOS1 + TOS. ( TOS = s[a] + b ) 84 | 85 | **7** 的部份，Lifts second and third stack item one position up, moves top down to position three. 86 | 87 | **8** 的部份，Implements TOS1[TOS] = TOS2. ( s[a] = s[a] + b)，我們在這邊的時候 **ERROR** 了，原因是因為它是 immutable。 88 | 89 | **9** 的部份，Returns with TOS to the caller of the function. 90 | 91 | 通過這個例子，可以學習到幾個重點， 92 | 93 | 1. Putting mutable items in tuples is not a good idea. 94 | 2. Augmented assignment is not an atomic operation - we just saw it throwing an exception after doing part of its job. 95 | 3. Inspecting Python bytecode is not too difficult, and is often helpful to see what is going on under the hood. 96 | 97 | 簡單翻譯 98 | 99 | 1. 將 mutable items 放在 tuples 中不是很理想 (會發生神奇的問題 )。 100 | 2. Augmented assignment 不是如同一個原子 ( 不可切割 ) 的操作，它只會執行到有問題時，然後丟出一個 exception。 101 | 3. 學習觀看 Python bytecode ，雖然我也花了一點時間研究:laughing: 102 | 103 | 最後補充一下，其實只要將 `t[2] += [50, 60]` 改成 `t[2].extend([50, 60])` 就可以正常 work:thumbsup: 104 | 105 | `t[2] += [50, 60]` 會有問題主要是因為它會企圖將值指回去 tuple ( 可參考 `__iadd__`) ，所以導致錯誤， 106 | 107 | 而 `t[2].extend([50, 60])` 之所以不會有問題，則是因為在 python 中主要是參考 ( reference ) 的概念 ( 以後會講 )， 108 | 109 | 而他也不知道它自己在 tuple 中，也不會像 `__iadd__` 會企圖將值指回去 tuple，所以正確 work。 110 | 111 | 如果你想更進一步的了解，可參考官方的說明 [Why does a_tuple[i] += [‘item’] raise an exception when the addition works](https://docs.python.org/3/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works)。 112 | 113 | ## 執行環境 114 | 115 | * Python 3.6.4 116 | 117 | ## Reference 118 | 119 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 120 | 121 | ## Donation 122 | 123 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 124 | 125 | ![alt tag](https://i.imgur.com/LRct9xa.png) 126 | 127 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 128 | -------------------------------------------------------------------------------- /A_Array_of_Sequences_part_2_3/README.md: -------------------------------------------------------------------------------- 1 | # An Array of Sequences - Part 2-3 2 | 3 | 準備中 4 | 5 | [Youtube Tutorial - An Array of Sequences - Part 2-3](xxx) 6 | 7 | ***list.sort and the sorted Built-In Function*** 8 | 9 | **list.sort** , without making a copy. 10 | 11 | ```text 12 | it returns None to remind us that it changes the target object, and does not create a new list. 13 | This is an important python api convention : functions or methods that change an object an object in place should return None to make it clear to the caller that the object itself was changed, and no new object was created. 14 | ``` 15 | 16 | list.sort 的範例可參考 [sort.py](https://github.com/twtrubiks/python-notes/blob/master/sort.py)。 17 | 18 | **sorted** , the built-in function sorted creates a new list and returns it ( always returns a newly created list ). 19 | 20 | sorted 的範例可參考 [sorted.py](https://github.com/twtrubiks/python-notes/blob/master/sorted.py)。 21 | 22 | Once your sequences are sorted, they can be very efficiently searched. 23 | 24 | 可以使用 standard binary search algorithm ( [bisect module](https://docs.python.org/3.6/library/bisect.html) ) 。 25 | 26 | ***Managing Ordered Sequences with Bisect*** 27 | 28 | ```text 29 | The bisect module offers two main functions - bisect and insort - that use the binary search algorithm to quickly find and insert items in any sorted sequence. 30 | ``` 31 | 32 | 使用 bisect 時，一定要是 sorted sequence，更多說明可參考 [bisect.html](https://docs.python.org/3.6/library/bisect.html)， 33 | 34 | ```text 35 | bisect is actually an alias for bisect_right. 36 | bisect_right returns an insertion point after the existing item. 37 | bisect_left returns the position of the existing item. 38 | ``` 39 | 40 | 看下面的例子， 41 | 42 | ```python 43 | >>> import bisect 44 | >>> Y = [0, 2, 2, 3, 6, 8, 12, 20] 45 | >>> x_insert_point = bisect.bisect_left(Y, 2) 46 | >>> x_insert_point 47 | 1 48 | >>> x_insert_point = bisect.bisect_right(Y, 2) 49 | >>> x_insert_point 50 | 3 51 | >>> x_insert_point = bisect.bisect(Y, 2) # bisect is actually an alias for bisect_right 52 | >>> x_insert_point 53 | 3 54 | ``` 55 | 56 | 教大家一個比較簡單的方法， bisect_left 從左邊看，bisect_right 則從右邊看。 57 | 58 | 再來看 `bisect.insort_right` 以及 `bisect.insort_left` ，其實它就是包含上面 ( `bisect.bisect` ) 的特性再加上 insert。 59 | 60 | ```python 61 | >>> import bisect 62 | >>> Y = [0, 2, 2, 3, 6, 8, 12, 20] 63 | >>> x_insort_left = bisect.insort_left(Y, 2) 64 | >>> Y 65 | [0, 2, 2, 2, 3, 6, 8, 12, 20] 66 | >>> Y = [0, 2, 2, 3, 6, 8, 12, 20] 67 | >>> x_insort_right = bisect.insort_right(Y, 1) # bisect.insort is actually an alias for bisect.insort_right 68 | >>> Y 69 | [0, 1, 2, 2, 3, 6, 8, 12, 20] 70 | ``` 71 | 72 | fluent python 中 bisect 的範例 [eg_2_17.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Array_of_Sequences_part_3/eg_2_17.py) 73 | 74 | ```python 75 | python eg_2_17.py 76 | ``` 77 | 78 | 或是 79 | 80 | ```python 81 | python eg_2_17.py left 82 | ``` 83 | 84 | 在 [官方文件](https://docs.python.org/3/library/bisect.html#other-examples) 中的範例， 85 | 86 | Given a test score, grade returns the corresponding letter grade , 87 | 88 | ```python 89 | >>> import bisect 90 | >>> def grade(score, breakpoints=[60, 70, 80, 90], grades='FDCBA'): 91 | ... i = bisect.bisect(breakpoints, score) 92 | ... return grades[i] 93 | ... 94 | >>> [grade(score) for score in [33, 99, 77, 70, 89, 90, 100]] 95 | ['F', 'A', 'C', 'C', 'B', 'A', 'A'] 96 | ``` 97 | 98 | fluent python 中 Insort 的範例 [eg_2_19.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Array_of_Sequences_part_3/eg_2_19.py) ( Insort keeps a sorted sequence always sorted. ) 99 | 100 | ***Arrays*** 101 | 102 | [array](https://docs.python.org/3/library/array.html) 103 | 104 | ```text 105 | If the list will only contain numbers, an array.array is more efficient than a list: it support all mutable sequence operations ( including .pop, .insert, and .extend ) , and additional methods for fast loading and saving such as .frombytes and .tofile. 106 | ``` 107 | 108 | python will not you put any number that does not match the type for the array. 109 | 110 | 看下面的例子， 111 | 112 | ```python 113 | >>> from array import array 114 | >>> array('b', ((999,2,3))) # 'b' is signed char 115 | Traceback (most recent call last): 116 | File "", line 1, in 117 | OverflowError: signed char is greater than maximum 118 | ``` 119 | 120 | 'b' 為 typecode ，上面的 array 就只能放 signed char ( -128 to 127 )，放其他的 type 就會 error。 121 | 122 | 其他的 typecode 說明，可參考 [array](https://docs.python.org/3/library/array.html)。 123 | 124 | fluent python 中的例子， Creating , saving and loading a large array of 10 floats 125 | 126 | ```python 127 | >>> from array import array 128 | >>> from random import random 129 | >>> floats = array('d', (random() for i in range(10 ** 7))) # 'd' is double float 130 | >>> floats[-1] 131 | 0.3937325241174071 132 | >>> fp = open('floats.bin', 'wb') 133 | >>> floats.tofile(fp) 134 | >>> fp.close() 135 | >>> floats2 = array('d') 136 | >>> fp = open('floats.bin', 'rb') 137 | >>> floats2.fromfile(fp, 10 ** 7) 138 | >>> fp.close() 139 | >>> floats2[-1] 140 | 0.3937325241174071 141 | ``` 142 | 143 | 從上面的例子你可以發現，`array.tofile` 和 `floats2.fromfile` 很容易使用，速度也快， 144 | 145 | 如果將上面改存成 text file，你會發現存成 binary file 節省非常多的空間。 146 | 147 | 還有一個也可以 pickle module 也可以參考。 148 | 149 | Another fast and more flexible way of saving numeric data is the [pickle module](https://docs.python.org/3/library/pickle.html) for object serialization. 150 | it handles almost all built-in types. 151 | 152 | 範例可參考 [pickle_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/pickle_tutorial.py) 153 | 154 | ***Memory View*** 155 | 156 | [memoryview](https://docs.python.org/3/library/stdtypes.html#memoryview) 157 | 158 | ```text 159 | The built-in memoryview class is a shared-memory sequence type that lets you handle slices of arrays without coping bytes. It was inspired by the NumPy library. 160 | ``` 161 | 162 | 如果你想進一步的了解，可參考 [When should a memoryview be used](https://stackoverflow.com/questions/4845418/when-should-a-memoryview-be-used/) 163 | 164 | ```text 165 | A memoryview is essentially a generalized NumPy array structure in Python itself (without the math). It allows you to share memory between data-structures (things like PIL images, SQLlite data-bases, NumPy arrays, etc.) without first copying. This is very important for large data sets. 166 | 167 | With it you can do things like memory-map to a very large file, slice a piece of that file and do calculations on that piece (easiest if you are using NumPy). 168 | ``` 169 | 170 | 先來看個例子， 171 | 172 | not use memoryview 173 | 174 | ```python 175 | >>> # not use memoryview 176 | >>> a1 = bytearray('abcde'.encode()) 177 | >>> b1 = a1[:2] # generate new str , does not affect a1 178 | >>> a1 179 | bytearray(b'abcde') 180 | >>> b1 181 | bytearray(b'ab') 182 | ``` 183 | 184 | 為什麼這這邊我不直接使用 str 就好，而要使用 bytearray :question: 185 | 186 | 原因是在 [A Array of Sequences - Part 2-1](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_1) 有提到 ， 187 | 188 | str is immutable sequences , bytearray is mutable sequences. 189 | 190 | 如果改成 str 就會出現 error ( 如下範例 ) 191 | 192 | ```python 193 | >>> a1 ='abcde'.encode() 194 | >>> ma1 = memoryview(a1) 195 | >>> mb2 = ma1[:2] 196 | >>> mb2[:2] = 'bb'.encode() 197 | Traceback (most recent call last): 198 | File "", line 1, in 199 | TypeError: cannot modify read-only memory 200 | ``` 201 | 202 | use memoryview 203 | 204 | ```python 205 | >>> # use memoryview 206 | >>> a2 = bytearray('abcde'.encode()) 207 | >>> ma2 = memoryview(a2) 208 | >>> mb2 = ma2[:2] # not generate new str 209 | >>> mb2[:2] = 'bb'.encode() 210 | >>> mb2.tobytes() # affect ma2 211 | b'bb' 212 | >>> ma2.tobytes() 213 | b'bbcde' 214 | ``` 215 | 216 | ***NumPy and SciPy*** 217 | 218 | For advanced array and matrix operations. 219 | 220 | 如果你有很大量的陣列以及要進去矩陣的操作，建議使用 NumPy 會更好，這邊就不在另外做介紹， 221 | 222 | 因為這個非常深的東西:relaxed: 223 | 224 | ***Deques*** 225 | 226 | ```text 227 | The .append and .pop methods make a list usable as a stack or a queue ( if you use .append and .pop(0), you get LIFO behavior ).But inserting and removing from the left of a list ( the 0-index end ) is costly because the entire list must be shifted. 228 | ``` 229 | 230 | 雖然 list 很好用，可以透過 .append and .pop(0) 的方法完成 LIFO ( Last in First out - 後進先出 )，但不管是 231 | 232 | 從 list 中的最左邊 ( 0-index ) inserting 還是 removing 付出的成本都很高，所以，如果你有 LIFO 的需求，使用 233 | 234 | deques 會更好:smile: 235 | 236 | ```text 237 | The class collections.deque is a thread-safe (synchronized) double-ended queue designed for fast inserting and removing from both ends. 238 | Deque is safe to use as a LIFO queue in multithreaded applications without the need for using locks. 239 | ``` 240 | 241 | fluent python 中的例子 242 | 243 | ```python 244 | >>> from collections import deque 245 | >>> dq = deque(range(10), maxlen=10) 246 | >>> dq 247 | deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], maxlen=10) 248 | >>> # rotating with n > 0 takes items from the right end and prepends then to the left ; 249 | >>> # when n < 0 items are taken from left and appended to the right 250 | >>> dq.rotate(3) 251 | >>> dq 252 | deque([7, 8, 9, 0, 1, 2, 3, 4, 5, 6], maxlen=10) 253 | >>> dq.rotate(-4) 254 | >>> dq 255 | deque([1, 2, 3, 4, 5, 6, 7, 8, 9, 0], maxlen=10) 256 | >>> # when it is full , it discards items from the opposite end when you append new ones 257 | >>> dq.appendleft(-1) 258 | >>> dq 259 | deque([-1, 1, 2, 3, 4, 5, 6, 7, 8, 9], maxlen=10) 260 | >>> # deque.extend( iterable ) 261 | >>> dq.extend([11, 22, 33]) 262 | >>> dq 263 | deque([3, 4, 5, 6, 7, 8, 9, 11, 22, 33], maxlen=10) 264 | >>> dq.extendleft([10, 20, 30, 40]) 265 | >>> dq 266 | deque([40, 30, 20, 10, 3, 4, 5, 6, 7, 8], maxlen=10) 267 | ``` 268 | 269 | ## 執行環境 270 | 271 | * Python 3.6.4 272 | 273 | ## Reference 274 | 275 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 276 | 277 | ## Donation 278 | 279 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 280 | 281 | ![alt tag](https://i.imgur.com/LRct9xa.png) 282 | 283 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 284 | -------------------------------------------------------------------------------- /A_Array_of_Sequences_part_2_3/eg_2_17.py: -------------------------------------------------------------------------------- 1 | import bisect 2 | import sys 3 | 4 | HAYSTACK = [1, 4, 5, 6, 8, 12, 15, 20, 21, 23, 23, 26, 29, 30] 5 | NEEDLES = [0, 1, 2, 5, 8, 10, 22, 23, 29, 30, 31] 6 | 7 | ROW_FMT = '{0:2d} @ {1:2d} {2}{0:<2d}' 8 | 9 | 10 | def demo(bisect_fn): 11 | for needle in reversed(NEEDLES): 12 | position = bisect_fn(HAYSTACK, needle) 13 | offset = position * ' |' 14 | print(ROW_FMT.format(needle, position, offset)) 15 | 16 | 17 | if __name__ == '__main__': 18 | 19 | if sys.argv[-1] == 'left': 20 | bisect_fn = bisect.bisect_left 21 | else: 22 | bisect_fn = bisect.bisect # bisect is actually an alias for bisect_right 23 | 24 | print('bisect_fn.__name__:', bisect_fn.__name__) 25 | print('haystack ->', ' '.join('%2d' % n for n in HAYSTACK)) 26 | demo(bisect_fn) 27 | -------------------------------------------------------------------------------- /A_Array_of_Sequences_part_2_3/eg_2_19.py: -------------------------------------------------------------------------------- 1 | # Insort keeps a sorted sequence always sorted 2 | import bisect 3 | import random 4 | 5 | SIZE = 7 6 | random.seed(1729) # Initialize the random number generator. 7 | 8 | my_list = [] 9 | for i in range(SIZE): 10 | 11 | # This is equivalent to choice(range(start, stop, step)), but doesn’t actually build a range object. 12 | new_item = random.randrange(SIZE * 2) 13 | 14 | bisect.insort(my_list, new_item) 15 | print('%2d ->' % new_item, my_list) 16 | -------------------------------------------------------------------------------- /A_Pythonic_Card_Deck/README.md: -------------------------------------------------------------------------------- 1 | # A Pythonic Card Deck 2 | 3 | [Youtube Tutorial - A Pythonic Card Deck](https://youtu.be/ofZkuxnA2rI) 4 | 5 | 詳細程式碼可搭配 [eg_1_1.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/A_Pythonic_Card_Deck/eg_1_1.py) 安心服用:smirk: 6 | 7 | **<1>** 的用法可參考 [random_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/random_tutorial.py) 8 | 9 | **<2>** 的用法可參考 [namedtuple_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/namedtuple_tutorial.py) 10 | 11 | **<3>** 說明 12 | 13 | 在 python3 中，可以直接寫 14 | 15 | ```python 16 | class FrenchDeck: 17 | pass 18 | ``` 19 | 20 | 而在 python2 中，則必須要寫 21 | 22 | ```python 23 | class FrenchDeck(object): 24 | pass 25 | ``` 26 | 27 | 如果沒有特殊需求，用 python 3 吧:smile: 28 | 29 | **<4>** 說明 30 | 31 | 這邊建議大家可以直接用 Python Console 玩玩看 32 | 33 | ```cmd 34 | >>> [str(n) for n in range(2, 11)] 35 | ['2', '3', '4', '5', '6', '7', '8', '9', '10'] 36 | ``` 37 | 38 | 先知道他可以這樣用就好，未來的介紹會再進一步的介紹他。 39 | 40 | range 的用法可參考 [range.py](https://github.com/twtrubiks/python-notes/blob/master/range.py)， 41 | 42 | 你可能會問為什麼 range(2, 11) 不包含 11 呢 ? 43 | 44 | ***Why Slices and Range Exclude the Last Item ?*** 45 | 46 | slices [下一篇](xx)介紹會說明，這裡我們先看 range 就好， 47 | 48 | ```text 49 | The Pythonic convention of excluding the last item in slices and ranges works well with the zero-based indexing used in Python, C, and many other languages. 50 | ``` 51 | 52 | 由於上面的原因，所以有一些特性， 53 | 54 | 1. 像是 range(10)，我們很快的就可以知道它的長度是 10。 55 | 56 | 2. 當你有 start 和 stop 時，也可以很快的知道長度，像是 range(2, 11)，11-2 = 9。 57 | 58 | 3. 不用擔心 overlapping 的問題，看下面的例子 59 | 60 | ```python 61 | >>> seqs = list(range(10)) 62 | >>> seqs 63 | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] 64 | >>> seqs[:2] 65 | [0, 1] 66 | >>> seqs[2:] 67 | [2, 3, 4, 5, 6, 7, 8, 9] 68 | ``` 69 | 70 | 將字串轉成 list。 71 | 72 | ```cmd 73 | >>> list('JQKA') 74 | ['J', 'Q', 'K', 'A'] 75 | ``` 76 | 77 | **<5>** 說明 78 | 79 | `__init__` 就是 python 中所謂的 special methods， 80 | 81 | 當一個物件被建立時，`__init__` 就會被 invoke，像是 82 | 83 | ```cmd 84 | >>> deck = FrenchDeck() 85 | >>> deck 86 | 87 | ``` 88 | 89 | deck 是一個 instance。 90 | 91 | 這邊一定會有人問，`__init__` 要怎麼唸 :question: 92 | 93 | 難道是唸 double underscores init :question: 94 | 95 | 不過這樣的話像是 private attribute `self.__x` 不就衝突了嗎:question: 96 | 97 | 又或是我們唸 double underscores init double underscores :question: 98 | 99 | 還是我們唸 double under init double under :question: 100 | 101 | 天啊，唸完我都覺得我快不行惹:triumph: 102 | 103 | 所以，這邊提供大家一個唸法， 104 | 105 | 你可以唸 dunder-init ( dunder methods ) 。 106 | 107 | 這邊提醒大家，既然它都被叫做 special methods， 108 | 109 | 就不要隨便自己定義 `__foo__`，因為未來有一天他可能會被定義。 110 | 111 | **<6>** 說明 112 | 113 | `self._cards` 這邊可以先把他想成類似 java 中的 protected attribute， 114 | 115 | 字面上也很明顯了，受保護的屬性，雖然我們還是可以讀到他，但不建議這樣使用 ( 如下 ) 116 | 117 | ```python 118 | >>> deck = FrenchDeck() 119 | >>> deck._cards 120 | [Card(rank='2', suit='spades'), Card(rank='3', suit='spades').....] 121 | ``` 122 | 123 | 如果你今天看到的是 `self.__cards` ( double-underscore )，則是 private attribute。 124 | 125 | 詳細的 `protected` 以及 `private` attributes 之後的文章會有進一步的介紹， 126 | 127 | 這邊先有個概念即可:relaxed: 128 | 129 | 簡單說明一下下方的 code， 130 | 131 | ```python 132 | self._cards = [Card(rank, suit) for suit in self.suits 133 | for rank in self.ranks] 134 | ``` 135 | 136 | 相等於 137 | 138 | ```python 139 | self._cards = [] 140 | for suit in self.suits: 141 | for rank in self.ranks: 142 | self._cards.append(Card(rank, suit)) 143 | ``` 144 | 145 | 因為 python 的特性，才能有前者的這種寫法。 146 | 147 | **<7>** 說明 148 | 149 | 一樣是 special methods 150 | 151 | ```cmd 152 | >>> len(deck) 153 | ``` 154 | 155 | 當執行 `len(deck)` 時，`__len__` 會被 invoke 156 | 157 | **<8>** 說明 158 | 159 | 一樣是 special methods 160 | 161 | ```cmd 162 | >>> deck[0] 163 | Card(rank='2', suit='spades') 164 | >>> deck[24] 165 | Card(rank='K', suit='diamonds') 166 | >>> deck[-1] 167 | Card(rank='A', suit='hearts') 168 | >>> deck[12::13] 169 | [Card(rank='A', suit='spades'), Card(rank='A', suit='diamonds')...] 170 | ``` 171 | 172 | 當執行以上 code 時，`__getitem__` 會被 invoke， 173 | 174 | 先知道這樣用即可，未來會再做更詳細的介紹，因為是 slice 讓這功能如此強大:smile: 175 | 176 | ## 執行環境 177 | 178 | * Python 3.6.4 179 | 180 | ## Reference 181 | 182 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 183 | 184 | ## Donation 185 | 186 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 187 | 188 | ![alt tag](https://i.imgur.com/LRct9xa.png) 189 | 190 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 191 | -------------------------------------------------------------------------------- /A_Pythonic_Card_Deck/eg_1_1.py: -------------------------------------------------------------------------------- 1 | import collections 2 | from random import choice # <1> 3 | 4 | Card = collections.namedtuple('Card', ['rank', 'suit']) # <2> 5 | 6 | 7 | class FrenchDeck: # <3> 8 | ranks = [str(n) for n in range(2, 11)] + list('JQKA') # <4> 9 | suits = 'spades diamonds clubs hearts'.split() 10 | 11 | def __init__(self): # <5> 12 | self._cards = [Card(rank, suit) for suit in self.suits # <6> 13 | for rank in self.ranks] 14 | 15 | def __len__(self): # <7> 16 | return len(self._cards) 17 | 18 | def __getitem__(self, position): # <8> 19 | return self._cards[position] 20 | 21 | 22 | deck = FrenchDeck() 23 | print("deck.ranks", deck.ranks) 24 | print("deck.suits", deck.suits) 25 | 26 | print("len(deck)", (len(deck))) 27 | 28 | print("deck[1]", deck[1]) 29 | 30 | print("choice(deck)", choice(deck)) 31 | 32 | print("deck[12]", deck[12]) 33 | print("deck[12::13]", deck[12::13]) 34 | 35 | # just by implementing the __getitem__ special method, 36 | # our deck is also iterable 37 | for card in deck: 38 | print(card) 39 | -------------------------------------------------------------------------------- /Dictionaries_and_Sets_part_3_1/README.md: -------------------------------------------------------------------------------- 1 | # Dictionaries and Sets - Part 3-1 2 | 3 | [Youtube Tutorial - Dictionaries and Sets - Part 3-1]() 4 | 5 | ***Because of their crucial role, Python dicts are highly optimized. Hash tables are the engines behind Python’s high-performance dicts.*** 6 | 7 | 8 | ```text 9 | All mapping types in the standard library use the basic dict in their implementation, 10 | so they share the limitation that the keys must be hashable (the values need not be 11 | hashable, only the keys). 12 | ``` 13 | 14 | 如果不認識 hashable 15 | 16 | [Youtube Tutorial - What is the Hashable](https://youtu.be/-Qw3V2VoEQg) 17 | 18 | [What is the Hashable](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_hashable) 19 | 20 | ## Dictionaries 介紹 21 | 22 | [dictionary_using_items.py](https://github.com/twtrubiks/python-notes/blob/master/dictionary_using_items.py) - dictionary.items() 23 | 24 | [dictionary_get.py](https://github.com/twtrubiks/python-notes/blob/master/dictionary_get.py) - dictionary get() 25 | 26 | [dictionary_update.py](https://github.com/twtrubiks/python-notes/blob/master/dictionary_update.py) - dictionary update() 27 | 28 | [dict.fromkeys_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/dict.fromkeys_tutorial.py) 29 | 30 | ### dict Comprehensions 31 | 32 | 以下例子 dictcomps 33 | 34 | ```python 35 | numbers = [1, 2, 3] 36 | {number: number * 2 for number in numbers} 37 | ``` 38 | 39 | dictcomps 和 liscomps 是很相似的, 40 | 41 | 如果不了解 liscomps, 可參考[An Array of Sequences - Part 2-1](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_1) 42 | 43 | ### Handling Missing Keys with setdefault 44 | 45 | A subtle mapping method is setdefault. We don’t always need it, but when we do, it 46 | provides a significant speedup by avoiding redundant key lookups. 47 | 48 | [setdefault_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/setdefault_tutorial.py) 49 | 50 | # Mappings with Flexible Key Lookup 51 | 52 | Sometimes it is convenient to have mappings that return some made-up value when a missing key is searched. 53 | 54 | 有兩種方法 55 | 56 | defaultdict , add `__missing__` method 57 | 58 | # defaultdict: Another Take on Missing Keys 59 | 60 | 先來介紹 defaultdict 61 | 62 | [defaultdict_tutorial.py](https://github.com/twtrubiks/python-notes/blob/master/defaultdict_tutorial.py) 63 | 64 | 舉個例子, `dd = defaultdict(list)` 65 | 66 | 假如 `new-key` 沒有在 dd 中, 當執行 dd['new-key'] 步驟如下 67 | 68 | 1. 先呼叫 `list()` 建立一個新的 list 69 | 70 | 2. 將 new-key 插入 dd 中 (使用 new-key 當作 key) 71 | 72 | 3. 回傳 list 73 | 74 | The callable that produces the default values is held in an instance attribute called default_factory. 75 | 76 | Create a defaultdict with the list constructor as default_factory. 77 | 78 | If no default_factory is provided, the usual KeyError is raised for missing keys. 79 | 80 | 81 | The default_factory of a defaultdict is only invoked to pro‐ 82 | vide default values for __getitem__ calls, and not for the other 83 | methods. For example, if dd is a defaultdict , and k is a missing key, dd[k] will call the default_factory to create a default value, but dd.get(k) still returns None . 84 | 85 | The mechanism that makes defaultdict work by calling default_factory is actually the __missing__ special method, 86 | 87 | 88 | # The __missing__ Method 89 | 90 | if you 91 | subclass dict and provide a __missing__ method, the standard dict.__getitem__ will 92 | call it whenever a key is not found, instead of raising KeyError. 93 | 94 | ```python 95 | # Tests for item retrieval using `d[key]` notation:: 96 | >>> d = StrKeyDict0([('2', 'two'), ('4', 'four')]) 97 | >>> d['2'] 98 | 'two' 99 | >>> d[4] 100 | 'four' 101 | >>> d[1] 102 | Traceback (most recent call last): 103 | ... 104 | KeyError: '1' 105 | 106 | # Tests for item retrieval using `d.get(key)` notation:: 107 | >>> d.get('2') 108 | 'two' 109 | >>> d.get(4) 110 | 'four' 111 | >>> d.get(1, 'N/A') 112 | 'N/A' 113 | 114 | 115 | # Tests for the `in` operator:: 116 | >>> 2 in d 117 | True 118 | >>> 1 in d 119 | False 120 | ``` 121 | 122 | [eg_3_6.py](eg_3_6.py) 123 | 124 | 1. StrKeyDict0 inherits from dict. 125 | 126 | 2. Check whether key is already a str. If it is, and it’s missing, raise KeyError. 127 | 128 | 3. Build str from key and look it up. 129 | 4. The get method delegates to __getitem__ by using the self[key] notation; that 130 | gives the opportunity for our __missing__ to act. 131 | 132 | 5. If a KeyError was raised, __missing__ already failed, so we return the default. 133 | 134 | 6. Search for unmodified key (the instance may contain non-str keys), then for a 135 | str built from the key. -------------------------------------------------------------------------------- /Dictionaries_and_Sets_part_3_1/eg_3_6.py: -------------------------------------------------------------------------------- 1 | class StrKeyDict0(dict): # <1> 2 | 3 | def __missing__(self, key): 4 | if isinstance(key, str): # <2> 5 | raise KeyError(key) 6 | return self[str(key)] # <3> 7 | 8 | def get(self, key, default=None): 9 | try: 10 | return self[key] # <4> 11 | except KeyError: 12 | return default # <5> 13 | 14 | def __contains__(self, key): 15 | return key in self.keys() or str(key) in self.keys() # <6> -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Fluent Python Study Notes 2 | 3 | Fluent Python Study Notes 📝 4 | 5 | ## 介紹 6 | 7 | 本篇文章為閱讀 Fluent Python 所做的一些重點筆記，筆記會持續更新 8 | 9 | ## 教學 10 | 11 | 1-1. [A Pythonic Card Deck](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Pythonic_Card_Deck) 12 | 13 | 2-1. [An Array of Sequences Part 2-1](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_1) 14 | 15 | 2-2. [An Array of Sequences Part 2-2](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_2) 16 | 17 | 2-3. [An Array of Sequences Part 2-3](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_3)- 整理中 18 | 19 | 3-1. [Dictionaries and Sets Part 3-1](https://github.com/twtrubiks/fluent-python-notes/tree/master/Dictionaries_and_Sets_part_3_1)- 整理中 20 | 21 | other-1. [What is the Python GIL](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_python_GIL) 22 | 23 | other-2. [What is the Closures](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_closures) 24 | 25 | other-3. [What is the Hashable](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_hashable) 26 | 27 | other-4. [What is the Monkey Patch](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_Monkey_Patch) 28 | 29 | other-5. [What is the Descriptor](https://github.com/twtrubiks/fluent-python-notes/tree/master/what_is_the_Descriptor) 30 | 31 | ## 執行環境 32 | 33 | * Python 3.6.4 34 | 35 | ## Reference 36 | 37 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 38 | 39 | ## Donation 40 | 41 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 42 | 43 | 綠界科技ECPAY ( 不需註冊會員 ) 44 | 45 | ![alt tag](https://payment.ecpay.com.tw/Upload/QRCode/201906/QRCode_672351b8-5ab3-42dd-9c7c-c24c3e6a10a0.png) 46 | 47 | [贊助者付款](http://bit.ly/2F7Jrha) 48 | 49 | 歐付寶 ( 需註冊會員 ) 50 | 51 | ![alt tag](https://i.imgur.com/LRct9xa.png) 52 | 53 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 54 | 55 | ## 贊助名單 56 | 57 | [贊助名單](https://github.com/twtrubiks/Thank-you-for-donate) 58 | 59 | ## License 60 | 61 | MIT license 62 | -------------------------------------------------------------------------------- /what_is_the_Descriptor/README.md: -------------------------------------------------------------------------------- 1 | # What is the Descriptor in python 2 | 3 | 這篇文章主要會介紹 Descriptor (描述器) 4 | 5 | 建議先對 [What is the property](https://github.com/twtrubiks/python-notes/tree/master/what_is_the_property) 有一定的認識:smile: 6 | 7 | 除了可以用 property 限制設定的值之外, 也可以使用今天要介紹的 Descriptor. 8 | 9 | ```python 10 | class DataDescriptor: 11 | 12 | """a.k.a. data descriptor or enforced descriptor""" 13 | 14 | def __init__(self, storage_name): 15 | self.storage_name = storage_name 16 | 17 | def __get__(self, instance, owner): 18 | return instance.__dict__[self.storage_name] 19 | 20 | def __set__(self, instance, value): 21 | if value > 0: 22 | instance.__dict__[self.storage_name] = value 23 | 24 | # error 25 | # leading to infinite recursion. 26 | # setattr(instance, self.storage_name, value) 27 | else: 28 | raise ValueError("value must > 0") 29 | 30 | 31 | class A: 32 | data = DataDescriptor("data") 33 | 34 | def __init__(self, data): 35 | self.data = data 36 | 37 | 38 | a = A(123) 39 | print(a.data) 40 | a1 = A(-1) # ValueError: value must > 0 41 | ``` 42 | 43 | 透過 DataDescriptor 可以攔截 instance 屬性的 get 和 set, 44 | 45 | ( 當 Descriptor 成為 某class 的 屬性成員時, 46 | 47 | 可攔截該 class 的 instance 屬性中的 `__get__` `__set__` `__delete__` ) 48 | 49 | 在這邊 `__set__` 不能使用 `setattr(instance, self.storage_name, value)` 50 | 51 | 原因是因為 `data = DataDescriptor("data")` 名稱都是 data, 會導致 infinite recursion. 52 | 53 | 如果你想要使用 `setattr(instance, self.storage_name, value)`, 需將名稱改變. 54 | 55 | 修改如下, 56 | 57 | ```python 58 | class DataDescriptorV2: 59 | 60 | """a.k.a. data descriptor or enforced descriptor""" 61 | 62 | def __init__(self, storage_name): 63 | self.storage_name = storage_name 64 | 65 | def __get__(self, instance, owner): 66 | return getattr(instance, self.storage_name) 67 | 68 | def __set__(self, instance, value): 69 | if value > 0: 70 | setattr(instance, self.storage_name, value) 71 | else: 72 | raise ValueError("value must > 0") 73 | 74 | 75 | class A: 76 | data = DataDescriptorV2("data1") 77 | 78 | def __init__(self, data): 79 | self.data = data 80 | 81 | 82 | a = A(123) 83 | print(a.data) 84 | a1 = A(-1) # ValueError: value must > 0 85 | ``` 86 | 87 | 既然相同的名稱會導致錯誤, 這樣嘗試讓他自動填入 attribute name, 88 | 89 | 修改如下, 90 | 91 | ```python 92 | class DataDescriptorV3: 93 | __counter = 0 94 | 95 | """a.k.a. data descriptor or enforced descriptor""" 96 | 97 | def __init__(self): 98 | cls = self.__class__ 99 | prefix = cls.__name__ 100 | index = cls.__counter 101 | self.storage_name = "_{}#{}".format(prefix, index) 102 | cls.__counter += 1 103 | 104 | def __get__(self, instance, owner): 105 | return getattr(instance, self.storage_name) 106 | 107 | def __set__(self, instance, value): 108 | if value > 0: 109 | setattr(instance, self.storage_name, value) 110 | else: 111 | raise ValueError("value must > 0") 112 | 113 | 114 | class A: 115 | data = DataDescriptorV3() 116 | 117 | def __init__(self, data): 118 | self.data = data 119 | 120 | 121 | a = A(123) 122 | print(a.data) 123 | # a1 = A(-1) # ValueError: value must > 0 124 | ``` 125 | 126 | 透過 `__counter`, 自動將 `storage_name` 命名為 `_DataDescriptorV3#0`. 127 | 128 | 也可以使用 Descriptor 實作 ReadOnly 129 | 130 | (但如果只是要實作簡單的 readonly, 建議直接使用 property 比較方便簡單:smile:) 131 | 132 | 程式碼如下, 133 | 134 | ```python 135 | class ReadOnlyDescriptor: 136 | def __init__(self, storage_name): 137 | self.storage_name = storage_name 138 | 139 | def __get__(self, instance, owner): 140 | return self.storage_name 141 | 142 | def __set__(self, instance, value): 143 | raise AttributeError("read-only attribute") 144 | 145 | 146 | class A: 147 | 148 | data = ReadOnlyDescriptor("data") 149 | 150 | 151 | a = A() 152 | print(a.data) 153 | a.data = 123 # AttributeError: read-only attribute 154 | ``` 155 | 156 | 只需要實作 `__set__`, 並顯示合適的訊息. 157 | 158 | 如果只有 `__get__`, 沒有 `__set__`, 則是所謂的 NonData Descriptor. 159 | 160 | ```python 161 | class NonDataDescriptor: 162 | 163 | """a.k.a. non-data or shadowable descriptor""" 164 | 165 | def __init__(self, storage_name): 166 | self.storage_name = storage_name 167 | 168 | def __get__(self, instance, owner): 169 | return instance.__dict__[self.storage_name] 170 | ``` 171 | 172 | NonDataDescriptor 很適合使用在 cache 快取, 173 | 174 | 假設某個運算花費很大, 可以把它快取起來, 程式碼如下 175 | 176 | ```python 177 | import time 178 | 179 | 180 | class NonDataDescriptor: 181 | 182 | """a.k.a. non-data or shadowable descriptor""" 183 | 184 | def __init__(self, storage_name): 185 | self.storage_name = storage_name 186 | 187 | def __get__(self, instance, owner): 188 | print("trigger __get__") 189 | time.sleep(5) # simulation expensive 190 | instance.__dict__[self.storage_name] = self.storage_name 191 | return self.storage_name 192 | 193 | 194 | class A: 195 | 196 | data = NonDataDescriptor("data") 197 | 198 | 199 | a = A() 200 | print(a.data) 201 | print(a.data) 202 | print(a.data) 203 | ``` 204 | 205 | 上面這段 code, 只會輸出一次 `trigger __get__`, 剩下都是直接透過快取取資料. 206 | 207 | 208 | ## Donation 209 | 210 | 如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 211 | 212 | ![alt tag](https://i.imgur.com/LRct9xa.png) 213 | 214 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 215 | 216 | ## License 217 | 218 | MIT license 219 | -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/README.md: -------------------------------------------------------------------------------- 1 | # What is the Monkey Patch 2 | 3 | [Youtube Tutorial - 認識 Monkey Patch in Python - part 1](https://youtu.be/eJXSIbEJIXo) 4 | 5 | [Youtube Tutorial - 認識 Monkey Patch in Python - part 2](https://youtu.be/4FRGP7iRsM0) 6 | 7 | [Youtube Tutorial - 認識 Monkey Patch in Python - part 3](https://youtu.be/JRCPzue4sFU) 8 | 9 | 這篇文章主要會介紹 **Monkey Patch**， 10 | 11 | 中文的意思我就不翻譯了，簡單說，你可以把它想成是一種補丁， 12 | 13 | 可以將 runtime ( 執行中 ) 的程式，動態的改變 class or module or function， 14 | 15 | 且不需要修改 source code，通常使用在增加功能 ( 暫時性 ) 和修正 bugs。 16 | 17 | 來看一個例子， 18 | 19 | [demo1.py](demo1.py) 20 | 21 | ```python 22 | class A: 23 | def speak(self): 24 | return "hello" 25 | 26 | def speak_patch(self): 27 | return "world" 28 | 29 | A.speak = speak_patch # <2> 30 | some_class = A() 31 | print('some_class.speak():', some_class.speak()) # <1> 32 | ``` 33 | 34 | <1> 的部分會顯示 `world`，而不會顯示 `hello`，原因是因為我們在 <2> 的部分 35 | 36 | 將它 Monkey Patch 了，使用 `speak_patch` 這個 function 去取代掉原來的 `speak`。 37 | 38 | 再來看一個例子， 39 | 40 | [demo2.py](demo2.py) 41 | 42 | ```python 43 | class A: 44 | def __init__(self, array): 45 | self._list = array 46 | 47 | # def __len__(self): # <2> 48 | # return len(self._list) 49 | 50 | def length(obj): # <1> 51 | return len(obj._list) # <2> 52 | 53 | 54 | A.__len__ = length # <3> 55 | 56 | a = A([1, 2, 3]) 57 | print('length:', len(a)) 58 | ``` 59 | 60 | A class 並沒有實作 `__len__` 的方法 ( 將 <2> 註解起來 )，如果是在 runtime 中， 61 | 62 | 我們可不可以把 `__len__` 方法實作上去:question: 63 | 64 | 可以，只要將 <1> 的部分的 function 先寫出來，記得，該有的參數還是要有，但名稱可以 65 | 66 | 不一樣，像是原本 <2> 是 self，在 <1> 可以任意寫 ( 例如 obj )，因為會在 <3> 的部分 67 | 68 | 自己去 mapping 起來，但是取長度時，也就是 <2> 的部分，就必須要寫 `obj._list`， 69 | 70 | 重點是後面的 `_list` 屬性。 71 | 72 | 經過以上的部分，就可以成功的在 runtime 時，實作 A class 的 `__len__`，這邊你可能 73 | 74 | 會問我，那為什麼不一開始就實作 `__len__` 就好，像是， 75 | 76 | ```python 77 | class A: 78 | def __init__(self, array): 79 | self._list = array 80 | 81 | def __len__(self): # <2> 82 | return len(self._list) 83 | ``` 84 | 85 | 原因是可能我們不希望去修改 A class 的 source code，而這修改也只是暫時性的，像是 86 | 87 | 測試之類的。 88 | 89 | 順帶一提， 90 | 91 | 其實它打破了 encapsulation ( 封裝 ) 的概念， 而且傾向於緊密耦合 ( tightly coupled ) 92 | 93 | ，也就是破壞 SOLID 的概念 ( SOLID 關鍵字 GOOGLE 就可以能到很多說明 )。 94 | 95 | 所以他是暫時的解決方法，對於整個 code 的整合，並不是一個推薦的技術。 96 | 97 | [Youtube Tutorial - 認識 Monkey Patch in Python - part 2](https://youtu.be/4FRGP7iRsM0) 98 | 99 | Python 中的 Monkey Patch 是有限制的，它限制你不能 patched built-in types， 100 | 101 | 舉個例子，假如我今天想要 patched str 這個物件， 102 | 103 | [demo3.py](demo3.py) 104 | 105 | ```python 106 | def find(self, sub, start=None, end=None): 107 | return 'ok' 108 | 109 | str.find = find 110 | ``` 111 | 112 | 你會發現得到 error messages，這個限制我覺得不錯，為什麼呢:question: 113 | 114 | 原因是可以保證 built-in types 的功能都是原本的，避免有人去 patched 它之後， 115 | 116 | 導致後續一堆奇怪的問題。 117 | 118 | 記得，Monkey Patch 不能濫用，否則會造成系統難以維護以及 code 很難理解 119 | 120 | ( 因為你東 patched 一個，西 patched 一個，誰看得懂 :triumph:) 121 | 122 | 回頭再來看 [demo1.py](demo1.py)， 123 | 124 | ```python 125 | class A: 126 | def speak(self): 127 | return "hello" 128 | 129 | def speak_patch(self): 130 | return "world" 131 | 132 | A.speak = speak_patch # <2> 133 | some_class = A() 134 | print('some_class.speak():', some_class.speak()) # <1> 135 | 136 | some_class2 = A() 137 | print('some_class2.speak():', some_class2.speak()) # <3> 138 | ``` 139 | 140 | 注意 <3> 的部分，會顯示 world ，而非 hello，因為他已經被 patched， 141 | 142 | 除非你把它重新 patched 回來，因為這個特性，所以絕對不能濫用， 143 | 144 | 不然使用者會覺得很奇怪，不是應該要輸出 hello 嗎:question: 145 | 146 | 但是卻是輸出 world。 147 | 148 | 動態修改 class or module or function 的這個特性，和 dessign patterns 中的 149 | 150 | adapter 類似 ( 可以先把他想成是變壓器的概念 )，只不過在 adapter 中，是 151 | 152 | 實作一個全新的 Adapter class 來處理同一個問題。 153 | 154 | 以後我會在介紹 adapter，可先參考 [python-patterns-adapter.py](https://github.com/faif/python-patterns/blob/master/structural/adapter.py)。 155 | 156 | python 中的 `gevent` module 也有使用到 Monkey Patch 的特性。 157 | 158 | [Youtube Tutorial - 認識 Monkey Patch in Python - part 3](https://youtu.be/JRCPzue4sFU) 159 | 160 | 這邊補充一下，`types` module 中的 `types.MethodType` 161 | 162 | 很類似 Monkey Patch，來看一個 `types.MethodType` 的例子， 163 | 164 | [demo4.py](demo4.py) 165 | 166 | ```python 167 | import types 168 | 169 | class A: 170 | def speak(self): 171 | return "hello" 172 | 173 | def speak_patch(self): 174 | return "world" 175 | 176 | a = A() 177 | a.speak = types.MethodType(speak_patch, a) # <1> 178 | print('a.speak():', a.speak()) # <2> 179 | 180 | a2 = A() 181 | print('a2.speak():', a2.speak()) # <3> 182 | ``` 183 | 184 | <1> 的部分，使用方法 `types.MethodType( fun, obj)`，這邊要注意的是， 185 | 186 | 我們是放一個 instance (a) 進去，所以 <2> 的輸出是 world，而 <3> 的 187 | 188 | 輸出則是 hello ( 因為 a2 的 instance 我們並沒有去修改 )。 189 | 190 | 再來看一個例子， 191 | 192 | [demo5.py](demo5.py) 193 | 194 | ```python 195 | import types 196 | 197 | class A: 198 | def speak(self): 199 | return "hello" 200 | 201 | def speak_patch(self): 202 | return "world" 203 | 204 | A.speak = types.MethodType(speak_patch, A) # <1> 205 | a = A() 206 | print('a.speak():', a.speak()) # <2> 207 | 208 | a2 = A() 209 | print('a2.speak:', a2.speak()) # <3> 210 | ``` 211 | 212 | <1> 的部分，我們是方一個 A class 進去，所以 <2> 和 <3> 的輸出 213 | 214 | 都會是 world，因為我們改變了 A class，這幾乎就和前面所介紹的 215 | 216 | Monkey Patch 是一樣的功能。 217 | 218 | 這邊主要是強調 patched instance ( [demo4.py](demo4.py) ) 或是 219 | 220 | patched class ( [demo5.py](demo5.py) ) 是不太一樣的。 221 | 222 | ## Donation 223 | 224 | 如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 225 | 226 | ![alt tag](https://i.imgur.com/LRct9xa.png) 227 | 228 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 229 | 230 | ## License 231 | 232 | MIT license 233 | -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/demo1.py: -------------------------------------------------------------------------------- 1 | class A: 2 | def speak(self): 3 | return "hello" 4 | 5 | 6 | def speak_patch(self): 7 | return "world" 8 | 9 | 10 | A.speak = speak_patch 11 | some_class = A() 12 | print('some_class.speak():', some_class.speak()) 13 | 14 | some_class2 = A() 15 | print('some_class2.speak():', some_class2.speak()) 16 | -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/demo2.py: -------------------------------------------------------------------------------- 1 | class A: 2 | def __init__(self, array): 3 | self._list = array 4 | 5 | # def __len__(self): 6 | # return len(self._list) 7 | 8 | 9 | def length(obj): 10 | return len(obj._list) 11 | 12 | 13 | A.__len__ = length 14 | 15 | a = A([1, 2, 3]) 16 | print('length:', len(a)) 17 | -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/demo3.py: -------------------------------------------------------------------------------- 1 | def find(self, sub, start=None, end=None): 2 | return 'ok' 3 | 4 | str.find = find -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/demo4.py: -------------------------------------------------------------------------------- 1 | import types 2 | 3 | 4 | class A: 5 | def speak(self): 6 | return "hello" 7 | 8 | 9 | def speak_patch(self): 10 | return "world" 11 | 12 | 13 | a = A() 14 | a.speak = types.MethodType(speak_patch, a) 15 | print('a.speak():', a.speak()) 16 | 17 | a2 = A() 18 | print('a2.speak():', a2.speak()) 19 | -------------------------------------------------------------------------------- /what_is_the_Monkey_Patch/demo5.py: -------------------------------------------------------------------------------- 1 | import types 2 | 3 | 4 | class A: 5 | def speak(self): 6 | return "hello" 7 | 8 | 9 | def speak_patch(self): 10 | return "world" 11 | 12 | 13 | A.speak = types.MethodType(speak_patch, A) 14 | a = A() 15 | print('a.speak():', a.speak()) 16 | 17 | a2 = A() 18 | print('a2.speak():', a2.speak()) 19 | 20 | 21 | -------------------------------------------------------------------------------- /what_is_the_closures/README.md: -------------------------------------------------------------------------------- 1 | # What is the closures in python 2 | 3 | [Youtube Tutorial - What is the closures in python](https://youtu.be/XxHg4fHDCmk) 4 | 5 | 這篇文章主要會介紹 closures，會使用 fluent python 中的範例來介紹， 6 | 7 | 以下是 fluent python 中對 closures 的解釋， 8 | 9 | ```text 10 | a closure is a function that retains the bindings of the free variables that 11 | exist when the function is defined, so that they can be used later when the 12 | function is invoked and the defining scope is no longer available. 13 | ``` 14 | 15 | closure ( 閉包 ) 是一種函數，它保留定義函數時存在的 free variables ( 自由變數 ) 的綁定， 16 | 17 | 以便之後在調用函數並且定義 scope ( 範圍 ) 不再可用時可以使用它們。 18 | 19 | 以上看翻譯看不懂沒關係，我們來看例子:smile: 20 | 21 | [demo1.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_closures/demo1.py) 22 | 23 | ```python 24 | def make_average(): 25 | # take advantage of the fact that series(lists) are ""mutable"" 26 | series = [] 27 | 28 | def average(new_value): 29 | # series is a free variable 30 | series.append(new_value) 31 | print('series', series) 32 | total = sum(series) 33 | return total / len(series) 34 | 35 | return average 36 | 37 | 38 | avg = make_average() 39 | print(avg(10)) # series [10] 40 | print(avg(11)) # series [10, 11] 41 | print(avg(12)) # series [10, 11, 12] 42 | 43 | print(avg.__code__.co_freevars) # ('series',) 44 | ``` 45 | 46 | 當呼叫 `make_average()` 時，他會回傳 `average`(`average`為 inner function )， 47 | 48 | 而當 `average` 被呼叫，就會將參數傳入，並且添加到 `series` 中，最後計算目前的平均。 49 | 50 | `series` 是 `make_average()` 中的 local variable，因為初始化 `series = []` 51 | 52 | 發生在這個 function 中，但是當 return average 時，他的 local scope 就消失了。 53 | 54 | ![img](https://i.imgur.com/m4lMALN.png) 55 | 56 | free variable 的意思是指 `series` 不會被綁定在 local scope 中。 57 | 58 | 剛剛上面我們有特別強調 `series` 是 **mutable**，假如今天我們換上一個 **immutable** 59 | 60 | 會發生什麼事情呢:question: 61 | 62 | [demo2.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_closures/demo2.py) 63 | 64 | ```python 65 | def make_average(): 66 | # count is a number or any immutable type 67 | count = 0 68 | total = 0 69 | 70 | def average(new_value): 71 | count += 1 72 | total += new_value 73 | return total / count 74 | 75 | return average 76 | 77 | 78 | avg = make_average() 79 | print('avg(10)', avg(10)) 80 | 81 | ``` 82 | 83 | 執行後，你會發現得到如下的錯誤， 84 | 85 | ```text 86 | UnboundLocalError: local variable 'count' referenced before assignment 87 | ``` 88 | 89 | 原因是因為在這邊我們是真的 assign 值給 count，使他變成一個 local variable， 90 | 91 | ( 而 count 根本還沒有定義，所以當然導致錯誤 ) 92 | 93 | 而在 [demo1.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_closures/demo1.py) 中會正常 work，原因是我們只是使用 `series.append` ( 並沒有 assign 值給 series 94 | 95 | ，而是 method call )。 96 | 97 | 那如果我們真的想用 **immutable**，有什麼方法可以解決呢 :question: 98 | 99 | 在 python3 中提供了 `nonlocal` 給我們使用，可以將一個變數變成 free variable ( 即使是被 assign 一個新的值 )， 100 | 101 | 這邊要注意的是 `nonlocal` 不是 local variables，也不是 global variables，通常會使用在 nested function 中。 102 | 103 | [demo3.py](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_closures/demo3.py) 104 | 105 | ```python 106 | def make_average(): 107 | count = 0 108 | total = 0 109 | 110 | def average(new_value): 111 | # the nonlocal declaration was introduced in Python 3. 112 | nonlocal count, total 113 | count += 1 114 | total += new_value 115 | return total / count 116 | 117 | return average 118 | 119 | 120 | avg = make_average() 121 | print('avg(10)', avg(10)) 122 | print('avg(11)', avg(11)) 123 | print('avg(12)', avg(12)) 124 | 125 | print(avg.__code__.co_freevars) # ('count', 'total') 126 | ``` 127 | 128 | 加上 `nonlocal` 之後，就可以正常 work 了。 129 | 130 | ## Donation 131 | 132 | 如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 133 | 134 | ![alt tag](https://i.imgur.com/LRct9xa.png) 135 | 136 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 137 | 138 | ## License 139 | 140 | MIT license 141 | -------------------------------------------------------------------------------- /what_is_the_closures/demo1.py: -------------------------------------------------------------------------------- 1 | def make_average(): 2 | # take advantage of the fact that series(lists) are ""mutable"" 3 | series = [] 4 | 5 | def average(new_value): 6 | # series is a free variable 7 | series.append(new_value) 8 | print('series', series) 9 | total = sum(series) 10 | return total / len(series) 11 | 12 | return average 13 | 14 | 15 | avg = make_average() 16 | print(avg(10)) # series [10] 17 | print(avg(11)) # series [10, 11] 18 | print(avg(12)) # series [10, 11, 12] 19 | 20 | print(avg.__code__.co_freevars) # ('series',) 21 | -------------------------------------------------------------------------------- /what_is_the_closures/demo2.py: -------------------------------------------------------------------------------- 1 | def make_average(): 2 | # count is a number or any immutable type 3 | count = 0 4 | total = 0 5 | 6 | def average(new_value): 7 | count += 1 8 | total += new_value 9 | return total / count 10 | 11 | return average 12 | 13 | 14 | avg = make_average() 15 | print('avg(10)', avg(10)) 16 | -------------------------------------------------------------------------------- /what_is_the_closures/demo3.py: -------------------------------------------------------------------------------- 1 | def make_average(): 2 | count = 0 3 | total = 0 4 | 5 | def average(new_value): 6 | # the nonlocal declaration was introduced in Python 3. 7 | nonlocal count, total 8 | count += 1 9 | total += new_value 10 | return total / count 11 | 12 | return average 13 | 14 | 15 | avg = make_average() 16 | print('avg(10)', avg(10)) 17 | print('avg(11)', avg(11)) 18 | print('avg(12)', avg(12)) 19 | 20 | print(avg.__code__.co_freevars) # ('count', 'total') 21 | -------------------------------------------------------------------------------- /what_is_the_hashable/README.md: -------------------------------------------------------------------------------- 1 | # What is the Hashable 2 | 3 | [Youtube Tutorial - What is the Hashable]( https://youtu.be/-Qw3V2VoEQg) 4 | 5 | 這篇文章主要會介紹 **Hashable**， 6 | 7 | 以下為 fluent python 中的一段說明， 8 | 9 | ```text 10 | An object is hashable if it has a hash value which never changes during its lifetime (it needs a __hash__() method), 11 | and can be compared to other objects (it needs an __eq__() method). Hashable objects which compare equal must have 12 | the same hash value. 13 | ``` 14 | 15 | 假如一個物件是 hashable，那麼在他的生命週期中，會有一個固定不變的 hash 值 ( 需要 `__hash__()` method)， 16 | 17 | 而且可以和其他的物件比較 ( 需要 `__eq__()` method )。 18 | 19 | 假如兩個物件相等，hashable objects 需要有相同的 hash 值才算是相等。 20 | 21 | immutable types ( tuple, str, and bytes ) are all hashable. 22 | 23 | 上面這句話沒有完全正確，來看一個例子， 24 | 25 | ```python 26 | >>> t1 = (1 , 2 , (30, 40)) # <1> 27 | >>> hash(t1) 28 | 1350807749 29 | >>> t2 = (1 , 2 , [30, 40]) # <2> 30 | >>> hash(t2) 31 | Traceback (most recent call last): 32 | File "", line 1, in 33 | TypeError: unhashable type: 'list' 34 | ``` 35 | 36 | <1> 的部分沒什麼問題，他是 hashable。 37 | 38 | <2> 的部分就有問題了，出現 error message，並且告訴你， 39 | 40 | list 是 unhashable ( 因為 list 是 mutable )，也就是說， 41 | 42 | 假如 tuple 裡面全部都是 immutable，這樣他就是 hashable， 43 | 44 | 但假如 tuple 裡面有包含 mutable ( 像是 <2> 的部分 )， 45 | 46 | 這樣他就是 unhashable。 47 | 48 | 補充，mutable types 49 | 50 | ```text 51 | list, bytearray, array.array, collections.deque, and memoryview. 52 | ``` 53 | 54 | 如果想了解更多，可參考我之前的文章 [An Array of Sequences - Part 2-1](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_1)。 55 | 56 | mutable 都是 unhashable， 57 | 58 | ```python 59 | >>> a = [1,2,3] 60 | >>> hash(a) 61 | Traceback (most recent call last): 62 | File "", line 1, in 63 | TypeError: unhashable type: 'list' 64 | ``` 65 | 66 | 所以前面才會和大家說，`immutable types are all hashable.` 67 | 68 | 這句話沒有完全正確。 69 | 70 | python [hashable](https://docs.python.org/3/glossary.html#term-hashable) 文件中有一段說明， 71 | 72 | ```text 73 | Objects which are instances of user-defined classes are hashable by default. 74 | They all compare unequal (except with themselves), and their hash value is derived from their id(). 75 | ``` 76 | 77 | 使用者自己定義的 class 的 instances 都是 hashable，因為是從 `id()` 中取回， 78 | 79 | 看下面的例子， 80 | 81 | ```python 82 | >>> class A: 83 | ... pass 84 | ... 85 | >>> a = A() 86 | >>> hash(a) # <1> 87 | 3660815 88 | >>> id(a) # <2> 89 | 58573040 90 | ``` 91 | 92 | 有沒有發現 <1> 和 <2> 的部分不相同，在舊版的 python 中，他們會是相等的， 93 | 94 | 但是後來的版本會是 `hash(a) == id(a)/16`，原因是 CPython 裡面的 id 都是 16 的倍數， 95 | 96 | 相關來源可參考 [here](https://stackoverflow.com/questions/11324271/what-is-the-default-hash-in-python)。 97 | 98 | 在 Python 中，dict 的 key 一定要是 hashable，主要是因為 dict 內部的實作，至於實作的部分， 99 | 100 | 就要談到 hash table，這部分又要談很久，所以留到下次介紹:smile: 101 | 102 | 延伸閱讀，利用 Hash Table 解題目，可參考 [Two_Sum_001.py](https://github.com/twtrubiks/leetcode-python/blob/master/Two_Sum_001.py) 103 | 104 | ## Donation 105 | 106 | 如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 107 | 108 | ![alt tag](https://i.imgur.com/LRct9xa.png) 109 | 110 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 111 | 112 | ## License 113 | 114 | MIT license 115 | -------------------------------------------------------------------------------- /what_is_the_python_GIL/README.md: -------------------------------------------------------------------------------- 1 | # What is the Python GIL 2 | 3 | [Youtube Tutorial - What is the Python GIL](https://youtu.be/weqxMa-tBfQ) 4 | 5 | **Cpython** ( 大多數我們看到的 Python 或是你正在使用的 Python 都是 Cpython ) 不是一個 thread-safe， 6 | 7 | 所以我們需要一個 GIL ( 全名為 Global Interpreter Lock )，主要是為了資料的完整性以及安全性才會有 GIL 8 | 9 | 的存在，換句話說，在同時間內，只能有一個 thread 執行 Python bytecodes ( 可參考 [dis module](https://docs.python.org/3/library/dis.html) )， 10 | 11 | 如果想更了解 [dis module](https://docs.python.org/3/library/dis.html)，可參考 [An Array of Sequences - Part 2-2](https://github.com/twtrubiks/fluent-python-notes/tree/master/A_Array_of_Sequences_part_2_2) 這篇。 12 | 13 | 這就是有時候你可能會聽到別人說 Python 因為 GIL 的關係導致很慢的原因 14 | 15 | ( 但這邊要注意，是 CPython interpreter 的關係，並不是 Python 本身的關係，像是 Jython 以及 IPython 16 | 17 | 就沒有 GIL，所以開頭特別強調 CPython )。 18 | 19 | 一般來說，在寫 Python code 的時候，我們不會去控制 GIL，但當執行一些非常耗時的任務時，像是 20 | 21 | built-in function 或一些額外用 C 寫的功能就會去釋放 GIL，事實上，用 C 寫的 Python library 可以管理 GIL， 22 | 23 | 但因為較為複雜，所以大部分的開發者不會去寫。 24 | 25 | 全部的 standard library function 實現 blocking I/O 時 ( 等待 OS 的 response ) 釋放 GIL ( 當你擁有 GIL 這把鎖的時候， 26 | 27 | 你才能執行 Python bytecodes，你可以將 GIL 想成是一張通行證 :smiley: )，也就是說 Python 是 I/O bound，當一個 28 | 29 | thread 在等待網路的 response 時，這個 blocked I/O function 就會釋放 GIL 給其他的 thread，像是執行 `time.sleep()` 30 | 31 | function 時會釋放 GIL。 32 | 33 | 接下來我們思考另一個問題，這邊大家可以先泡杯咖啡:relaxed: 34 | 35 | 一個 CPU ( 單核心 ) 執行一個 thread，那今天假如我有四核心，這樣是不是一次可以執行四個 thread :question: 36 | 37 | 也就是執行速度快四倍:confused: 38 | 39 | 在 Python 的世界中，答案是快沒多少，甚至多核心可能更慢 ( 也更浪費 )。 40 | 41 | 為什麼在 Python 中會這樣呢:question: 主要還是因為 GIL 的關係， 42 | 43 | 先來看一段 code ( [thread_demo](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_python_GIL/thread_demo.py) )，使用電腦的核心數作為 thread 的數量， 44 | 45 | ```python 46 | from threading import Thread 47 | import os 48 | 49 | 50 | def loop(): 51 | i = 0 52 | while True: 53 | i += 1 54 | 55 | 56 | def main(): 57 | print('start') 58 | for _ in range(os.cpu_count()): 59 | t = Thread(target=loop) 60 | t.start() 61 | 62 | 63 | if __name__ == '__main__': 64 | main() 65 | ``` 66 | 67 | 執行結果如下， 68 | 69 | ![alt tag](https://i.imgur.com/KyGkDjD.png) 70 | 71 | 有沒有很奇怪，理論上，CPU 應該要跑到 100 % 才對，可是它卻沒有 :question: 72 | 73 | 也就是沒有有效的利用多核心的優勢，儘管你是多核心，執行起來卻像是單核心。 74 | 75 | 先來看單核心的情況，當主 thread 達到 **閾值** 釋放 GIL 時，被喚醒的 thread 可以很順利的拿到 GIL 並執行 76 | 77 | Python bytecodes。 78 | 79 | 但當多核心的時候就會有問題，當主 thread ( CPU0 ) 達到 **閾值** 釋放 GIL 時，其他 CPU 上的 thread 被喚醒後 80 | 81 | 開始搶這個GIL ，但有時候，主 thread ( CPU0 ) 又拿到 GIL，所以導致其他被喚醒的 thread 白白浪費 CPU 時間在 82 | 83 | 那邊等待，然後又睡著，接著又被喚醒，這種惡性循環，也就是 thrashing，而且切換 GIL 的成本是很高的。 84 | 85 | 所以就算你的 CPU 有八核心，在 Python 中，一次還是只能運作在一個核心上 ( 因為 GIL ):expressionless: 86 | 87 | 補充說明閾值，在 Python2 中是執行 1000 個 bytecodes 釋放 GIL，而在 Python3 中則是執行超過 15ms 釋放 GIL。 88 | 89 | 那我如果真的想要有效的利用多核心我該怎麼做呢:confused: 90 | 91 | 這時候可以使用 process，process 不會有 GIL 的問題，因為各個 process 之間有自己獨立的 GIL，所以不會互相 92 | 93 | 爭搶，我們來看一段 code ( [process_demo](https://github.com/twtrubiks/fluent-python-notes/blob/master/what_is_the_python_GIL/process_demo.py) ) ，使用電腦的核心數作為 process 的數量， 94 | 95 | ```python 96 | from multiprocessing import Process 97 | import os 98 | 99 | 100 | def loop(): 101 | i = 0 102 | while True: 103 | i += 1 104 | 105 | 106 | def main(): 107 | print('start') 108 | for _ in range(os.cpu_count()): 109 | t = Process(target=loop) 110 | t.start() 111 | 112 | 113 | if __name__ == '__main__': 114 | main() 115 | ``` 116 | 117 | 執行結果如下， 118 | 119 | ![alt tag](https://i.imgur.com/6DTRH5B.png) 120 | 121 | CPU 的確衝到了 100 %，也就是成功的利用了多核心的優勢，並且不受 GIL 的影響。 122 | 123 | 以下小結論， 124 | 125 | 當有高 CPU ( CPU-bound ) 計算的工作時，我們使用 [Multiprocessing](https://docs.python.org/3.6/library/multiprocessing.html)。 126 | 127 | 當有大量 I/O ( I/O-bound ) 的工作時，我們使用 [Threading](https://docs.python.org/3/library/threading.html)，像是爬蟲。 128 | 129 | ## 執行環境 130 | 131 | * Python 3.6.4 132 | 133 | ## Reference 134 | 135 | * [fluentpython/example-code](https://github.com/fluentpython/example-code) 136 | * [Grok the GIL: How to write fast and thread-safe Python](https://opensource.com/article/17/4/grok-gil) 137 | 138 | ## Donation 139 | 140 | 文章都是我自己研究內化後原創，如果有幫助到您，也想鼓勵我的話，歡迎請我喝一杯咖啡:laughing: 141 | 142 | ![alt tag](https://i.imgur.com/LRct9xa.png) 143 | 144 | [贊助者付款](https://payment.opay.tw/Broadcaster/Donate/9E47FDEF85ABE383A0F5FC6A218606F8) 145 | -------------------------------------------------------------------------------- /what_is_the_python_GIL/process_demo.py: -------------------------------------------------------------------------------- 1 | from multiprocessing import Process 2 | import os 3 | 4 | 5 | def loop(): 6 | i = 0 7 | while True: 8 | i += 1 9 | 10 | 11 | def main(): 12 | print('start') 13 | for _ in range(os.cpu_count()): 14 | t = Process(target=loop) 15 | t.start() 16 | 17 | 18 | if __name__ == '__main__': 19 | main() 20 | -------------------------------------------------------------------------------- /what_is_the_python_GIL/thread_demo.py: -------------------------------------------------------------------------------- 1 | from threading import Thread 2 | import os 3 | 4 | 5 | def loop(): 6 | i = 0 7 | while True: 8 | i += 1 9 | 10 | 11 | def main(): 12 | print('start') 13 | for _ in range(os.cpu_count()): 14 | t = Thread(target=loop) 15 | t.start() 16 | 17 | 18 | if __name__ == '__main__': 19 | main() 20 | --------------------------------------------------------------------------------