├── .gitignore ├── CONTRIBUTORS.markdown ├── Makefile ├── README.markdown ├── appendix.markdown ├── listings ├── access.py ├── call.py ├── closer.py ├── descriptor.py ├── fileobject.py ├── list.py ├── slate.py └── word.py ├── magicmarkdown.py ├── magicmethods.html ├── magicmethods.markdown ├── magicmethods.pdf ├── magicmethods.py ├── magicmethods.tex ├── style.css └── table.markdown /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | -------------------------------------------------------------------------------- /CONTRIBUTORS.markdown: -------------------------------------------------------------------------------- 1 | ## Thanks to the following people for helping out ## 2 | 3 | - stevelosh on Reddit for contributing the stylesheet 4 | - Andrew Dalke for pointing out that I was missing `__index__`, `__getstate__`, and `__setstate__` 5 | - ewiethoff on Reddit for suggesting a table mapping magic methods to any special syntax that invokes them 6 | - richleland (Richard Leland) for providing styles for Pygments syntax highlighting 7 | - Simon Sapin for pointing out a number of bugs 8 | - sarenji (David Peter) for pointing out some bugs in the examples 9 | - michaelcontento (Michael Contento) for fixing a missing argument in `__exit__` 10 | - nosklo for giving some constructive criticism on #python 11 | - rhettinger for pointing out a number of omissions in the guide 12 | - aasted for pointing out an error in the organization of the guide 13 | - seliopou for making the guide a little more visually appealing 14 | - redtoad for correcting some typos 15 | - bobuss for finding an error in the `AccessCounter` example 16 | - Andrew-Crosio for finding yet another typo 17 | - william-mi-leslie for finding weaknesses and errors in the guide's treatment of operator overloading 18 | - petrushev for expanding the guide's description of `__reversed__` -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | docs: magicmethods.html magicmethods.pdf clean 2 | 3 | html: magicmethods.html 4 | 5 | pdf: magicmethods.pdf 6 | 7 | magicmethods.html: table.markdown magicmethods.markdown appendix.markdown 8 | python magicmarkdown.py 9 | 10 | magicmethods.pdf: magicmethods.tex 11 | pdflatex magicmethods.tex 12 | 13 | clean: 14 | rm -f markedup.html magicmethods.log magicmethods.dvi magicmethods.aux 15 | -------------------------------------------------------------------------------- /README.markdown: -------------------------------------------------------------------------------- 1 | ## Python 的神奇方法指南 2 | 3 | 4 | 原文:[A Guide to Python's Magic Methods](http://www.rafekettler.com/magicmethods.html) 5 | 6 | 作者:Rafe Kettler 7 | 8 | 译文:[Python 的神奇方法指南](./magicmethods.markdown) 9 | 10 | 本文在 Creative Commons CC--NC-BY-SA (see http://creativecommons.org/licenses/by-nc-sa/3.0/) 协议下发布。 11 | 12 | ## 开始阅读 13 | 14 | [目录](./table.markdown) 15 | -------------------------------------------------------------------------------- /appendix.markdown: -------------------------------------------------------------------------------- 1 | ##Appendix 1: How to Call Magic Methods## 2 | 3 | Some of the magic methods in Python directly map to built-in functions; in this case, how to invoke them is fairly obvious. However, in other 4 | cases, the invocation is far less obvious. This appendix is devoted to exposing non-obvious syntax that leads to magic methods getting called. 5 | 6 | Magic Method | When it gets invoked (example) | Explanation 7 | ---------------------- | ---------------------------------- | -------------------------------------------- 8 | `__new__(cls [,...])` | `instance = MyClass(arg1, arg2)` | `__new__` is called on instance creation 9 | `__init__(self [,...])` | `instance = MyClass(arg1, arg2)` | `__init__` is called on instance creation 10 | `__cmp__(self, other)` | `self == other`, `self > other`, etc. | Called for any comparison 11 | `__pos__(self)` | `+self` | Unary plus sign 12 | `__neg__(self)` | `-self` | Unary minus sign 13 | `__invert__(self)` | `~self` | Bitwise inversion 14 | `__index__(self)` | `x[self]` | Conversion when object is used as index 15 | `__nonzero__(self)` | `bool(self)` | Boolean value of the object 16 | `__getattr__(self, name)` | `self.name # name doesn't exist` | Accessing nonexistent attribute 17 | `__setattr__(self, name, val)` | `self.name = val` | Assigning to an attribute 18 | `__delattr__(self, name)` | `del self.name` | Deleting an attribute 19 | `__getattribute__(self, name)` | `self.name` | Accessing any attribute 20 | `__getitem__(self, key)` | `self[key]` | Accessing an item using an index 21 | `__setitem__(self, key, val)` | `self[key] = val` | Assigning to an item using an index 22 | `__delitem__(self, key)` | `del self[key]` | Deleting an item using an index 23 | `__iter__(self)` | `for x in self` | Iteration 24 | `__contains__(self, value)` | `value in self`, `value not in self` | Membership tests using `in` 25 | `__call__(self [,...])` | `self(args)` | "Calling" an instance 26 | `__enter__(self)` | `with self as x:` | `with` statement context managers 27 | `__exit__(self, exc, val, trace)` | `with self as x:` | `with` statement context managers 28 | `__getstate__(self)` | `pickle.dump(pkl_file, self)` | Pickling 29 | `__setstate__(self)` | `data = pickle.load(pkl_file)` | Pickling 30 | 31 | Hopefully, this table should have cleared up any questions you might have had about what syntax invokes which magic method. 32 | 33 | ##Appendix 2: Changes in Python 3## 34 | 35 | Here, we document a few major places where Python 3 differs from 2.x in terms of its object model: 36 | 37 | - Since the distinction between string and unicode has been done away with in Python 3, `__unicode__` is gone and `__bytes__` (which behaves similarly to `__str__` and `__unicode__` in 2.7) exists for a new built-in for constructing byte arrays. 38 | - Since division defaults to true division in Python 3, `__div__` is gone in Python 3 39 | - `__coerce__` is gone due to redundancy with other magic methods and confusing behavior 40 | - `__cmp__` is gone due to redundancy with other magic methods 41 | - `__nonzero__` has been renamed to `__bool__` 42 | -------------------------------------------------------------------------------- /listings/access.py: -------------------------------------------------------------------------------- 1 | class AccessCounter(object): 2 | '''A class that contains a value and implements an 3 | access counter. The counter increments each time the 4 | value is changed.''' 5 | 6 | def __init__(self, val): 7 | super(AccessCounter, self).__setattr__('counter', 0) 8 | super(AccessCounter, self).__setattr__('value', val) 9 | 10 | def __setattr__(self, name, value): 11 | if name == 'value': 12 | super(AccessCounter, self).__setattr__('counter', 13 | self.counter + 1) 14 | # Make this unconditional. 15 | # If you want to prevent other attributes to be set, 16 | # raise AttributeError(name) 17 | super(AccessCounter, self).__setattr__(name, value) 18 | 19 | def __delattr__(self, name): 20 | if name == 'value': 21 | super(AccessCounter, self).__setattr__('counter', 22 | self.counter + 1) 23 | super(AccessCounter, self).__delattr__(name)] 24 | -------------------------------------------------------------------------------- /listings/call.py: -------------------------------------------------------------------------------- 1 | class Entity: 2 | '''Class to represent an entity. Callable to update 3 | the entity's position.''' 4 | 5 | def __init__(self, size, x, y): 6 | self.x, self.y = x, y 7 | self.size = size 8 | 9 | def __call__(self, x, y): 10 | '''Change the position of the entity.''' 11 | self.x, self.y = x, y 12 | 13 | # snip... 14 | -------------------------------------------------------------------------------- /listings/closer.py: -------------------------------------------------------------------------------- 1 | class Closer: 2 | '''A context manager to automatically close an object with a 3 | close() method in a with statement.''' 4 | 5 | def __init__(self, obj): 6 | self.obj = obj 7 | 8 | def __enter__(self): 9 | return self.obj # bound to target 10 | 11 | def __exit__(self, exception_type, exception_val, trace): 12 | try: 13 | self.obj.close() 14 | except AttributeError: # obj isn't closable 15 | print 'Not closable.' 16 | return True # exception handled successfully 17 | -------------------------------------------------------------------------------- /listings/descriptor.py: -------------------------------------------------------------------------------- 1 | class Meter(object): 2 | '''Descriptor for a meter.''' 3 | 4 | def __init__(self, value=0.0): 5 | self.value = float(value) 6 | def __get__(self, instance, owner): 7 | return self.value 8 | def __set__(self, instance, value): 9 | self.value = float(value) 10 | 11 | class Foot(object): 12 | '''Descriptor for a foot.''' 13 | 14 | def __get__(self, instance, owner): 15 | return instance.meter * 3.2808 16 | def __set__(self, instance, value): 17 | instance.meter = float(value) / 3.2808 18 | 19 | class Distance(object): 20 | '''Class to represent distance holding two descriptors for feet and 21 | meters.''' 22 | meter = Meter() 23 | foot = Foot() 24 | 25 | -------------------------------------------------------------------------------- /listings/fileobject.py: -------------------------------------------------------------------------------- 1 | from os.path import join 2 | 3 | class FileObject: 4 | '''Wrapper for file objects to make sure the file gets closed 5 | on deletion.''' 6 | 7 | def __init__(self, filepath='~', filename='sample.txt'): 8 | # open a file filename in filepath in read and write mode 9 | self.file = open(join(filepath, filename), 'r+') 10 | 11 | def __del__(self): 12 | self.file.close() 13 | del self.file 14 | -------------------------------------------------------------------------------- /listings/list.py: -------------------------------------------------------------------------------- 1 | class FunctionalList: 2 | '''A class wrapping a list with some extra functional 3 | magic, like head, tail, init, last, drop, and take.''' 4 | 5 | def __init__(self, values=None): 6 | if values is None: 7 | self.values = [] 8 | else: 9 | self.values = values 10 | 11 | def __len__(self): 12 | return len(self.values) 13 | 14 | def __getitem__(self, key): 15 | # if key is of invalid type or value, the list values 16 | # will raise the error 17 | return self.values[key] 18 | 19 | def __setitem__(self, key, value): 20 | self.values[key] = value 21 | 22 | def __delitem__(self, key): 23 | del self.values[key] 24 | 25 | def __iter__(self): 26 | return iter(self.values) 27 | 28 | def __reversed__(self): 29 | return reversed(self.values) 30 | 31 | def append(self, value): 32 | self.values.append(value) 33 | def head(self): 34 | # get the first element 35 | return self.values[0] 36 | def tail(self): 37 | # get all elements after the first 38 | return self.values[1:] 39 | def init(self): 40 | # get elements up to the last 41 | return self.values[:-1] 42 | def last(self): 43 | # get last element 44 | return self.values[-1] 45 | def drop(self, n): 46 | # get all elements except first n 47 | return self.values[n:] 48 | def take(self, n): 49 | # get first n elements 50 | return self.values[:n] 51 | -------------------------------------------------------------------------------- /listings/slate.py: -------------------------------------------------------------------------------- 1 | import time 2 | 3 | class Slate: 4 | '''Class to store a string and a changelog, and forget its value when 5 | pickled.''' 6 | 7 | def __init__(self, value): 8 | self.value = value 9 | self.last_change = time.asctime() 10 | self.history = {} 11 | 12 | def change(self, new_value): 13 | # Change the value. Commit last value to history 14 | self.history[self.last_change] = self.value 15 | self.value = new_value 16 | self.last_change = time.asctime() 17 | 18 | def print_changes(self): 19 | print 'Changelog for Slate object:' 20 | for k, v in self.history.items(): 21 | print '%s\t %s' % (k, v) 22 | 23 | def __getstate__(self): 24 | # Deliberately do not return self.value or self.last_change. 25 | # We want to have a "blank slate" when we unpickle. 26 | return self.history 27 | 28 | def __setstate__(self, state): 29 | # Make self.history = state and last_change and value undefined 30 | self.history = state 31 | self.value, self.last_change = None, None 32 | -------------------------------------------------------------------------------- /listings/word.py: -------------------------------------------------------------------------------- 1 | class Word(str): 2 | '''Class for words, defining comparison based on word length.''' 3 | 4 | def __new__(cls, word): 5 | # Note that we have to use __new__. This is because str is an 6 | # immutable type, so we have to initialize it early (at creation) 7 | if ' ' in word: 8 | print "Value contains spaces. Truncating to first space." 9 | word = word[:word.index(' ')] 10 | # Word is now all chars before first space 11 | return str.__new__(cls, word) 12 | 13 | def __gt__(self, other): 14 | return len(self) > len(other) 15 | def __lt__(self, other): 16 | return len(self) < len(other) 17 | def __ge__(self, other): 18 | return len(self) >= len(other) 19 | def __le__(self, other): 20 | return len(self) <= len(other) 21 | -------------------------------------------------------------------------------- /magicmarkdown.py: -------------------------------------------------------------------------------- 1 | """ 2 | magicmarkdown.py 3 | utility script for changing markdown from magic methods guide into HTML 4 | """ 5 | 6 | import markdown 7 | 8 | HEADER = """ 9 | 10 | 11 | A Guide to Python's Magic Methods « rafekettler.com 12 | 13 | 14 | 15 | 16 | """ 17 | 18 | FOOTER = """ 30 | 31 | 32 | """ 33 | 34 | table = open('table.markdown').read() 35 | body = open('magicmethods.markdown').read() 36 | appendix = open('appendix.markdown').read() 37 | 38 | table_text = markdown.markdown(table) 39 | body_text = markdown.markdown(body, 40 | ['def_list', 'codehilite']) 41 | appendix_text = markdown.markdown(appendix, ['tables']) 42 | 43 | with open('magicmethods.html', 'w') as out: 44 | out.write(HEADER) 45 | out.write(table_text) 46 | out.write(body_text) 47 | out.write(appendix_text) 48 | out.write(FOOTER) 49 | 50 | -------------------------------------------------------------------------------- /magicmethods.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | A Guide to Python's Magic Methods « rafekettler.com 5 | 6 | 7 | 8 | 9 |

A Guide to Python's Magic Methods

10 |

Rafe Kettler

11 |

Copyright © 2012 Rafe Kettler

12 |

Version 1.17

13 |

A PDF version of this guide can be obtained from my site or Github. The magic methods guide has a git repository at http://www.github.com/RafeKettler/magicmethods. Any issues can be reported 14 | there, along with comments, (or even contributions!).

15 |

Table of Contents

16 |
    17 |
  1. Introduction
  2. 18 |
  3. Construction and Initialization
  4. 19 |
  5. Making Operators Work on Custom Classes 23 |
  6. 24 |
  7. Representing your Classes
  8. 25 |
  9. Controlling Attribute Access
  10. 26 |
  11. Making Custom Sequences
  12. 27 |
  13. Reflection
  14. 28 |
  15. Abstract Base Classes
  16. 29 |
  17. Callable Objects
  18. 30 |
  19. Context Managers
  20. 31 |
  21. Building Descriptor Objects
  22. 32 |
  23. Copying
  24. 33 |
  25. Pickling your Objects
  26. 34 |
  27. Conclusion
  28. 35 |
  29. Appendix 1: How to Call Magic Methods
  30. 36 |
  31. Appendix 2: Changes in Python 3
  32. 37 |

Introduction

38 |

This guide is the culmination of a few months' worth of blog posts. The subject is magic methods.

39 |

What are magic methods? They're everything in object-oriented Python. They're special methods that you can define to add "magic" to your classes. They're always surrounded by double underscores (e.g. __init__ or __lt__). They're also not as well documented as they need to be. All of the magic methods for Python appear in the same section in the Python docs, but they're scattered about and only loosely organized. There's hardly an example to be found in that section (and that may very well be by design, since they're all detailed in the language reference, along with boring syntax descriptions, etc.).

40 |

So, to fix what I perceived as a flaw in Python's documentation, I set out to provide some more plain-English, example-driven documentation for Python's magic methods. I started out with weekly blog posts, and now that I've finished with those, I've put together this guide.

41 |

I hope you enjoy it. Use it as a tutorial, a refresher, or a reference; it's just intended to be a user-friendly guide to Python's magic methods.

42 |

Construction and Initialization

43 |

Everyone knows the most basic magic method, __init__. It's the way that we can define the initialization behavior of an object. However, when I call x = SomeClass(), __init__ is not the first thing to get called. Actually, it's a method called __new__, which actually creates the instance, then passes any arguments at creation on to the initializer. At the other end of the object's lifespan, there's __del__. Let's take a closer look at these 3 magic methods:

44 |
45 |
__new__(cls, [...)
46 |
__new__ is the first method to get called in an object's instantiation. It takes the class, then any other arguments that it will pass along to __init__. __new__ is used fairly rarely, but it does have its purposes, particularly when subclassing an immutable type like a tuple or a string. I don't want to go in to too much detail on __new__ because it's not too useful, but it is covered in great detail in the Python docs.
47 |
__init__(self, [...)
48 |
The initializer for the class. It gets passed whatever the primary constructor was called with (so, for example, if we called x = SomeClass(10, 'foo'), __init__ would get passed 10 and 'foo' as arguments. __init__ is almost universally used in Python class definitions.
49 |
__del__(self)
50 |
If __new__ and __init__ formed the constructor of the object, __del__ is the destructor. It doesn't implement behavior for the statement del x (so that code would not translate to x.__del__()). Rather, it defines behavior for when an object is garbage collected. It can be quite useful for objects that might require extra cleanup upon deletion, like sockets or file objects. Be careful, however, as there is no guarantee that __del__ will be executed if the object is still alive when the interpreter exits, so __del__ can't serve as a replacement for good coding practices (like always closing a connection when you're done with it. In fact, __del__ should almost never be used because of the precarious circumstances under which it is called; use it with caution!
51 |
52 |

Putting it all together, here's an example of __init__ and __del__ in action:

53 |
from os.path import join
 54 | 
 55 | class FileObject:
 56 |     '''Wrapper for file objects to make sure the file gets closed on deletion.'''
 57 | 
 58 |     def __init__(self, filepath='~', filename='sample.txt'):
 59 |         # open a file filename in filepath in read and write mode
 60 |         self.file = open(join(filepath, filename), 'r+')
 61 | 
 62 |     def __del__(self):
 63 |         self.file.close()
 64 |         del self.file
 65 | 
66 | 67 | 68 |

Making Operators Work on Custom Classes

69 |

One of the biggest advantages of using Python's magic methods is that they provide a simple way to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, and nonstandard ways of performing basic operators. In some languages, it's common to do something like this:

70 |
if instance.equals(other_instance):
 71 |     # do something
 72 | 
73 | 74 | 75 |

You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. Different libraries might use different names for the same operations, making the client do way more work than necessary. With the power of magic methods, however, we can define one method (__eq__, in this case), and say what we mean instead:

76 |
if instance == other_instance:
 77 |     #do something
 78 | 
79 | 80 | 81 |

That's part of the power of magic methods. The vast majority of them allow us to define meaning for operators so that we can use them on our own classes just like they were built in types.

82 |

Comparison magic methods

83 |

Python has a whole slew of magic methods designed to implement intuitive comparisons between objects using operators, not awkward method calls. They also provide a way to override the default Python behavior for comparisons of objects (by reference). Here's the list of those methods and what they do:

84 |
85 |
__cmp__(self, other)
86 |
__cmp__ is the most basic of the comparison magic methods. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.), but it might not do it the way you want (for example, if whether one instance was equal to another were determined by one criterion and and whether an instance is greater than another were determined by something else). __cmp__ should return a negative integer if self < other, zero if self == other, and positive if self > other. It's usually best to define each comparison you need rather than define them all at once, but __cmp__ can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria.
87 |
__eq__(self, other)
88 |
Defines behavior for the equality operator, ==.
89 |
__ne__(self, other)
90 |
Defines behavior for the inequality operator, !=.
91 |
__lt__(self, other)
92 |
Defines behavior for the less-than operator, <.
93 |
__gt__(self, other)
94 |
Defines behavior for the greater-than operator, >.
95 |
__le__(self, other)
96 |
Defines behavior for the less-than-or-equal-to operator, <=.
97 |
__ge__(self, other)
98 |
Defines behavior for the greater-than-or-equal-to operator, >=.
99 |
100 |

For an example, consider a class to model a word. We might want to compare words lexicographically (by the alphabet), which is the default comparison behavior for strings, but we also might want to do it based on some other criterion, like length or number of syllables. In this example, we'll compare by length. Here's an implementation:

101 |
class Word(str):
102 |     '''Class for words, defining comparison based on word length.'''
103 | 
104 |     def __new__(cls, word):
105 |         # Note that we have to use __new__. This is because str is an immutable
106 |         # type, so we have to initialize it early (at creation)
107 |         if ' ' in word:
108 |             print "Value contains spaces. Truncating to first space."
109 |             word = word[:word.index(' ')] # Word is now all chars before first space
110 |         return str.__new__(cls, word)
111 | 
112 |     def __gt__(self, other):
113 |         return len(self) > len(other)
114 |     def __lt__(self, other):
115 |         return len(self) < len(other)
116 |     def __ge__(self, other):
117 |         return len(self) >= len(other)
118 |     def __le__(self, other):
119 |         return len(self) <= len(other)
120 | 
121 | 122 | 123 |

Now, we can create two Words (by using Word('foo') and Word('bar')) and compare them based on length. Note, however, that we didn't define __eq__ and __ne__. This is because this would lead to some weird behavior (notably that Word('foo') == Word('bar') would evaluate to true). It wouldn't make sense to test for equality based on length, so we fall back on str's implementation of equality.

124 |

Now would be a good time to note that you don't have to define every comparison magic method to get rich comparisons. The standard library has kindly provided us with a class decorator in the module functools that will define all rich comparison methods if you only define __eq__ and one other (e.g. __gt__, __lt__, etc.) This feature is only available in Python 2.7, but when you get a chance it saves a great deal of time and effort. You can use it by placing @total_ordering above your class definition.

125 |

Numeric magic methods

126 |

Just like you can create ways for instances of your class to be compared with comparison operators, you can define behavior for numeric operators. Buckle your seat belts, folks, there's a lot of these. For organization's sake, I've split the numeric magic methods into 5 categories: unary operators, normal arithmetic operators, reflected arithmetic operators (more on this later), augmented assignment, and type conversions.

127 |

Unary operators and functions

128 |

Unary operators and functions only have one operand, e.g. negation, absolute value, etc.

129 |
130 |
__pos__(self)
131 |
Implements behavior for unary positive (e.g. +some_object)
132 |
__neg__(self)
133 |
Implements behavior for negation (e.g. -some_object)
134 |
__abs__(self)
135 |
Implements behavior for the built in abs() function.
136 |
__invert__(self)
137 |
Implements behavior for inversion using the ~ operator. For an explanation on what this does, see the Wikipedia article on bitwise operations.
138 |
__round__(self, n)
139 |
Implements behavior for the buil in round() function. n is the number of decimal places to round to.
140 |
__floor__(self)
141 |
Implements behavior for math.floor(), i.e., rounding down to the nearest integer.
142 |
__ceil__(self)
143 |
Implements behavior for math.ceil(), i.e., rounding up to the nearest integer.
144 |
__trunc__(self)
145 |
Implements behavior for math.trunc(), i.e., truncating to an integral.
146 |
147 |

Normal arithmetic operators

148 |

Now, we cover the typical binary operators (and a function or two): +, -, * and the like. These are, for the most part, pretty self-explanatory.

149 |
150 |
__add__(self, other)
151 |
Implements addition.
152 |
__sub__(self, other)
153 |
Implements subtraction.
154 |
__mul__(self, other)
155 |
Implements multiplication.
156 |
__floordiv__(self, other)
157 |
Implements integer division using the // operator.
158 |
__div__(self, other)
159 |
Implements division using the / operator.
160 |
__truediv__(self, other)
161 |
Implements true division. Note that this only works when from __future__ import division is in effect.
162 |
__mod__(self, other)
163 |
Implements modulo using the % operator.
164 |
__divmod__(self, other)
165 |
Implements behavior for long division using the divmod() built in function.
166 |
__pow__
167 |
Implements behavior for exponents using the ** operator.
168 |
__lshift__(self, other)
169 |
Implements left bitwise shift using the << operator.
170 |
__rshift__(self, other)
171 |
Implements right bitwise shift using the >> operator.
172 |
__and__(self, other)
173 |
Implements bitwise and using the & operator.
174 |
__or__(self, other)
175 |
Implements bitwise or using the | operator.
176 |
__xor__(self, other)
177 |
Implements bitwise xor using the ^ operator.
178 |
179 |

Reflected arithmetic operators

180 |

You know how I said I would get to reflected arithmetic in a bit? Some of you might think it's some big, scary, foreign concept. It's actually quite simple. Here's an example:

181 |
some_object + other
182 | 
183 | 184 | 185 |

That was "normal" addition. The reflected equivalent is the same thing, except with the operands switched around:

186 |
other + some_object
187 | 
188 | 189 | 190 |

So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around. In most cases, the result of a reflected operation is the same as its normal equivalent, so you may just end up defining __radd__ as calling __add__ and so on. Note that the object on the left hand side of the operator (other in the example) must not define (or return NotImplemented) for its definition of the non-reflected version of an operation. For instance, in the example, some_object.__radd__ will only be called if other does not define __add__.

191 |
192 |
__radd__(self, other)
193 |
Implements reflected addition.
194 |
__rsub__(self, other)
195 |
Implements reflected subtraction.
196 |
__rmul__(self, other)
197 |
Implements reflected multiplication.
198 |
__rfloordiv__(self, other)
199 |
Implements reflected integer division using the // operator.
200 |
__rdiv__(self, other)
201 |
Implements reflected division using the / operator.
202 |
__rtruediv__(self, other)
203 |
Implements reflected true division. Note that this only works when from __future__ import division is in effect.
204 |
__rmod__(self, other)
205 |
Implements reflected modulo using the % operator.
206 |
__rdivmod__(self, other)
207 |
Implements behavior for long division using the divmod() built in function, when divmod(other, self) is called.
208 |
__rpow__
209 |
Implements behavior for reflected exponents using the ** operator.
210 |
__rlshift__(self, other)
211 |
Implements reflected left bitwise shift using the << operator.
212 |
__rrshift__(self, other)
213 |
Implements reflected right bitwise shift using the >> operator.
214 |
__rand__(self, other)
215 |
Implements reflected bitwise and using the & operator.
216 |
__ror__(self, other)
217 |
Implements reflected bitwise or using the | operator.
218 |
__rxor__(self, other)
219 |
Implements reflected bitwise xor using the ^ operator.
220 |
221 |

Augmented assignment

222 |

Python also has a wide variety of magic methods to allow custom behavior to be defined for augmented assignment. You're probably already familiar with augmented assignment, it combines "normal" operators with assignment. If you still don't know what I'm talking about, here's an example:

223 |
x = 5
224 | x += 1 # in other words x = x + 1
225 | 
226 | 227 | 228 |

Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for a += b, __iadd__ might return a + b, which would be assigned to a). Here's the list:

229 |
230 |
__iadd__(self, other)
231 |
Implements addition with assignment.
232 |
__isub__(self, other)
233 |
Implements subtraction with assignment.
234 |
__imul__(self, other)
235 |
Implements multiplication with assignment.
236 |
__ifloordiv__(self, other)
237 |
Implements integer division with assignment using the //= operator.
238 |
__idiv__(self, other)
239 |
Implements division with assignment using the /= operator.
240 |
__itruediv__(self, other)
241 |
Implements true division with assignment. Note that this only works when from __future__ import division is in effect.
242 |
__imod_(self, other)
243 |
Implements modulo with assignment using the %= operator.
244 |
__ipow__
245 |
Implements behavior for exponents with assignment using the **= operator.
246 |
__ilshift__(self, other)
247 |
Implements left bitwise shift with assignment using the <<= operator.
248 |
__irshift__(self, other)
249 |
Implements right bitwise shift with assignment using the >>= operator.
250 |
__iand__(self, other)
251 |
Implements bitwise and with assignment using the &= operator.
252 |
__ior__(self, other)
253 |
Implements bitwise or with assignment using the |= operator.
254 |
__ixor__(self, other)
255 |
Implements bitwise xor with assignment using the ^= operator.
256 |
257 |

Type conversion magic methods

258 |

Python also has an array of magic methods designed to implement behavior for built in type conversion functions like float(). Here they are:

259 |
260 |
__int__(self)
261 |
Implements type conversion to int.
262 |
__long__(self)
263 |
Implements type conversion to long.
264 |
__float__(self)
265 |
Implements type conversion to float.
266 |
__complex__(self)
267 |
Implements type conversion to complex.
268 |
__oct__(self)
269 |
Implements type conversion to octal.
270 |
__hex__(self)
271 |
Implements type conversion to hexadecimal.
272 |
__index__(self)
273 |
Implements type conversion to an int when the object is used in a slice expression. If you define a custom numeric type that might be used in slicing, you should define __index__.
274 |
__trunc__(self)
275 |
Called when math.trunc(self) is called. __trunc__ should return the value of `self truncated to an integral type (usually a long).
276 |
__coerce__(self, other)
277 |
Method to implement mixed mode arithmetic. __coerce__ should return None if type conversion is impossible. Otherwise, it should return a pair (2-tuple) of self and other, manipulated to have the same type.
278 |
279 |

Representing your Classes

280 |

It's often useful to have a string representation of a class. In Python, there's a few methods that you can implement in your class definition to customize how built in functions that return representations of your class behave.

281 |
282 |
__str__(self)
283 |
Defines behavior for when str() is called on an instance of your class.
284 |
__repr__(self)
285 |
Defines behavior for when repr() is called on an instance of your class. The major difference between str() and repr() is intended audience. repr() is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereas str() is intended to be human-readable.
286 |
__unicode__(self)
287 |
Defines behavior for when unicode() is called on an instance of your class. unicode() is like str(), but it returns a unicode string. Be wary: if a client calls str() on an instance of your class and you've only defined __unicode__(), it won't work. You should always try to define __str__() as well in case someone doesn't have the luxury of using unicode.
288 |
__format__(self, formatstr)
289 |
Defines behavior for when an instance of your class is used in new-style string formatting. For instance, "Hello, {0:abc}!".format(a) would lead to the call a.__format__("abc"). This can be useful for defining your own numerical or string types that you might like to give special formatting options.
290 |
__hash__(self)
291 |
Defines behavior for when hash() is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing __eq__ as well. Live by the following rule: a == b implies hash(a) == hash(b).
292 |
__nonzero__(self)
293 |
Defines behavior for when bool() is called on an instance of your class. Should return True or False, depending on whether you would want to consider the instance to be True or False.
294 |
__dir__(self)
295 |
Defines behavior for when dir() is called on an instance of your class. This method should return a list of attributes for the user. Typically, implementing __dir__ is unnecessary, but it can be vitally important for interactive use of your classes if you redefine __getattr__ or __getattribute__ (which you will see in the next section) or are otherwise dynamically generating attributes.
296 |
__sizeof__(self)
297 |
Defines behavior for when sys.getsizeof() is called on an instance of your class. This should return the size of your object, in bytes. This is generally more useful for Python classes implemented in C extensions, but it helps to be aware of it.
298 |
299 |

We're pretty much done with the boring (and example-free) part of the magic methods guide. Now that we've covered some of the more basic magic methods, it's time to move to more advanced material.

300 |

Controlling Attribute Access

301 |

Many people coming to Python from other languages complain that it lacks true encapsulation for classes (e.g. no way to define private attributes and then have public getter and setters). This couldn't be farther than the truth: it just happens that Python accomplishes a great deal of encapsulation through "magic", instead of explicit modifiers for methods or fields. Take a look:

302 |
303 |
__getattr__(self, name)
304 |
You can define behavior for when a user attempts to access an attribute that doesn't exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing an AttributeError. It only gets called when a nonexistent attribute is accessed, however, so it isn't a true encapsulation solution.
305 |
__setattr__(self, name, value)
306 |
Unlike __getattr__, __setattr__ is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes. However, you have to be careful with how you use __setattr__, as the example at the end of the list will show.
307 |
__delattr__
308 |
This is the exact same as __setattr__, but for deleting attributes instead of setting them. The same precautions need to be taken as with __setattr__ as well in order to prevent infinite recursion (calling del self.name in the implementation of __delattr__ would cause infinite recursion).
309 |
__getattribute__(self, name)
310 |
After all this, __getattribute__ fits in pretty well with its companions __setattr__ and __delattr__. However, I don't recommend you use it. __getattribute__ can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassing object. It allows you to define rules for whenever an attribute's value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class's __getattribute__ method to prevent this). It also mainly obviates the need for __getattr__, which only gets called when __getattribute__ is implemented if it is called explicitly or an AttributeError is raised. This method can be used (after all, it's your choice), but I don't recommend it because it has a small use case (it's far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free.
311 |
312 |

You can easily cause a problem in your definitions of any of the methods controlling attribute access. Consider this example:

313 |
def __setattr__(self, name, value):
314 |     self.name = value
315 |     # since every time an attribute is assigned, __setattr__() is called, this
316 |     # is recursion.
317 |     # so this really means self.__setattr__('name', value). Since the method
318 |     # keeps calling itself, the recursion goes on forever causing a crash
319 | 
320 | def __setattr__(self, name, value):
321 |     self.__dict__[name] = value # assigning to the dict of names in the class
322 |     # define custom behavior here
323 | 
324 | 325 | 326 |

Again, Python's magic methods are incredibly powerful, and with great power comes great responsibility. It's important to know the proper way to use magic methods so you don't break any code.

327 |

So, what have we learned about custom attribute access in Python? It's not to be used lightly. In fact, it tends to be excessively powerful and counter-intuitive. But the reason why it exists is to scratch a certain itch: Python doesn't seek to make bad things impossible, but just to make them difficult. Freedom is paramount, so you can really do whatever you want. Here's an example of some of the special attribute access methods in action (note that we use super because not all classes have an attribute __dict__):

328 |
class AccessCounter(object):
329 |     '''A class that contains a value and implements an access counter.
330 |     The counter increments each time the value is changed.'''
331 | 
332 |     def __init__(self, val):
333 |         super(AccessCounter, self).__setattr__('counter', 0)
334 |         super(AccessCounter, self).__setattr__('value', val)
335 | 
336 |     def __setattr__(self, name, value):
337 |         if name == 'value':
338 |             super(AccessCounter, self).__setattr__('counter', self.counter + 1)
339 |         # Make this unconditional.
340 |         # If you want to prevent other attributes to be set, raise AttributeError(name)
341 |         super(AccessCounter, self).__setattr__(name, value)
342 | 
343 |     def __delattr__(self, name):
344 |         if name == 'value':
345 |             super(AccessCounter, self).__setattr__('counter', self.counter + 1)
346 |         super(AccessCounter, self).__delattr__(name)]
347 | 
348 | 349 | 350 |

Making Custom Sequences

351 |

There's a number of ways to get your Python classes to act like built in sequences (dict, tuple, list, string, etc.). These are by far my favorite magic methods in Python because of the absurd degree of control they give you and the way that they magically make a whole array of global functions work beautifully on instances of your class. But before we get down to the good stuff, a quick word on requirements.

352 |

Requirements

353 |

Now that we're talking about creating your own sequences in Python, it's time to talk about protocols. Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they're more like guidelines.

354 |

Why are we talking about protocols now? Because implementing custom container types in Python involves using some of these protocols. First, there's the protocol for defining immutable containers: to make an immutable container, you need only define __len__ and __getitem__ (more on these later). The mutable container protocol requires everything that immutable containers require plus __setitem__ and __delitem__. Lastly, if you want your object to be iterable, you'll have to define __iter__, which returns an iterator. That iterator must conform to an iterator protocol, which requires iterators to have methods called __iter__(returning itself) and next.

355 |

The magic behind containers

356 |

Without any more wait, here are the magic methods that containers use:

357 |
358 |
__len__(self)
359 |
Returns the length of the container. Part of the protocol for both immutable and mutable containers.
360 |
__getitem__(self, key)
361 |
Defines behavior for when an item is accessed, using the notation self[key]. This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions: TypeError if the type of the key is wrong and KeyError if there is no corresponding value for the key.
362 |
__setitem__(self, key, value)
363 |
Defines behavior for when an item is assigned to, using the notation self[nkey] = value. This is part of the mutable container protocol. Again, you should raise KeyError and TypeError where appropriate.
364 |
__delitem__(self, key)
365 |
Defines behavior for when an item is deleted (e.g. del self[key]). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used.
366 |
__iter__(self)
367 |
Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by the iter() built in function and when a container is looped over using the form for x in container:. Iterators are their own objects, and they also must define an __iter__ method that returns self.
368 |
__reversed__(self)
369 |
Called to implement behavior for the reversed() built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple.
370 |
__contains__(self, item)
371 |
__contains__ defines behavior for membership tests using in and not in. Why isn't this part of a sequence protocol, you ask? Because when __contains__ isn't defined, Python just iterates over the sequence and returns True if it comes across the item it's looking for.
372 |
__missing__(self, key)
373 |
__missing__ is used in subclasses of dict. It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionary d and said d["george"] when "george" is not a key in the dict, d.__missing__("george") would be called).
374 |
375 |

An example

376 |

For our example, let's look at a list that implements some functional constructs that you might be used to from other languages (Haskell, for example).

377 |
class FunctionalList:
378 |     '''A class wrapping a list with some extra functional magic, like head,
379 |     tail, init, last, drop, and take.'''
380 | 
381 |     def __init__(self, values=None):
382 |         if values is None:
383 |             self.values = []
384 |         else:
385 |             self.values = values
386 | 
387 |     def __len__(self):
388 |         return len(self.values)
389 | 
390 |     def __getitem__(self, key):
391 |         # if key is of invalid type or value, the list values will raise the error
392 |         return self.values[key]
393 | 
394 |     def __setitem__(self, key, value):
395 |         self.values[key] = value
396 | 
397 |     def __delitem__(self, key):
398 |         del self.values[key]
399 | 
400 |     def __iter__(self):
401 |         return iter(self.values)
402 | 
403 |     def __reversed__(self):
404 |         return FunctionalList(reversed(self.values))
405 | 
406 |     def append(self, value):
407 |         self.values.append(value)
408 |     def head(self):
409 |         # get the first element
410 |         return self.values[0]
411 |     def tail(self):
412 |         # get all elements after the first
413 |         return self.values[1:]
414 |     def init(self):
415 |         # get elements up to the last
416 |         return self.values[:-1]
417 |     def last(self):
418 |         # get last element
419 |         return self.values[-1]
420 |     def drop(self, n):
421 |         # get all elements except first n
422 |         return self.values[n:]
423 |     def take(self, n):
424 |         # get first n elements
425 |         return self.values[:n]
426 | 
427 | 428 | 429 |

There you have it, a (marginally) useful example of how to implement your own sequence. Of course, there are more useful applications of custom sequences, but quite a few of them are already implemented in the standard library (batteries included, right?), like Counter, OrderedDict, and NamedTuple.

430 |

Reflection

431 |

You can also control how reflection using the built in functions isinstance() and issubclass()behaves by defining magic methods. The magic methods are:

432 |
433 |
__instancecheck__(self, instance)
434 |
Checks if an instance is an instance of the class you defined (e.g. isinstance(instance, class).
435 |
__subclasscheck__(self, subclass)
436 |
Checks if a class subclasses the class you defined (e.g. issubclass(subclass, class)).
437 |
438 |

The use case for these magic methods might seem small, and that may very well be true. I won't spend too much more time on reflection magic methods because they aren't very important, but they reflect something important about object-oriented programming in Python and Python in general: there is almost always an easy way to do something, even if it's rarely necessary. These magic methods might not seem useful, but if you ever need them you'll be glad that they're there (and that you read this guide!).

439 |

Callable Objects

440 |

As you may already know, in Python, functions are first-class objects. This means that they can be passed to functions and methods just as if they were objects of any other kind. This is an incredibly powerful feature.

441 |

A special magic method in Python allows instances of your classes to behave as if they were functions, so that you can "call" them, pass them to functions that take functions as arguments, and so on. This is another powerful convenience feature that makes programming in Python that much sweeter.

442 |
443 |
__call__(self, [args...])
444 |
Allows an instance of a class to be called as a function. Essentially, this means that x() is the same as x.__call__(). Note that __call__ takes a variable number of arguments; this means that you define __call__ as you would any other function, taking however many arguments you'd like it to.
445 |
446 |

__call__ can be particularly useful in classes whose instances that need to often change state. "Calling" the instance can be an intuitive and elegant way to change the object's state. An example might be a class representing an entity's position on a plane:

447 |
class Entity:
448 |     '''Class to represent an entity. Callable to update the entity's position.'''
449 | 
450 |     def __init__(self, size, x, y):
451 |         self.x, self.y = x, y
452 |         self.size = size
453 | 
454 |     def __call__(self, x, y):
455 |         '''Change the position of the entity.'''
456 |         self.x, self.y = x, y
457 | 
458 |     # snip...
459 | 
460 | 461 | 462 |

Context Managers

463 |

In Python 2.5, a new keyword was introduced in Python along with a new method for code reuse, the with statement. The concept of context managers was hardly new in Python (it was implemented before as a part of the library), but not until PEP 343 was accepted did it achieve status as a first class language construct. You may have seen with statements before:

464 |
with open('foo.txt') as bar:
465 |     # perform some action with bar
466 | 
467 | 468 | 469 |

Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a with statement. The behavior of the context manager is determined by two magic methods:

470 |
471 |
__enter__(self)
472 |
Defines what the context manager should do at the beginning of the block created by the with statement. Note that the return value of __enter__ is bound to the target of the with statement, or the name after the as.
473 |
__exit__(self, exception_type, exception_value, traceback)
474 |
Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, exception_type, exception_value, and traceback will be None. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure __exit__ returns True after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen.
475 |
476 |

__enter__ and __exit__ can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects. Here's an example:

477 |
class Closer:
478 |     '''A context manager to automatically close an object with a close method
479 |     in a with statement.'''
480 | 
481 |     def __init__(self, obj):
482 |         self.obj = obj
483 | 
484 |     def __enter__(self):
485 |         return self.obj # bound to target
486 | 
487 |     def __exit__(self, exception_type, exception_val, trace):
488 |         try:
489 |            self.obj.close()
490 |         except AttributeError: # obj isn't closable
491 |            print 'Not closable.'
492 |            return True # exception handled successfully
493 | 
494 | 495 | 496 |

Here's an example of Closer in action, using an FTP connection to demonstrate it (a closable socket):

497 |
>>> from magicmethods import Closer
498 | >>> from ftplib import FTP
499 | >>> with Closer(FTP('ftp.somesite.com')) as conn:
500 | ...     conn.dir()
501 | ...
502 | # output omitted for brevity
503 | >>> conn.dir()
504 | # long AttributeError message, can't use a connection that's closed
505 | >>> with Closer(int(5)) as i:
506 | ...     i += 1
507 | ...
508 | Not closable.
509 | >>> i
510 | 6
511 | 
512 | 513 | 514 |

See how our wrapper gracefully handled both proper and improper uses? That's the power of context managers and magic methods. Note that the Python standard library includes a module contextlib that contains a context manager, contextlib.closing(), that does approximately the same thing (without any handling of the case where an object does not have a close() method).

515 |

Abstract Base Classes

516 |

See http://docs.python.org/2/library/abc.html.

517 |

Building Descriptor Objects

518 |

Descriptors are classes which, when accessed through either getting, setting, or deleting, can also alter other objects. Descriptors aren't meant to stand alone; rather, they're meant to be held by an owner class. Descriptors can be useful when building object-oriented databases or classes that have attributes whose values are dependent on each other. Descriptors are particularly useful when representing attributes in several different units of measurement or representing computed attributes (like distance from the origin in a class to represent a point on a grid).

519 |

To be a descriptor, a class must have at least one of __get__, __set__, and __delete__ implemented. Let's take a look at those magic methods:

520 |
521 |
__get__(self, instance, owner)
522 |
Define behavior for when the descriptor's value is retrieved. instance is the instance of the owner object. owner is the owner class itself.
523 |
__set__(self, instance, value)
524 |
Define behavior for when the descriptor's value is changed. instance is the instance of the owner class and value is the value to set the descriptor to.
525 |
__delete__(self, instance)
526 |
Define behavior for when the descriptor's value is deleted. instance is the instance of the owner object.
527 |
528 |

Now, an example of a useful application of descriptors: unit conversions.

529 |
class Meter(object):
530 |     '''Descriptor for a meter.'''
531 | 
532 |     def __init__(self, value=0.0):
533 |         self.value = float(value)
534 |     def __get__(self, instance, owner):
535 |         return self.value
536 |     def __set__(self, instance, value):
537 |         self.value = float(value)
538 | 
539 | class Foot(object):
540 |     '''Descriptor for a foot.'''
541 | 
542 |     def __get__(self, instance, owner):
543 |         return instance.meter * 3.2808
544 |     def __set__(self, instance, value):
545 |         instance.meter = float(value) / 3.2808
546 | 
547 | class Distance(object):
548 |     '''Class to represent distance holding two descriptors for feet and
549 |     meters.'''
550 |     meter = Meter()
551 |     foot = Foot()
552 | 
553 | 554 | 555 |

Copying

556 |

Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python's copy comes into play. However (fortunately), Python modules are not sentient, so we don't have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things.

557 |
558 |
__copy__(self)
559 |
Defines behavior for copy.copy() for instances of your class. copy.copy() returns a shallow copy of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original).
560 |
__deepcopy__(self, memodict={})
561 |
Defines behavior for copy.deepcopy() for instances of your class. copy.deepcopy() returns a deep copy of your object -- the object and its data are both copied. memodict is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, call copy.deepcopy() on that attribute with memodict as the first argument.
562 |
563 |

What are some use cases for these magic methods? As always, in any case where you need more fine-grained control than what the default behavior gives you. For instance, if you are attempting to copy an object that stores a cache as a dictionary (which might be large), it might not make sense to copy the cache as well -- if the cache can be shared in memory between instances, then it should be.

564 |

Pickling Your Objects

565 |

If you spend time with other Pythonistas, chances are you've at least heard of pickling. Pickling is a serialization process for Python data structures, and can be incredibly useful when you need to store an object and retrieve it later (usually for caching). It's also a major source of worries and confusion.

566 |

Pickling is so important that it doesn't just have its own module (pickle), but its own protocol and the magic methods to go with it. But first, a brief word on how to pickle existing types(feel free to skip it if you already know).

567 |

Pickling: A Quick Soak in the Brine

568 |

Let's dive into pickling. Say you have a dictionary that you want to store and retrieve later. You couldwrite it's contents to a file, carefully making sure that you write correct syntax, then retrieve it using either exec() or processing the file input. But this is precarious at best: if you store important data in plain text, it could be corrupted or changed in any number of ways to make your program crash or worse run malicious code on your computer. Instead, we're going to pickle it:

569 |
import pickle
570 | 
571 | data = {'foo': [1, 2, 3],
572 |         'bar': ('Hello', 'world!'),
573 |         'baz': True}
574 | jar = open('data.pkl', 'wb')
575 | pickle.dump(data, jar) # write the pickled data to the file jar
576 | jar.close()
577 | 
578 | 579 | 580 |

Now, a few hours later, we want it back. All we have to do is unpickle it:

581 |
import pickle
582 | 
583 | pkl_file = open('data.pkl', 'rb') # connect to the pickled data
584 | data = pickle.load(pkl_file) # load it into a variable
585 | print data
586 | pkl_file.close()
587 | 
588 | 589 | 590 |

What happens? Exactly what you expect. It's just like we had data all along.

591 |

Now, for a word of caution: pickling is not perfect. Pickle files are easily corrupted on accident and on purpose. Pickling may be more secure than using flat text files, but it still can be used to run malicious code. It's also incompatible across versions of Python, so don't expect to distribute pickled objects and expect people to be able to open them. However, it can also be a powerful tool for caching and other common serialization tasks.

592 |

Pickling your own Objects

593 |

Pickling isn't just for built-in types. It's for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it's a bit different for C extensions, but that's not in our scope):

594 |
595 |
__getinitargs__(self)
596 |
If you'd like for __init__ to be called when your class is unpickled, you can define __getinitargs__, which should return a tuple of the arguments that you'd like to be passed to __init__. Note that this method will only work for old-style classes.
597 |
__getnewargs__(self)
598 |
For new-style classes, you can influence what arguments get passed to __new__ upon unpickling. This method should also return a tuple of arguments that will then be passed to __new__.
599 |
__getstate__(self)
600 |
Instead of the object's __dict__ attribute being stored, you can return a custom state to be stored when the object is pickled. That state will be used by __setstate__ when the object is unpickled.
601 |
__setstate__(self, state)
602 |
When the object is unpickled, if __setstate__ is defined the object's state will be passed to it instead of directly applied to the object's __dict__. This goes hand in hand with __getstate__: when both are defined, you can represent the object's pickled state however you want with whatever you want.
603 |
__reduce__(self)
604 |
When defining extension types (i.e., types implemented using Python's C API), you have to tell Python how to pickle them if you want them to pickle them. __reduce__() is called when an object defining it is pickled. It can either return a string representing a global name that Python will look up and pickle, or a tuple. The tuple contains between 2 and 5 elements: a callable object that is called to recreate the object, a tuple of arguments for that callable object, state to be passed to __setstate__ (optional), an iterator yielding list items to be pickled (optional), and an iterator yielding dictionary items to be pickled (optional).
605 |
__reduce_ex__(self)
606 |
__reduce_ex__ exists for compatibility. If it is defined, __reduce_ex__ will be called over __reduce__ on pickling. __reduce__ can be defined as well for older versions of the pickling API that did not support __reduce_ex__.
607 |
608 |

An Example

609 |

Our example is a Slate, which remembers what its values have been and when those values were written to it. However, this particular slate goes blank each time it is pickled: the current value will not be saved.

610 |
import time
611 | 
612 | class Slate:
613 |     '''Class to store a string and a changelog, and forget its value when
614 |     pickled.'''
615 | 
616 |     def __init__(self, value):
617 |         self.value = value
618 |         self.last_change = time.asctime()
619 |         self.history = {}
620 | 
621 |     def change(self, new_value):
622 |         # Change the value. Commit last value to history
623 |         self.history[self.last_change] = self.value
624 |         self.value = new_value
625 |         self.last_change = time.asctime()
626 | 
627 |     def print_changes(self):
628 |         print 'Changelog for Slate object:'
629 |         for k, v in self.history.items():
630 |             print '%s\t %s' % (k, v)
631 | 
632 |     def __getstate__(self):
633 |         # Deliberately do not return self.value or self.last_change.
634 |         # We want to have a "blank slate" when we unpickle.
635 |         return self.history
636 | 
637 |     def __setstate__(self, state):
638 |         # Make self.history = state and last_change and value undefined
639 |         self.history = state
640 |         self.value, self.last_change = None, None
641 | 
642 | 643 | 644 |

Conclusion

645 |

The goal of this guide is to bring something to anyone that reads it, regardless of their experience with Python or object-oriented programming. If you're just getting started with Python, you've gained valuable knowledge of the basics of writing feature-rich, elegant, and easy-to-use classes. If you're an intermediate Python programmer, you've probably picked up some slick new concepts and strategies and some good ways to reduce the amount of code written by you and clients. If you're an expert Pythonista, you've been refreshed on some of the stuff you might have forgotten about and maybe picked up a few new tricks along the way. Whatever your experience level, I hope that this trip through Python's special methods has been truly magical (I couldn't resist the final pun).

Appendix 1: How to Call Magic Methods

646 |

Some of the magic methods in Python directly map to built-in functions; in this case, how to invoke them is fairly obvious. However, in other 647 | cases, the invocation is far less obvious. This appendix is devoted to exposing non-obvious syntax that leads to magic methods getting called.

648 | 649 | 650 | 651 | 652 | 653 | 654 | 655 | 656 | 657 | 658 | 659 | 660 | 661 | 662 | 663 | 664 | 665 | 666 | 667 | 668 | 669 | 670 | 671 | 672 | 673 | 674 | 675 | 676 | 677 | 678 | 679 | 680 | 681 | 682 | 683 | 684 | 685 | 686 | 687 | 688 | 689 | 690 | 691 | 692 | 693 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | 701 | 702 | 703 | 704 | 705 | 706 | 707 | 708 | 709 | 710 | 711 | 712 | 713 | 714 | 715 | 716 | 717 | 718 | 719 | 720 | 721 | 722 | 723 | 724 | 725 | 726 | 727 | 728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 | 739 | 740 | 741 | 742 | 743 | 744 | 745 | 746 | 747 | 748 | 749 | 750 | 751 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | 759 | 760 | 761 | 762 | 763 | 764 | 765 | 766 | 767 | 768 |
Magic MethodWhen it gets invoked (example)Explanation
__new__(cls [,...])instance = MyClass(arg1, arg2)__new__ is called on instance creation
__init__(self [,...])instance = MyClass(arg1, arg2)__init__ is called on instance creation
__cmp__(self, other)self == other, self > other, etc.Called for any comparison
__pos__(self)+selfUnary plus sign
__neg__(self)-selfUnary minus sign
__invert__(self)~selfBitwise inversion
__index__(self)x[self]Conversion when object is used as index
__nonzero__(self)bool(self)Boolean value of the object
__getattr__(self, name)self.name # name doesn't existAccessing nonexistent attribute
__setattr__(self, name, val)self.name = valAssigning to an attribute
__delattr__(self, name)del self.nameDeleting an attribute
__getattribute__(self, name)self.nameAccessing any attribute
__getitem__(self, key)self[key]Accessing an item using an index
__setitem__(self, key, val)self[key] = valAssigning to an item using an index
__delitem__(self, key)del self[key]Deleting an item using an index
__iter__(self)for x in selfIteration
__contains__(self, value)value in self, value not in selfMembership tests using in
__call__(self [,...])self(args)"Calling" an instance
__enter__(self)with self as x:with statement context managers
__exit__(self, exc, val, trace)with self as x:with statement context managers
__getstate__(self)pickle.dump(pkl_file, self)Pickling
__setstate__(self)data = pickle.load(pkl_file)Pickling
769 |

Hopefully, this table should have cleared up any questions you might have had about what syntax invokes which magic method.

770 |

Appendix 2: Changes in Python 3

771 |

Here, we document a few major places where Python 3 differs from 2.x in terms of its object model:

772 | 790 | 791 | 792 | -------------------------------------------------------------------------------- /magicmethods.markdown: -------------------------------------------------------------------------------- 1 | ## 简介 2 | 3 | 有关 Python 内编写类的各种技巧和方法(构建和初始化、重载操作符、类描述、属性访问控制、自定义序列、反射机制、可调用对象、上下文管理、构建描述符对象、Pickling)。 4 | 你可以把它当作一个教程,进阶,或者使用参考;我希望它能够成为一份针对 Python 方法的用户友好指南。 5 | 6 | 7 | 8 | ## 1.介绍 9 | 10 | 这份指南是几个月内最有价值的 Blog 投稿精华。它的主题是向大家讲述 Python 中的神奇方法。 11 | 12 | 何为神奇方法呢?它们是面向 Python 中的一切,是一些特殊的方法允许在自己的定义类中定义增加“神奇”的功能。它们总是使用双下划线(比如 `__init__` 或 `__lt__`),但它们的文档没有很好地把它们表现出来。所有这些神奇方法都出现在Python的官方文档中,但内容相对分散,组织结构也显得松散。还有你会难以发现一个实例(虽然他们被设计很棒,在语言参考中被详细描述,可之后就会伴随着枯燥的语法描述等)。 13 | 14 | 所以,为了解决我认为在 Python 文档中的一大败笔,我打算用更多纯英语,实例驱动的文档来说明 Python 的神奇方法。然后我就开始花了几周的时间来写 blog,而现在我已经完成了它们,并将它们合订成一份指南。 15 | 16 | 我希望你喜欢它。把它当作一个教程,进阶,或者使用参考;我希望它能够成为一份针对 Python 方法的用户友好指南。 17 | 18 | 19 | ## 2.构建和初始化 20 | 21 | 相信大家都熟悉这个最基础的神奇方法 `__init__`。它令你能自定义一个对象的初始化行为。而当我调用 `x=SomeClass()` 时,`__init__` 并不是最先被调用的。实际上有一个叫做 `__new__` 的方法,事实上是它创建了实例,它传递任何参数给初始化程序来达到创建的目的。在对象生命周期结束时,调用 `__del__`。让我们更近地观察下这 3 个神奇方法吧: 22 | 23 | __new__(cls,[...) 24 | 25 | 一个对象的实例化时 `__new__` 是第一个被调用的方法。在类中传递其他任何参数到 `__init__`。`__new__`很少被使用,这样做确实有其目的,特别是当一个子类继承一个不可改变的类型(一个元组或一个字符串)时。我不打算再继续深入追求 `__new__` 的细节了,因为这不会产生多大用处,因为在 Python Docs 内已经涵盖了一份巨详细的说明了。 26 | 27 | __init__(self,[...) 28 | 29 | 类的初始化。它会获得初始构建调用传过来的任何东西(举例来说就是,当我们调用`x=SomeClass(10,'foo')`,`__init__` 就会把传过来的 10 和 'foo' 作为参数。`__init__`在 Python 的类定义中几乎普遍被使用) 30 | 31 | __del__(self) 32 | 33 | 如果 `__new__`和 `__init__` 是对象的构造器,那么 `__del__` 就是析构器。它不实现声明为`del x`(这样的代码不会解释成 `x.__del__()`)的行为。相反,它定义为当一个对象被垃圾回收时的行为。这可能对可能需要额外清理的对象相当有用,比如 sockets 或文件对象。但要小心,如果对象仍处于存活状态而当被解释退出时,`__del__` 没有保证就会被执行,因此这样的`__del__` 不能作为良好的编码规范的替代。(就像当你完成操作总是要关闭一次连接。但事实上,`__del__` 几乎永远不会执行,就因为它处于不安全情况被调用了。使用时保持警惕!) 34 | 35 | 把上述这些内容合在一起,就成了一份 `__init__` 和 `__del__` 的实际使用用例: 36 | 37 | from os.path import join 38 | class FileObject: 39 | '''对文件对象的包装,确保文件在关闭时得到删除''' 40 | 41 | def __init__(self, filepath='~', filename='sample.txt'): 42 | # 按filepath,读写模式打开名为filename的文件 43 | self.file=open(join(filepath,filename), 'r+') 44 | 45 | def __del__(self): 46 | self.file.close() 47 | del self.file 48 | 49 | 50 | ## 3.使操作符在自定义类内工作 51 | 52 | 使用 Python 神奇方法的优势之一就是它提供了一种简单的方式能让对象的行为像内建类型。这意味着你可以避免用丑陋,反直觉和非标准方法执行基本运算。在某些语言中,通常会这样做: 53 | 54 | if instance.equals(other_instance): 55 | # do something 56 | 57 | 你也应该在 Python 确实会这样做,但同时它会增加用户的疑惑以及不必要的冗长。不同的库可能会对相同的运算采用不同的命名,这使得用户比平常干了更多的事。依靠神奇方法的力量,你可以定义一个方法(比如 `__eq__`),然后带代替我们真实的意图: 58 | 59 | if instance == other_instance: 60 | # do something 61 | 62 | 现在你看到的是神奇方法力量的一部分。绝大多数都允许我们定义为运算符本身的意义,当用在我们自己定义的类上就像它们是内建类型。 63 | 64 | 65 | ### 3.1 神奇方法——比较 66 | 67 | Python 有一整套神奇方法被设计用来通过操作符实现对象间直观的比较,而非别扭的方法调用。它们同样提供了一套覆盖 Python 对象比较的默认行为(通过引用)。以下是这些方法的列表以及做法: 68 | 69 | __cmp__(self, other) 70 | 71 | `__cmp__`是神奇方法中最基础的一个。实际上它实现所有比较操作符行为(`<`,`==`,`!=`,等),但它有可能不按你想要的方法工作(例如,一个实例是否等于另一个这取决于比较的准则,以及一个实例是否大于其他的这也取决于其他的准则)。如果 `self < other`,那 `__cmp__` 应当返回一个负整数;如果 `self == other`,则返回 0;如果 `self > other`,则返回正整数。它通常是最好的定义,而不需要你一次就全定义好它们,但当你需要用类似的准则进行所有的比较时,`__cmp__` 会是一个很好的方式,帮你节省重复性和提高明确度。 72 | 73 | __eq__(self, other) 74 | 定义了相等操作符,`==`的行为。 75 | 76 | __ne__(self, other) 77 | 定义了不相等操作符,`!=` 的行为。 78 | 79 | __lt__(self, other) 80 | 定义了小于操作符,`<` 的行为。 81 | 82 | __gt__(self, other) 83 | 定义了大于操作符,`>` 的行为。 84 | 85 | __le__(self, other) 86 | 定义了小于等于操作符,`<=`的行为。 87 | 88 | __ge__(self, other) 89 | 定义了大于等于操作符,`>=` 的行为。 90 | 91 | 举一个例子,设想对单词进行类定义。我们可能希望能够按内部对 string 的默认比较行为,即字典序(通过字母)来比较单词,也希望能够基于某些其他的准则,像是长度或音节数。在本例中,我们通过单词长度排序,以下给出实现: 92 | 93 | class Word(str): 94 | '''单词类,比较定义是基于单词长度的''' 95 | 96 | def __new__(cls, word): 97 | # 注意,我们使用了__new__,这是因为str是一个不可变类型, 98 | # 所以我们必须更早地初始化它(在创建时) 99 | if ' ' in word: 100 | print "单词内含有空格,截断到第一部分" 101 | word = word[:word.index(' ')] # 在出现第一个空格之前全是字符了现在 102 | return str.__new__(cls, word) 103 | 104 | def __gt__(self, other): 105 | return len(self) > len(other) 106 | def __lt__(self, other): 107 | return len(self) < len(other) 108 | def __ge__(self, other): 109 | return len(self) >= len(other) 110 | def __le__(self, other): 111 | return len(self) <= len(other) 112 | 113 | 现在,我们可以创建 2 个单词(通过 `Word('foo')` 和 `Word('bar')`)并基于它们的长度进行比较了。注意,我们没有定义 `__eq__` 和 `__ne__`。这是因为这可能导致某些怪异的行为(特别是当比较 `Word('foo') == Word('bar')` 将会得到 `True` 的结果)。基于单词长度的相等比较会令人摸不清头脑,因此我们就沿用了str 本身的相等比较的实现。 114 | 115 | 现在可能是一个好时机来提醒你一下,你不必重载每一个比较相关的神奇方法来获得各种比较。标准库已经友好地为我们在模板 `functools` 中提供了一个装饰(`decorator`)类,定义了所有比较方法。你可以只重载 `__eq__` 和一个其他的方法(比如 `__gt__`,`__lt__`,等)。这个特性只在 Python2.7(后?)适用,但当你有机会的话应该尝试一下,它会为你省下大量的时间和麻烦。你可以通过在你自己的重载方法在加上 `@total_ordering` 来使用。 116 | 117 | 118 | ### 3.2 神奇方法——数字 119 | 120 | 就像你可以通过重载比较操作符的途径来创建你自己的类实例,你同样可以重载数字操作符。系好你们的安全带,朋友们,还有很多呢。处于本文组织的需要,我会把数字的神奇方法分割成5块:一元操作符,常规算术操作符,反射算术操作符,增量赋值,类型转换。 121 | 122 | #### 一元操作符 123 | 124 | 一元运算和函数仅有一个操作数,比如负数,绝对值等 125 | 126 | __pos__(self) 127 | 实现一元正数的行为(如:`+some_object`) 128 | 129 | __neg__(self) 130 | 实现负数的行为(如: `-some_object`) 131 | 132 | __abs__(self) 133 | 实现内建 `abs()` 函数的行为 134 | 135 | __invert__(self) 136 | 实现用~操作符进行的取反行为。你可以参考 Wiki:bitwise operations 来解释这个运算符究竟会干什么 137 | 138 | #### 常规算术操作符 139 | 140 | 现在我们涵盖了基本的二元运算符:`+`,`-`,`*` 等等。其中大部分都是不言自明的。 141 | 142 | __add__(self, other) 143 | 实现加法 144 | 145 | __sub__(self, other) 146 | 实现减法 147 | 148 | __mul__(self, other) 149 | 实现乘法 150 | 151 | __floordiv__(self, other) 152 | 实现地板除法,使用 `//` 操作符 153 | 154 | __div__(self, other) 155 | 实现传统除法,使用 `/` 操作符 156 | 157 | __truediv__(self, other) 158 | 实现真正除法。注意,只有当你 `from __future__ import division` 时才会有效 159 | 160 | __mod__(self, other) 161 | 实现求模,使用 `%` 操作符 162 | 163 | __divmod__(self, other) 164 | 实现内建函数 `divmod()` 的行为 165 | 166 | __pow__(self, other) 167 | 实现乘方,使用 `**` 操作符 168 | 169 | __lshift__(self, other) 170 | 实现左按位位移,使用 `<<` 操作符 171 | 172 | __rshift__(self, other) 173 | 实现右按位位移,使用 `>>` 操作符 174 | 175 | __and__(self, other) 176 | 实现按位与,使用 `&` 操作符 177 | 178 | __or__(self, other) 179 | 实现按位或,使用 `|` 操作符 180 | 181 | __xor__(self, other) 182 | 实现按位异或,使用 `^` 操作符 183 | 184 | #### 反射算术操作符 185 | 186 | 你知道我会如何解释反射算术操作符?你们中的有些人或许会觉得它很大,很可怕,是国外的概念。但它实际上很简单,下面给一个例子: 187 | 188 | some_object + other 189 | 190 | 这是“常规的”加法。而反射其实相当于一回事,除了操作数改变了改变下位置: 191 | 192 | other + some_object 193 | 194 | 因此,所有这些神奇的方法会做同样的事等价于常规算术操作符,除了改变操作数的位置关系,比如第一个操作数和自身作为第二个。此外没有其他的操作方式。在大多数情况下,反射算术操作的结果等价于常规算术操作,所以你尽可以在刚重载完 `__radd__`就调用 `__add__`。干脆痛快: 195 | 196 | __radd__(self, other) 197 | 实现反射加法 198 | 199 | __rsub__(self, other) 200 | 实现反射减法 201 | 202 | __rmul__(self, other) 203 | 实现反射乘法 204 | 205 | __rfloordiv__(self, other) 206 | 实现反射地板除,用 `//` 操作符 207 | 208 | __rdiv__(self, other) 209 | 实现传统除法,用 `/` 操作符 210 | 211 | __rturediv__(self, other) 212 | 实现真实除法,注意,只有当你 `from __future__ import division` 时才会有效 213 | 214 | __rmod__(self, other) 215 | 实现反射求模,用 `%` 操作符 216 | 217 | __rdivmod__(self, other) 218 | 实现内置函数 `divmod()` 的长除行为,当调用 `divmod(other,self)` 时被调用 219 | 220 | __rpow__(self, other) 221 | 实现反射乘方,用 `**` 操作符 222 | 223 | __rlshift__(self, other) 224 | 实现反射的左按位位移,使用 `<<` 操作符 225 | 226 | __rrshift__(self, other) 227 | 实现反射的右按位位移,使用 `>>` 操作符 228 | 229 | __rand__(self, other) 230 | 实现反射的按位与,使用 `&` 操作符 231 | 232 | __ror__(self, other) 233 | 实现反射的按位或,使用 `|` 操作符 234 | 235 | __rxor__(self, other) 236 | 实现反射的按位异或,使用 `^` 操作符 237 | 238 | #### 增量赋值 239 | 240 | Python 也有各种各样的神奇方法允许用户自定义增量赋值行为。你可能已经熟悉增量赋值,它结合了“常规的”操作符和赋值。如果你仍不明白我在说什么,下面有一个例子: 241 | 242 | x = 5 243 | x += 1 # 等价 x = x + 1 244 | 245 | 这些方法都不会有返回值,因为赋值在 Python 中不会有任何返回值。反而它们只是改变类的状态。列表如下: 246 | 247 | __iadd__(self, other) 248 | 实现加法和赋值 249 | 250 | __isub__(self, other) 251 | 实现减法和赋值 252 | 253 | __imul__(self, other) 254 | 实现乘法和赋值 255 | 256 | __ifloordiv__(self, other) 257 | 实现地板除和赋值,用 `//=` 操作符 258 | 259 | __idiv__(self, other) 260 | 实现传统除法和赋值,用 `/=` 操作符 261 | 262 | __iturediv__(self, other) 263 | 实现真实除法和赋值,注意,只有当你 `from __future__ import division` 时才会有效 264 | 265 | __imod__(self, other) 266 | 实现求模和赋值,用 `%=` 操作符 267 | 268 | __ipow__(self, other) 269 | 实现乘方和赋值,用 `**=` 操作符 270 | 271 | __ilshift__(self, other) 272 | 实现左按位位移和赋值,使用 `<<=` 操作符 273 | 274 | __irshift__(self, other) 275 | 实现右按位位移和赋值,使用 `>>=` 操作符 276 | 277 | __iand__(self, other) 278 | 实现按位与和赋值,使用 `&=` 操作符 279 | 280 | __ior__(self, other) 281 | 实现按位或和赋值,使用 `|=` 操作符 282 | 283 | __ixor__(self, other) 284 | 实现按位异或和赋值,使用 `^=` 操作符 285 | 286 | #### 类型转换的神奇方法 287 | 288 | Python 也有一组神奇方法被设计用来实现内置类型转换函数的行为,如 `float()` 289 | 290 | __int__(self) 291 | 实现到 `int` 的类型转换 292 | 293 | __long__(self) 294 | 实现到 `long` 的类型转换 295 | 296 | __float__(self) 297 | 实现到 `float` 的类型转换 298 | 299 | __complex__(self) 300 | 实现到复数的类型转换 301 | 302 | __oct__(self) 303 | 实现到 8 进制的类型转换 304 | 305 | __hex__(self) 306 | 实现到 16 进制的类型转换 307 | 308 | __index__(self) 309 | 实现一个当对象被切片到 `int` 的类型转换。如果你自定义了一个数值类型,考虑到它可能被切片,所以你应该重载 `__index__` 310 | 311 | __trunc__(self) 312 | 当 `math.trunc(self)` 被调用时调用。`__trunc__` 应当返回一个整型的截断,(通常是 `long`) 313 | 314 | __coerce__(self, other) 315 | 该方法用来实现混合模式的算术。如果类型转换不可能那 `__coerce__` 应当返回 `None`。 316 | 否则,它应当返回一对包含 `self` 和 `other`(2 元组),且调整到具有相同的类型 317 | 318 | 319 | ## 4.描述你的类 320 | 321 | 用一个字符串来说明一个类这通常是有用的。 322 | 在 Python 中提供了一些方法让你可以在你自己的类中自定义内建函数返回你的类行为的描述。 323 | 324 | __str__(self) 325 | 当你定义的类中一个实例调用了 `str()`,用于给它定义行为 326 | 327 | __repr__(self) 328 | 当你定义的类中一个实例调用了 `repr()`,用于给它定义行为。 329 | `str()` 和 `repr()` 主要的区别在于它的阅读对象。 330 | `repr()` 产生的输出主要为计算机可读(在很多情况下,这甚至可能是一些有效的 Python 代码),而 `str()` 则是为了让人类可读。 331 | 332 | __unicode__(self) 333 | 当你定义的类中一个实例调用了 `unicode()`,用于给它定义行为。 334 | `unicode()` 像是 `str()`,只不过它返回一个 unicode 字符串。 335 | 警惕!如果用户用你的类中的一个实例调用了 `str()`,而你仅定义了 `__unicode__()`,那它是不会工作的。 336 | 以防万一,你应当总是定义好 `__str__()`,哪怕用户不会使用 unicode 337 | 338 | __hash__(self) 339 | 当你定义的类中一个实例调用了 `hash()`,用于给它定义行为。 340 | 它必须返回一个整型,而且它的结果是用于来在字典中作为快速键比对。 341 | 342 | __nonzero__(self) 343 | 当你定义的类中一个实例调用了 `bool()`,用于给它定义行为。 344 | 返回 `True` 或 `False`,取决于你是否考虑一个实例是 `True` 或 `False` 的。 345 | 346 | 我们已经相当漂亮地干完了神奇方法无聊的部分(无示例),至此我们已经讨论了一些基础的神奇方法,是时候让我们向高级话题移动了。 347 | 348 | 349 | ## 5.属性访问控制 350 | 351 | 有许多从其他语言阵营转到 Python 来的人抱怨 Python 对类缺乏真正的封装(比如,没有办法自定义 `private` 属性,已经给出 `public` 的 `getter` 和 `setter`)。 352 | 这可不是真相哟:Python 通过神奇的方法实现了大量的封装,而不是通过明确的方法或字段修饰符。 353 | 354 | 请看: 355 | 356 | __getattr__(self, name) 357 | 358 | 你可以为用户在试图访问不存在(不论是存在或尚未建立)的类属性时定义其行为。 359 | 这对捕捉和重定向常见的拼写错误,给出使用属性警告是有用的(只要你愿意,你仍旧可选计算,返回那个属性)或抛出一个 `AttributeError` 异常。 360 | 这个方法只适用于访问一个不存在的属性,所以,这不算一个真正封装的解决之道。 361 | 362 | __setattr__(self, name, value) 363 | 364 | 不像 `__getattr__`,`__setattr__` 是一个封装的解决方案。 365 | 它允许你为一个属性赋值时候的行为,不论这个属性是否存在。 366 | 这意味着你可以给属性值的任意变化自定义规则。 367 | 然而,你需要在意的是你要小心使用 `__setattr__`,在稍后的列表中会作为例子给出。 368 | 369 | __delattr__ 370 | 371 | 这等价于 `__setattr__`, 但是作为删除类属性而不是 `set` 它们。 372 | 它需要相同的预防措施,就像 `__setattr__`,防止无限递归(当在 `__delattr__` 中调用 `del self.name` 会引起无限递归)。 373 | 374 | __getattribute__(self, name) 375 | 376 | `__getattribute__` 良好地适合它的同伴们 `__setattr__` 和 `__delattr__`。 377 | 可我却不建议你使用它。 378 | `__getattribute__` 只能在新式类中使用(在 Python 的最新版本中,所有的类都是新式类,在稍旧的版本中你可以通过继承 `object` 类来创建一个新式类。 379 | 它允许你定规则,在任何时候不管一个类属性的值那时候是否可访问的。) 380 | 它会因为他的同伴中的出错连坐受到某些无限递归问题的困扰(这时你可以通过调用基类的 `__getattribute__` 方法来防止发生)。 381 | 当 `__getattribute__` 被实现而又只调用了该方法如果 `__getattribute__` 被显式调用或抛出一个 `AttributeError` 异常,同时也主要避免了对 `__getattr__` 的依赖。 382 | 这个方法可以使用(毕竟,这是你自己的选择),不过我不推荐它是因为它有一个小小的用例(虽说比较少见,但我们需要特殊行为以获取一个值而不是赋值)以及它真的很难做到实现 0bug。 383 | 384 | 你可以很容易地在你自定义任何类属性访问方法时引发一个问题。参考这个例子: 385 | 386 | def __setattr__(self, name, value): 387 | self.name = value 388 | # 当每次给一个类属性赋值时,会调用__setattr__(),这就形成了递归 389 | # 因为它真正的含义是 self.__setattr__('name', value) 390 | # 所以这方法不停地调用它自己,变成了一个无法退出的递归最终引发crash 391 | 392 | def __setattr__(self, name, value): 393 | self.__dict__[name] = value # 给字典中的name赋值 394 | # 在此自定义行为 395 | 396 | 再一次,Python 的神奇方法向我们展示了其难以置信的能力,同时巨大的力量也伴随着重大的责任。 397 | 重要的是让你明白正确使用神奇方法,这样你就不会破坏其他代码。 398 | 399 | 那么,我们在关于定制类属性访问中学习了什么? 400 | 不要轻易地使用,事实上它过于强大以及反直觉。 401 | 这也是它为何存在的理由:Python 寻求干坏事的可能性,但会把它们弄得很难。 402 | 自由是至高无上的,所以你可以做任何你想做的事情。 403 | 以下是一个关于特殊属性访问方法的实际例子(注意,我们使用 `super` 因为并非所有类都有 `__dict__` 类属性): 404 | 405 | class AccessCounter: 406 | '''一个类包含一个值和实现了一个访问计数器。 407 | 当值每次发生变化时,计数器+1''' 408 | 409 | def __init__(self, val): 410 | super(AccessCounter, self).__setattr__('counter',0) 411 | super(AccessCounter, self).__setattr__('value', val) 412 | 413 | def __setattr__(self, name, value): 414 | if name == 'value': 415 | super(AccessCounter, self).__setattr__('counter', self.counter + 1) 416 | # Make this unconditional. 417 | # 如果你想阻止其他属性被创建,抛出AttributeError(name)异常 418 | super(AccessCounter, self).__setattr__(name, value) 419 | 420 | def __delattr__(self, name) 421 | if name == 'value': 422 | super(AccessCounter, self).__setattr__('counter', self.counter + 1) 423 | super(AccessCounter, self).__delattr__(name) 424 | 425 | 426 | ## 6.制作自定义序列 427 | 428 | 很有多种方式可以让你的类表现得像内建序列(字典,元组,列表,字符串等)。 429 | 这些是我迄今为止最喜欢的神奇方法了,因为不合理的控制它们赋予了你一种魔术般地让你的类实例整个全局函数数组漂亮工作的方式。 430 | 在我们开始讲解这个内容之前,让我们先快速理清需求。 431 | 432 | #### 需求 433 | 434 | 现在我们正在谈论如何创建你自己的序列。 435 | 也是什么谈一谈 protocol 了。 436 | protocol 在某些地方跟接口很相似。 437 | 接口在其他语言中,是一组给定的方法,而你必须定义它们。 438 | 然而,在 Python 中 protocol 是完全非正式的,而且不要求显式声明去实现。 439 | 更进一步说,它们更像是准则。 440 | 441 | 为何我们现在要谈论 protocol? 442 | 因为在 Python 中要实现自定义容器类型会涉及使用到这其中某些 protocol。 443 | 444 | 首先,有一个 protocol 是为定义不可变容器的:为了制作一个不可变容器,你只需要定义 `__len__` 和 `__getitem__`(稍后详述)。 445 | 可变容器 protocol 要求所有不可变容器增加 `__setitem__` 和 `__delitem__`。 446 | 然后,如果你希望你的对象是可迭代的,那你还得定义一个会返回迭代器 `iterator` 的 `__iter__` 方法。 447 | 并且这个迭代器必须遵守一个迭代 `protocol`,也就是要求迭代器有回调方法 `__iter__` (返回自身)和 `next`。 448 | 449 | #### 隐藏在容器背后的魔法 450 | 451 | 已经迫不及待了?以下便是容器使用的神奇魔法: 452 | 453 | __len__(self) 454 | 455 | 返回容器的长度。部分 protocol 同时支持可变和不可变容器 456 | 457 | __getitem__(self, key) 458 | 459 | 定义当某一个 item 被访问时的行为,使用 `self[key]` 表示法。 460 | 这个同样也是部分可变和不可变容器 protocol。 461 | 这也可抛出适当的异常: `TypeError` 当 `key` 的类型错误,或没有值对应 Key 时。 462 | 463 | __setitem__(self, key, value) 464 | 465 | 定义当某一个 item 被赋值时候的行为,使用 `self[key]=value` 表示法。 466 | 这也是部分可变和不可变容器 protocol。 467 | 再一次重申,你应当在适当之处抛出 `KeyError` 和 `TypeError` 异常。 468 | 469 | __delitem__(self, key) 470 | 471 | 定义当某一个 item 被删除(例如 `del self[key]`)时的行为。 472 | 这仅是部分可变容器的 protocol。在一个无效 `key` 被使用后,你必须抛出一个合适的异常。 473 | 474 | __iter__(self) 475 | 476 | 应该给容器返回一个迭代器。 477 | 迭代器会返回若干内容,大多使用内建函数 `iter()` 表示。 478 | 当一个容器使用形如 `for x in container:` 的循环。 479 | 迭代器本身就是其对象,同时也要定义好一个 `__iter__` 方法来返回自身。 480 | 481 | __reversed__(self) 482 | 483 | 当定义调用内建函数 `reversed()` 时的行为。应该返回一个反向版本的列表。 484 | 485 | __contains__(self, item) 486 | 487 | `__contains__` 为成员关系,用 `in` 和 `not in` 测试时定义行为。 488 | 那你会问这个为何不是一个序列的 protocol 的一部分? 489 | 这是因为当 `__contains__` 未定义,Python 就会遍历序列,如果遇到正在寻找的 item 就会返回 `True`。 490 | 491 | __concat__(self, other) 492 | 493 | 最后,你可通过 `__concat__` 定义你的序列和另外一个序列的连接。 494 | 应该从 `self` 和 `other` 返回一个新构建的序列。 495 | 当调用 2 个序列时 `__concat__` 涉及操作符 `+` 496 | 497 | #### 一个例子 498 | 499 | 在我们的例子中,让我们看一下一个 list 实现的某些基础功能性的构建。 500 | 可能会让你想起你使用的其他语言(比如 Haskell)。 501 | 502 | class FunctionalList: 503 | '''类覆盖了一个list的某些额外的功能性魔法,像head, 504 | tail,init,last,drop,and take''' 505 | def __init__(self, values=None): 506 | if values is None: 507 | self.values = [] 508 | else: 509 | self.values = values 510 | 511 | def __len__(self): 512 | return len(self.values) 513 | 514 | def __getitem__(self, key): 515 | # 如果key是非法的类型和值,那么list valuse会抛出异常 516 | return self.values[key] 517 | 518 | def __setitem__(self, key, value): 519 | self.values[key] = value 520 | 521 | def __delitem__(self, key): 522 | del self.values[key] 523 | 524 | def __iter__(self): 525 | return iter(self.values) 526 | 527 | def __reversed__(self): 528 | return reversed(self.values) 529 | 530 | def append(self, value): 531 | self.values.append(value) 532 | def head(self): 533 | # 获得第一个元素 534 | return self.values[0] 535 | def tail(self): 536 | # 获得在第一个元素后的其他所有元素 537 | return self.values[1:] 538 | def init(self): 539 | # 获得除最后一个元素的序列 540 | return self.values[:-1] 541 | def last(last): 542 | # 获得最后一个元素 543 | return self.values[-1] 544 | def drop(self, n): 545 | # 获得除前n个元素的序列 546 | return self.values[n:] 547 | def take(self, n): 548 | # 获得前n个元素 549 | return self.values[:n] 550 | 551 | 通过这个(轻量的)有用的例子你知道了如何实现你自己的序列。 552 | 当然,还有很多更有用的应用,但是它们其中的很多已经被标准库实现了,像 Counter, OrderedDict, NamedTuple 553 | 554 | 555 | ## 7.反射 556 | 557 | 你也可以通过定义神奇方法来控制如何反射使用内建函数 `isinstance()` 和 `issubclass()` 的行为。 558 | 这些神奇方法是: 559 | 560 | __instancecheck__(self, instance) 561 | 562 | 检查一个实例是否是你定义类中的一个实例(比如,`isinstance(instance, class)`) 563 | 564 | __subclasscheck__(self, subclass) 565 | 566 | 检查一个类是否是你定义类的子类(比如,`issubclass(subclass, class)`) 567 | 568 | 这对于神奇方法的用例情况来说可能较小,可它的确是真的。 569 | 我并不想花费太多的时间在反射方法上面,因为他们不是那么地重要。 570 | 不过它们反映了在 Python 中关于面对对象编程一些重要的东西,而且在 Python 中的普遍:总是在找一种简单的方式来做某些事情,即使它能被用到的不多。 571 | 这些神奇方法似乎看上去不那么有用,但当你需要使用它们的时候你会感激它们的存在(和你阅读的这本指南!)。 572 | 573 | 574 | ## 8.可调用对象 575 | 576 | 正如你可能已经知道,在 Python 中函数是第一类对象。 577 | 这就意味着它们可以被传递到函数和方法,就像是任何类型的对象。 578 | 这真是一种难以置信强大的特性。 579 | 580 | 这是 Python 中一个特别的神奇方法,它允许你的类实例像函数。 581 | 所以你可以“调用”它们,把他们当做参数传递给函数等等。 582 | 这是另一个强大又便利的特性让 Python 的编程变得更可爱了。 583 | 584 | __call__(self, [args...]) 585 | 586 | 允许类实例像函数一样被调用。 587 | 本质上,这意味着 `x()` 等价于 `x.__call__()`。 588 | 注意,`__call__` 需要的参数数目是可变的,也就是说可以对任何函数按你的喜好定义参数的数目定义 `__call__` 589 | 590 | `__call__` 可能对于那些经常改变状态的实例来说是极其有用的。 591 | “调用”实例是一种顺应直觉且优雅的方式来改变对象的状态。 592 | 下面一个例子是一个类表示一个实体在一个平面上的位置: 593 | 594 | class Entity: 595 | '''描述实体的类,被调用的时候更新实体的位置''' 596 | 597 | def __init__(self, size, x, y): 598 | self.x, self.y = x, y 599 | self.size = size 600 | 601 | def __call__(self, x, y): 602 | '''改变实体的位置''' 603 | self.x, self.y = x, y 604 | 605 | #省略... 606 | 607 | 608 | ## 9.上下文管理 609 | 610 | 在 Python2.5 里引入了一个新关键字(`with`)使得一个新方法得到了代码复用。 611 | 上下文管理这个概念在 Python 中早已不是新鲜事了(之前它作为库的一部分被实现过),但直到 [PEP343](http://www.python.org/dev/peps/pep-0343/) 才作为第一个类语言结构取得了重要地位而被接受。 612 | 你有可能早就已经见识过 `with` 声明: 613 | 614 | with open('foo.txt') as bar: 615 | # 对bar执行某些动作 616 | 617 | 上下文管理允许对对象进行设置和清理动作,用 `with` 声明进行已经封装的操作。 618 | 上下文操作的行为取决于 2 个神奇方法: 619 | 620 | __enter__(self) 621 | 622 | 定义块用 `with` 声明创建出来时上下文管理应该在块开始做什么。 623 | 注意,`__enter__` 的返回值必须绑定 `with` 声明的目标,或是 `as` 后面的名称。 624 | 625 | __exit__(self, exception_type, exception_value, traceback) 626 | 627 | 定义在块执行(或终止)之后上下文管理应该做什么。 628 | 它可以用来处理异常,进行清理,或行动处于块之后某些总是被立即处理的事。 629 | 如果块执行成功的话,`excepteion_type`,`exception_value`,和 `traceback` 将会置 `None`。 630 | 否则,你可以选择去处理异常,或者让用户自己去处理。 631 | 如果你想处理,确保在全部都完成之后 `__exit__` 会返回 `True`。 632 | 如果你不想让上下文管理处理异常,那就让它发生好了。 633 | 634 | `__enter__` 和 `__exit__` 对那些已有良好定义和对设置,清理行为有共同行为的特殊类是有用。 635 | 你也可以使用这些方法去创建封装其他对象通用的上下文管理。 636 | 看下面的例子: 637 | 638 | class Closer: 639 | '''用with声明一个上下文管理用一个close方法自动关闭一个对象''' 640 | 641 | def __init__(self, obj): 642 | self.obj = obj 643 | 644 | def __enter__(self): 645 | return self.obj # 绑定目标 646 | 647 | def __exit__(self, exception_type, exception_val, trace): 648 | try: 649 | self.obj.close() 650 | except AttributeError: #obj不具备close 651 | print 'Not closable.' 652 | return True # 成功处理异常 653 | 654 | 以下是一个对于 `Closer` 实际应用的一个例子,使用一个 FTP 连接进行的演示(一个可关闭的套接字): 655 | 656 | >>> from magicmethods import Closer 657 | >>> from ftplib import :;; 658 | >>> with Closer(FTP('ftp.somsite.com')) as conn: 659 | ... conn.dir() 660 | ... 661 | # 省略的输出 662 | >>> conn.dir() 663 | # 一个很长的AttributeError消息, 不能关闭使用的一个连接 664 | >>> with Closer(int(5)) as i: 665 | ... i += 1 666 | ... 667 | Not closeable. 668 | >>> i 669 | 6 670 | 671 | 瞧见我们如何漂亮地封装处理正确或不正确的用例了吗?那就是上下文管理和神奇方法的威力。 672 | 673 | 674 | ## 10.构建描述符对象 675 | 676 | 描述符可以改变其他对象,也可以是访问类中任一的 `getting`,`setting`,`deleting`。 677 | 描述符不意味着孤立;相反,它们意味着会被它们的所有者类控制。 678 | 当建立面向对象数据库或那些拥有相互依赖的属性的类时,描述符是有用的。 679 | 当描述符在几个不同单元或描述计算属性时显得更为有用。 680 | 681 | 作为一个描述符,一个类必须至少实现 `__get__`,`__set__`,和 `__delete__`中的一个。 682 | 让我们快点看一下这些神奇方法吧: 683 | 684 | __get__(self, instance, owner) 685 | 当描述符的值被取回时定义其行为。`instance` 是 `owner` 对象的一个实例,`owner` 是所有类。 686 | 687 | __set__(self, instance, value) 688 | 当描述符的值被改变时定义其行为。`instance` 是 `owner` 对象的一个实例,`value` 是设置的描述符的值 689 | 690 | __delete__(self, instance) 691 | 当描述符的值被删除时定义其行为。`instance` 是 `owner` 对象的一个实例。 692 | 693 | 694 | 现在,有一个有用的描述符应用例子:单位转换策略 695 | 696 | class Meter(object): 697 | '''米描述符''' 698 | 699 | def __init__(self, value=0.0): 700 | self.value = float(value) 701 | def __get__(self, instance, owner): 702 | return self.value 703 | def __set__(self, instance, value): 704 | self.value = float(value) 705 | 706 | class Foot(object): 707 | '''英尺描述符''' 708 | 709 | def __get__(self, instance, owner): 710 | return instance.meter * 3.2808 711 | def __set__(self, instance, value): 712 | instance.meter = float(value) / 3.2808 713 | 714 | class Distance(object): 715 | '''表示距离的类,控制2个描述符:feet和meters''' 716 | meter = Meter() 717 | foot = Foot() 718 | 719 | 720 | ## 11.Pickling 你的对象 721 | 722 | 假如你花时间和其他 Pythonistas 打交道,那么你至少有可能听到过 Pickling 这个词。 723 | Pickling 是一种对 Python 数据结构的序列化过程。 724 | 如果你需要存储一个对象,之后再取回它(通常是为了缓存)那么它就显得格外地有用了。 725 | 同时,它也是产生忧虑和困惑的主要来源。 726 | 727 | Pickling 是那么地重要以至于它不仅有自己专属的模块(pickle),还有自己的 protocol 和神奇方法与其相伴。 728 | 但首先用简要的文字来解释下如何 pickle 已经存在的类型(如果你已经懂了可以随意跳过这部分内容) 729 | 730 | ### Pickling:盐水中的快速浸泡 731 | 732 | 让我们跳入 pickling。 733 | 话说你有一个词典你想要保存它并在稍后取回。 734 | 你可以把它的内容写到一个文件中去,需要非常小心地确保你写了正确的语法,然后用 `exec()` 或处理文件的输入取回写入的内容。 735 | 但这是不稳定的:如果你你在纯文本中保存重要的数据,它有可能被几种方法改变,导致你的程序 crash 或在你的计算机上运行了恶意代码而出错。 736 | 于是,我们准备 pickle 它: 737 | 738 | import pickle 739 | 740 | data = {'foo': [1,2,3], 741 | 'bar': ('Hello','world!'), 742 | 'baz': True} 743 | jar = open('data.pk1', 'wb') 744 | pickle.dump(data, jar) # 把pickled数据写入jar文件 745 | jar.close() 746 | 747 | 好了现在,已经过去了几个小时。 748 | 我们希望拿回数据,而我们需要做的事仅仅是 `unpickle` 它: 749 | 750 | import pickle 751 | 752 | pk1_file = open('data.pk1','rb') #连接pickled数据 753 | data = pickle.load(pk1_file) #把数据load到一个变量中去 754 | print data 755 | pk1_file.close() 756 | 757 | 发生了什么事?正如你的预期,我们获得了 `data`。 758 | 759 | 现在,我要给你一些忠告:pickling 并非完美。 760 | Pickle 文件很容易因意外或出于故意行为而被损毁。 761 | Pickling 可能比起使用纯文本文件安全些,但它仍旧有可能会被用来跑恶意代码。 762 | 还有因为 Python 版本的不兼容问题,所以不要期望发布 Pickled 对象,也不要期望人们能够打开它们。 763 | 但是,它依然是一个强大的缓存工具和其他常见序列化任务。 764 | 765 | ### Pickling你自定义的对象 766 | 767 | Pickling 不仅可用在内建类型上,还可以用于遵守 pickle 协议的任何类。 768 | pickle 协议有 4 个可选方法用于定制 Python 对象如何运行(这跟 C 扩展有点不同,但那不在我们讨论的范围内): 769 | 770 | __getinitargs__(self) 771 | 如果你想当你的类 unpickled 时调用 `__init__`,那你可以定义`__getinitargs__`,该方法应该返回一个元组的参数,然后你可以把他传递给 `__init__`。注意,该方法仅适用于旧式类。 772 | 773 | __getnewargs__(self) 774 | 对于新式类,你可以影响有哪些参数会被传递到 `__new__` 进行 unpickling。 775 | 该方法同样应该返回一个元组参数,然后能传递给 `__new__` 776 | 777 | __getstate__(self) 778 | 代替对象的 `__dict__` 属性被保存。 779 | 当对象 pickled,你可返回一个自定义的状态被保存。 780 | 当对象 unpickled 时,这个状态将会被 `__setstate__` 使用。 781 | 782 | __setstate__(self, state) 783 | 对象 unpickled 时,如果 `__setstate__` 定义对象状态会传递来代替直接用对象的 `__dict__` 属性。 784 | 这正好跟 `__getstate__` 手牵手:当二者都被定义了,你可以描述对象的 pickled 状态,任何你想要的。 785 | 786 | 一个例子: 787 | 788 | 我们的例子是 `Slate` 类,它会记忆它曾经的值和已经写入的值。 789 | 然而,当这特殊的 `slate` 每一次 `pickle` 都会被清空:当前值不会被保存。 790 | 791 | import time 792 | 793 | class Slate: 794 | '''存储一个字符串和一个变更log,当Pickle时会忘记它的值''' 795 | 796 | def __init__(self, value): 797 | self.value = value 798 | self.last_change = time.asctime() 799 | self.history = {} 800 | 801 | def change(self, new_value): 802 | # 改变值,提交最后的值到历史记录 803 | self.history[self.last_change] = self.value 804 | self.value = new_value 805 | self.last_change = time.asctime() 806 | 807 | def print_changes(self): 808 | print 'Changelog for Slate object:' 809 | for k, v in self.history.items(): 810 | print '%st %s' % (k, v) 811 | 812 | def __getstate__(self): 813 | # 故意不返回self.value 或 self.last_change. 814 | # 当unpickle,我们希望有一块空白的"slate" 815 | return self.history 816 | 817 | def __setstate__(self, state): 818 | # 让 self.history = state 和 last_change 和 value被定义 819 | self.history = state 820 | self.value, self.last_change = None, None 821 | 822 | 823 | ## 12.总结 824 | 825 | 这份指南的目标就是任何人读一读它,不管读者们是否具备 Python 或面对对象的编程经验。 826 | 如果你正准备学习 Python,那你已经获得了编写功能丰富,优雅,易用的类的宝贵知识。 827 | 如果你是一名中级 Python 程序员,你有可能已经拾起了一些新概念和策略和一些好的方法来减少你和你的用户编写的代码量。 828 | 如果你是一名 Pythonista 专家,你可能已经回顾了某些你可能已经被你遗忘的知识点,或着你又学习到了一些新技巧。 829 | 不管你的的经验等级,我希望这次 Python 神奇方法的旅程达到了真正神奇的效果。(我无法控制自己在最后不用个双关语) 830 | 831 | 832 | ## 附录:如果调用神奇方法 833 | 834 | Python 中的一些神奇方法直接映射到内建函数;在这种情况下,调用它们的方法是相当明显的。 835 | 然而,在其他情况下,那些调用方法就不这么明显了。 836 | 本附录致力于揭开能够引导神奇方法被调用的非明显语法。 837 | 838 | 839 | 840 | 841 | 842 | 843 | 844 | 845 | 846 | 847 | 848 | 849 | 850 | 851 | 852 | 853 | 854 | 855 | 856 | 857 | 858 | 859 | 860 | 861 | 862 | 863 | 864 | 865 | 866 | 867 | 868 | 869 | 870 | 871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | 879 | 880 | 881 | 882 | 883 | 884 | 885 | 886 | 887 | 888 | 889 | 890 | 891 | 892 | 893 | 894 | 895 | 896 | 897 | 898 | 899 | 900 | 901 | 902 | 903 | 904 | 905 | 906 | 907 | 908 | 909 | 910 | 911 | 912 | 913 | 914 | 915 | 916 | 917 | 918 | 919 | 920 | 921 | 922 | 923 | 924 | 925 | 926 | 927 | 928 | 929 | 930 | 931 | 932 | 933 | 934 | 935 | 936 | 937 | 938 | 939 | 940 | 941 | 942 | 943 | 944 | 945 | 946 | 947 | 948 | 949 | 950 | 951 | 952 | 953 | 954 | 955 | 956 | 957 | 958 | 959 | 960 |
神奇方法调用方法说明
`__new__(cls [,...])``instance = MyClass(arg1, arg2)``__new__` 在创建实例的时候被调用
`__init__(self [,...])``instance = MyClass(arg1, arg2)``__init__` 在创建实例的时候被调用
`__cmp__(self, other)``self == other`, `self > other`, 等在比较的时候调用
`__pos__(self)``+self`一元加运算符
`__neg__(self)``-self`一元减运算符
`__invert__(self)``~self`取反运算符
`__index__(self)``x[self]`对象被作为索引使用的时候
`__nonzero__(self)``bool(self)`对象的布尔值
`__getattr__(self, name)``self.name # name不存在`访问一个不存在的属性时
`__setattr__(self, name, val)``self.name = val`对一个属性赋值时
`__delattr__(self, name)``del self.name`删除一个属性时
`__getattribute(self, name)``self.name`访问任何属性时
`__getitem__(self, key)``self[key]`使用索引访问元素时
`__setitem__(self, key, val)``self[key] = val`对某个索引值赋值时
`__delitem__(self, key)``del self[key]`删除某个索引值时
`__iter__(self)``for x in self`迭代时
`__contains__(self, value)``value in self`, `value not in self`使用 `in` 操作测试关系时
`__concat__(self, value)``self + other`连接两个对象时
`__call__(self [,...])``self(args)`“调用”对象时
`__enter__(self)``with self as x:``with` 语句上下文管理
`__exit__(self, exc, val, trace)``with self as x:``with` 语句上下文管理
`__getstate__(self)``pickle.dump(pkl_file, self)`序列化
`__setstate__(self)``data = pickle.load(pkl_file)`序列化
961 | 962 | 希望这张表格可以帮你扫清你有关语法涉及到神奇方法的问题。 963 | -------------------------------------------------------------------------------- /magicmethods.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/justjavac/magicmethods-zh_CN/5b4f5dd3dc60c6267318a1c9475efe85711f27fa/magicmethods.pdf -------------------------------------------------------------------------------- /magicmethods.py: -------------------------------------------------------------------------------- 1 | """ 2 | magicmethods.py 3 | Want to try out the examples? Don't want to type them up yourself? Never worry, 4 | magicmethods.py is a convenient Python module with all the class definitions 5 | for the examples in the magic methods guide in it. 6 | """ 7 | 8 | # FileObject class, demonstrating __init__ and __del__ 9 | from os.path import join 10 | 11 | class FileObject: 12 | """Wrapper for file objects to make sure the file gets closed on deletion.""" 13 | 14 | def __init__(self, filepath="~", filename="sample.txt"): 15 | # open a file filename in filepath in read and write mode 16 | self.file = open(join(filepath, filename), "r+") 17 | 18 | def __del__(self): 19 | self.file.close() 20 | del self.file 21 | 22 | # Word class, demonstrating __new__, comparisons 23 | class Word(str): 24 | """Class for words, defining comparison based on word length.""" 25 | 26 | def __new__(cls, word): 27 | # Note that we have to use __new__. This is because str is an immutable 28 | # type, so we have to initialize it early (at creation) 29 | if " " in word: 30 | print "Value contains spaces. Truncating to first space." 31 | word = word[:word.index(" ")] # Word is now all chars before first space 32 | return str.__new__(cls, word) 33 | 34 | def __gt__(self, other): 35 | return len(self) > len(other) 36 | def __lt__(self, other): 37 | return len(self) < len(other) 38 | def __ge__(self, other): 39 | return len(self) >= len(other) 40 | def __le__(self, other): 41 | return len(self) <= len(other) 42 | 43 | # AccessCounter class, demonstrating __setattr__, __getattr__, and __delattr__ 44 | class AccessCounter: 45 | """A class that contains a value and implements an access counter. 46 | The counter increments each time the value is changed.""" 47 | 48 | def __init__(self, val): 49 | self.__dict__["counter"] = 0 50 | self.__dict__["value"] = val 51 | 52 | def __setattr__(self, name, value): 53 | if name == "value": 54 | self.__dict__["counter"] += 1 55 | self.__dict__["value"] = value 56 | 57 | def __delattr__(self, name): 58 | if name == "value": 59 | self.__dict__["counter"] += 1 60 | del self.__dict__["value"] 61 | 62 | # FunctionalList class, demonstrating __len__, __getitem__, __setitem__, __delitem__, 63 | # __iter__, and __reversed__ 64 | class FunctionalList: 65 | """A class wrapping a list with some extra functional magic, like head, 66 | tail, init, last, drop, and take.""" 67 | 68 | def __init__(self, values=None): 69 | if values is None: 70 | self.values = [] 71 | else: 72 | self.values = values 73 | 74 | def __len__(self): 75 | return len(self.values) 76 | 77 | def __getitem__(self, key): 78 | # if key is of invalid type or value, the list values will raise the error 79 | return self.values[key] 80 | 81 | def __setitem__(self, key, value): 82 | self.values[key] = value 83 | 84 | def __delitem__(self, key): 85 | del self.values[key] 86 | 87 | def __iter__(self): 88 | return iter(self.values) 89 | 90 | def __reversed__(self): 91 | return reversed(self.values) 92 | 93 | def append(self, value): 94 | self.values.append(value) 95 | def head(self): 96 | # get the first element 97 | return self.values[0] 98 | def tail(self): 99 | # get all elements after the first 100 | return self.values[1:] 101 | def init(self): 102 | # get elements up to the last 103 | return self.values[:-1] 104 | def last(self): 105 | # get last element 106 | return self.values[-1] 107 | def drop(self, n): 108 | # get all elements except first n 109 | return self.values[n:] 110 | def take(self, n): 111 | # get first n elements 112 | return self.values[:n] 113 | 114 | # Entity class demonstrating __call__ 115 | class Entity: 116 | """Class to represent an entity. Callable to update the entity"s position.""" 117 | 118 | def __init__(self, size, x, y): 119 | self.x, self.y = x, y 120 | self.size = size 121 | 122 | def __call__(self, x, y): 123 | """Change the position of the entity.""" 124 | self.x, self.y = x, y 125 | 126 | # snip... 127 | 128 | # Wrapper class to close an object in a with statement 129 | class Closer: 130 | """A context manager to automatically close an object with a close method 131 | in a with statement.""" 132 | 133 | def __init__(self, obj): 134 | self.obj = obj 135 | 136 | def __enter__(self): 137 | return self.obj # bound to target 138 | 139 | def __exit__(self, exception_type, exception_val, trace): 140 | try: 141 | self.obj.close() 142 | except AttributeError: # obj isn"t closable 143 | print "Not closable." 144 | return True # exception handled successfully 145 | 146 | # Classes to represent descriptors and their use 147 | class Meter(object): 148 | """Descriptor for a meter.""" 149 | 150 | def __init__(self, value=0.0): 151 | self.value = float(value) 152 | def __get__(self, instance, owner): 153 | return self.value 154 | def __set__(self, instance, value): 155 | self.value = float(value) 156 | 157 | class Foot(object): 158 | """Descriptor for a foot.""" 159 | 160 | def __get__(self, instance, owner): 161 | return instance.meter * 3.2808 162 | def __set__(self, instance, value): 163 | instance.meter = float(value) / 3.2808 164 | 165 | class Distance(object): 166 | """Class to represent distance holding two descriptors for feet and 167 | meters.""" 168 | meter = Meter() 169 | foot = Foot() 170 | 171 | # Class to demo fine-tuning pickling 172 | import time 173 | 174 | class Slate: 175 | """Class to store a string and a changelog, and forget its value when 176 | pickled.""" 177 | 178 | def __init__(self, value): 179 | self.value = value 180 | self.last_change = time.asctime() 181 | self.history = {} 182 | 183 | def change(self, new_value): 184 | # Change the value. Commit last value to history 185 | self.history[self.last_change] = self.value 186 | self.value = new_value 187 | self.last_change = time.asctime() 188 | 189 | def print_changes(self): 190 | print "Changelog for Slate object:" 191 | for k, v in self.history.items(): 192 | print "%s\t %s" % (k, v) 193 | 194 | def __getstate__(self): 195 | # Deliberately do not return self.value or self.last_change. 196 | # We want to have a "blank slate" when we unpickle. 197 | return self.history 198 | 199 | def __setstate__(self, state): 200 | # Make self.history = state and last_change and value undefined 201 | self.history = state 202 | self.value, self.last_change = None, None 203 | 204 | 205 | -------------------------------------------------------------------------------- /magicmethods.tex: -------------------------------------------------------------------------------- 1 | \documentclass[a4paper,11pt]{article} 2 | 3 | \title{A Guide to Python's Magic Methods} 4 | \author{Rafe Kettler} 5 | \date{\today} 6 | 7 | \usepackage{fullpage} 8 | \usepackage{underscore} 9 | \usepackage{listings} 10 | \lstloadlanguages{Python} 11 | \lstset{ 12 | language = Python, 13 | basicstyle = \ttfamily\footnotesize, 14 | keepspaces = true, 15 | showstringspaces = false 16 | } 17 | 18 | \newcommand{\code}[1]{\texttt{#1}} 19 | 20 | \begin{document} 21 | \maketitle 22 | 23 | \section{Introduction} 24 | 25 | This guide is the culmination of a few months' worth of blog posts. The subject is \textbf{magic methods}. 26 | 27 | What are magic methods? They're everything in object-oriented Python. They're special methods that you can define to add "magic" to your classes. They're always surrounded by double underscores (e.g. \code{__init__} or \code{__lt__}). They're also not as well documented as they need to be. All of the magic methods for Python appear in the same section in the Python docs, but they're scattered about and only loosely organized. There's hardly an example to be found in that section (and that may very well be by design, since they're all detailed in the \emph{language reference}, along with boring syntax descriptions, etc.). 28 | 29 | So, to fix what I perceived as a flaw in Python's documentation, I set out to provide some more plain-English, example-driven documentation for Python's magic methods. I started out with weekly blog posts, and now that I've finished with those, I've put together this guide. 30 | 31 | I hope you enjoy it. Use it as a tutorial, a refresher, or a reference; it's just intended to be a user-friendly guide to Python's magic methods. 32 | 33 | \section{Construction and Initialization} 34 | 35 | Everyone knows the most basic magic method, \code{__init__}. It's the way that we can define the initialization behavior of an object. However, when I call \code{x = SomeClass()}, \code{__init__} is not the first thing to get called. Actually, it's a method called \code{__new__}, which actually creates the instance, then passes any arguments at creation on to the initializer. At the other end of the object's lifespan, there's \code{__del__}. Let's take a closer look at these 3 magic methods: 36 | 37 | \begin{description} 38 | 39 | \item[\code{__new__(cls, [...)}] 40 | \code{__new__} is the first method to get called in an object's instantiation. It takes the class, then any other arguments that it will pass along to \code{__init__}. \code{__new__} is used fairly rarely, but it does have its purposes, particularly when subclassing an immutable type like a tuple or a string. I don't want to go in to too much detail on \code{__new__} because it's not too useful, but it is covered in great detail in the Python docs. 41 | \item[\code{__init__(self, [...)}] 42 | The initializer for the class. It gets passed whatever the primary constructor was called with (so, for example, if we called \code{x = SomeClass(10, 'foo')}, \code{__init__} would get passed \code{10} and \code{'foo'} as arguments. \code{__init__} is almost universally used in Python class definitions. 43 | \item[\code{__del__(self)}] 44 | If \code{__new__} and \code{__init__} formed the constructor of the object, \code{__del__} is the destructor. It doesn't implement behavior for the statement \code{del x} (so that code would not translate to \code{x.__del__()}). Rather, it defines behavior for when an object is garbage collected. It can be quite useful for objects that might require extra cleanup upon deletion, like sockets or file objects. Be careful, however, as there is no guarantee that \code{__del__} will be executed if the object is still alive when the interpreter exits, so \code{__del__} can't serve as a replacement for good coding practices (like always closing a connection when you're done with it. In fact, \code{__del__} should almost never be used because of the precarious circumstances under which it is called; use it with caution! 45 | 46 | \end{description} 47 | 48 | \noindent 49 | Putting it all together, here's an example of \code{__init__} and \code{__del__} in action: 50 | 51 | \lstinputlisting{listings/fileobject.py} 52 | 53 | \section{Making Operators Work on Custom Classes} 54 | 55 | One of the biggest advantages of using Python's magic methods is that they provide a simple way to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, and nonstandard ways of performing basic operators. In some languages, it's common to do something like this: 56 | 57 | \begin{lstlisting} 58 | if instance.equals(other_instance): 59 | # do something 60 | \end{lstlisting} 61 | 62 | \noindent 63 | You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. Different libraries might use different names for the same operations, making the client do way more work than necessary. With the power of magic methods, however, we can define one method (\code{__eq__}, in this case), and say what we \emph{mean} instead: 64 | 65 | \begin{lstlisting} 66 | if instance == other_instance: 67 | #do something 68 | \end{lstlisting} 69 | 70 | \noindent 71 | That's part of the power of magic methods. The vast majority of them allow us to define meaning for operators so that we can use them on our own classes just like they were built in types. 72 | 73 | \subsection{Comparison magic methods} 74 | 75 | Python has a whole slew of magic methods designed to implement intuitive comparisons between objects using operators, not awkward method calls. They also provide a way to override the default Python behavior for comparisons of objects (by reference). Here's the list of those methods and what they do: 76 | 77 | \begin{description} 78 | 79 | \item[\code{__cmp__(self, other)}] 80 | : \code{__cmp__} is the most basic of the comparison magic methods. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.), but it might not do it the way you want (for example, if whether one instance was equal to another were determined by one criterion and and whether an instance is greater than another were determined by something else). \code{__cmp__} should return a negative integer if \code{self < other}, zero if \code{self == other}, and positive if \code{self > other}. It's usually best to define each comparison you need rather than define them all at once, but \code{__cmp__} can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria. 81 | 82 | \item[\code{__eq__(self, other)}] 83 | Defines behavior for the equality operator, \code{==}. 84 | \item[\code{__ne__(self, other)}] 85 | Defines behavior for the inequality operator, \code{!=}. 86 | \item[\code{__lt__(self, other)}] 87 | Defines behavior for the less-than operator, \code{<}. 88 | \item[\code{__gt__(self, other)}] 89 | Defines behavior for the greater-than operator, \code{>}. 90 | \item[\code{__le__(self, other)}] 91 | Defines behavior for the less-than-or-equal-to operator, \code{<=}. 92 | \item[\code{__ge__(self, other)}] 93 | Defines behavior for the greater-than-or-equal-to operator, \code{>=}. 94 | 95 | \end{description} 96 | 97 | For an example, consider a class to model a word. We might want to compare words lexicographically (by the alphabet), which is the default comparison behavior for strings, but we also might want to do it based on some other criterion, like length or number of syllables. In this example, we'll compare by length. Here's an implementation: 98 | 99 | \lstinputlisting{listings/word.py} 100 | 101 | Now, we can create two \code{Word}s (by using \code{Word('foo')} and \code{Word('bar')}) and compare them based on length. Note, however, that we didn't define \code{__eq__} and \code{__ne__}. This is because this would lead to some weird behavior (notably that \code{Word('foo') == Word('bar')} would evaluate to true). It wouldn't make sense to test for equality based on length, so we fall back on \code{str}'s implementation of equality. 102 | 103 | Now would be a good time to note that you don't have to define every comparison magic method to get rich comparisons. The standard library has kindly provided us with a class decorator in the module \code{functools} that will define all rich comparison methods if you only define \code{__eq__} and one other (e.g. \code{__gt__}, \code{__lt__}, etc.) This feature is only available in Python 2.7, but when you get a chance it saves a great deal of time and effort. You can use it by placing \code{@total_ordering} above your class definition. 104 | 105 | \subsection{Numeric Magic Methods} 106 | 107 | Just like you can create ways for instances of your class to be compared with comparison operators, you can define behavior for numeric operators. Buckle your seat belts, folks, there's a lot of these. For organization's sake, I've split the numeric magic methods into 5 categories: unary operators, normal arithmetic operators, reflected arithmetic operators (more on this later), augmented assignment, and type conversions. 108 | 109 | \subsubsection{Unary operators and functions} 110 | 111 | Unary operators and functions only have one operand, e.g. negation, absolute value, etc. 112 | 113 | \begin{description} 114 | 115 | \item[\code{__pos__(self)}] 116 | Implements behavior for unary positive (e.g. \code{+some_object}) 117 | \item[\code{__neg__(self)}] 118 | Implements behavior for negation (e.g. \code{-some_object}) 119 | \item[\code{__abs__(self)}] 120 | Implements behavior for the built in \code{abs()} function. 121 | \item[\code{__invert__(self)}] 122 | Implements behavior for inversion using the \code{\char126} operator. 123 | \item[\code{__round__(self, n)}] 124 | Implements behavior for the buil in \code{round()} function. \code{n} is the number of decimal places to round to. 125 | \item[\code{__floor__(self)}] 126 | : Implements behavior for \code{math.floor()}, i.e., rounding down to the nearest integer. 127 | \item[\code{__ceil__(self)}] 128 | : Implements behavior for \code{math.ceil()}, i.e., rounding up to the nearest integer. 129 | \item[\code{__trunc__(self)}] 130 | : Implements behavior for \code{math.trunc()}, i.e., truncating to an integral. 131 | 132 | \end{description} 133 | 134 | \subsection{Normal arithmetic operators} 135 | 136 | Now, we cover the typical binary operators (and a function or two): +, -, * and the like. These are, for the most part, pretty self-explanatory. 137 | 138 | \begin{description} 139 | 140 | \item[\code{__add__(self, other)}] 141 | Implements addition. 142 | \item[\code{__sub__(self, other)}] 143 | Implements subtraction. 144 | \item[\code{__mul__(self, other)}] 145 | Implements multiplication. 146 | \item[\code{__floordiv__(self, other)}] 147 | Implements integer division using the \code{//} operator. 148 | \item[\code{__div__(self, other)}] 149 | Implements division using the \code{/} operator. 150 | \item[\code{__truediv__(self, other)}] 151 | Implements _true_ division. Note that this only works when \code{from __future__ import division} is in effect. 152 | \item[\code{__mod__(self, other)}] 153 | Implements modulo using the \code{\%} operator. 154 | \item[\code{__divmod__(self, other)}] 155 | Implements behavior for long division using the \code{divmod()} built in function. 156 | \item[\code{__pow__}] 157 | Implements behavior for exponents using the \code{**} operator. 158 | \item[\code{__lshift__(self, other)}] 159 | Implements left bitwise shift using the \code{<<} operator. 160 | \item[\code{__rshift__(self, other)}] 161 | Implements right bitwise shift using the \code{>>} operator. 162 | \item[\code{__and__(self, other)}] 163 | Implements bitwise and using the \code{\&} operator. 164 | \item[\code{__or__(self, other)}] 165 | Implements bitwise or using the \code{|} operator. 166 | \item[\code{__xor__(self, other)}] 167 | Implements bitwise xor using the \code{\char94} operator. 168 | 169 | \end{description} 170 | 171 | \subsubsection{Reflected arithmetic operators} 172 | 173 | You know how I said I would get to reflected arithmetic in a bit? Some of you might think it's some big, scary, foreign concept. It's actually quite simple. Here's an example: 174 | 175 | \begin{lstlisting} 176 | some_object + other 177 | \end{lstlisting} 178 | 179 | \noindent 180 | That was "normal" addition. The reflected equivalent is the same thing, except with the operands switched around: 181 | 182 | \begin{lstlisting} 183 | other + some_object 184 | \end{lstlisting} 185 | 186 | \noindent 187 | So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around. In most cases, the result of a reflected operation is the same as its normal equivalent, so you may just end up defining \code{__radd__} as calling \code{__add__} and so on. Note that the object on the left hand side of the operator (\code{other} in the example) must not define (or return \code{NotImplemented}) for its definition of the non-reflected version of an operation. For instance, in the example, \code{some_object.__radd__} will only be called if `other` does not define \code{__add__}. 188 | 189 | \begin{description} 190 | 191 | \item[\code{__radd__(self, other)}] 192 | Implements reflected addition. 193 | \item[\code{__rsub__(self, other)}] 194 | Implements reflected subtraction. 195 | \item[\code{__rmul__(self, other)}] 196 | Implements reflected multiplication. 197 | \item[\code{__rfloordiv__(self, other)}] 198 | Implements reflected integer division using the \code{//} operator. 199 | \item[\code{__rdiv__(self, other)}] 200 | Implements reflected division using the \code{/} operator. 201 | \item[\code{__rtruediv__(self, other)}] 202 | Implements reflected _true_ division. Note that this only works when \code{from __future__ import division} is in effect. 203 | \item[\code{__rmod__(self, other)}] 204 | Implements reflected modulo using the \code{\%} operator. 205 | \item[\code{__rdivmod__(self, other)}] 206 | Implements behavior for long division using the \code{divmod()} built in function, when \code{divmod(other, self)} is called. 207 | \item[\code{__rpow__}] 208 | Implements behavior for reflected exponents using the \code{**} operator. 209 | \item[\code{__rlshift__(self, other)}] 210 | Implements reflected left bitwise shift using the \code{<<} operator. 211 | \item[\code{__rrshift__(self, other)}] 212 | Implements reflected right bitwise shift using the \code{>>} operator. 213 | \item[\code{__rand__(self, other)}] 214 | Implements reflected bitwise and using the \code{\&} operator. 215 | \item[\code{__ror__(self, other)}] 216 | Implements reflected bitwise or using the \code{|} operator. 217 | \item[\code{__rxor__(self, other)}] 218 | Implements reflected bitwise xor using the \code{\char94} operator. 219 | 220 | \end{description} 221 | 222 | \subsubsection{Augmented assignment} 223 | 224 | Python also has a wide variety of magic methods to allow custom behavior to be defined for augmented assignment. You're probably already familiar with augmented assignment, it combines ``normal'' operators with assignment. If you still don't know what I'm talking about, here's an example: 225 | 226 | \begin{lstlisting} 227 | x = 5 228 | x += 1 # in other words x = x + 1 229 | \end{lstlisting} 230 | 231 | Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for \code{a += b}, \code{__iadd__} might return \code{a + b}, which would be assigned to \code{a}). Here's the list: 232 | 233 | \begin{description} 234 | 235 | \item[\code{__iadd__(self, other)}] 236 | Implements addition with assignment. 237 | \item[\code{__isub__(self, other)}] 238 | Implements subtraction with assignment. 239 | \item[\code{__imul__(self, other)}] 240 | Implements multiplication with assignment. 241 | \item[\code{__ifloordiv__(self, other)}] 242 | Implements integer division with assignment using the \code{//=} operator. 243 | \item[\code{__idiv__(self, other)}] 244 | Implements division with assignment using the \code{/=} operator. 245 | \item[\code{__itruediv__(self, other)}] 246 | Implements _true_ division with assignment. Note that this only works when \code{from __future__ import division} is in effect. 247 | \item[\code{__imod_(self, other)}] 248 | Implements modulo with assignment using the \code{\%=} operator. 249 | \item[\code{__ipow__}] 250 | Implements behavior for exponents with assignment using the \code{**=} operator. 251 | \item[\code{__ilshift__(self, other)}] 252 | Implements left bitwise shift with assignment using the \code{<<=} operator. 253 | \item[\code{__irshift__(self, other)}] 254 | Implements right bitwise shift with assignment using the \code{>>=} operator. 255 | \item[\code{__iand__(self, other)}] 256 | Implements bitwise and with assignment using the \code{\&=} operator. 257 | \item[\code{__ior__(self, other)}] 258 | Implements bitwise or with assignment using the \code{|=} operator. 259 | \item[\code{__ixor__(self, other)}] 260 | Implements bitwise xor with assignment using the \code{\char94=} operator. 261 | 262 | \end{description} 263 | 264 | \subsubsection{Type conversion magic methods} 265 | 266 | Python also has an array of magic methods designed to implement behavior for built in type conversion functions like \code{float()}. Here they are: 267 | 268 | \begin{description} 269 | 270 | \item[\code{__int__(self)}] 271 | Implements type conversion to int. 272 | \item[\code{__long__(self)}] 273 | Implements type conversion to long. 274 | \item[\code{__float__(self)}] 275 | Implements type conversion to float. 276 | \item[\code{__complex__(self)}] 277 | Implements type conversion to complex. 278 | \item[\code{__oct__(self)}] 279 | Implements type conversion to octal. 280 | \item[\code{__hex__(self)}] 281 | Implements type conversion to hexadecimal. 282 | \item[\code{__index__(self)}] 283 | Implements type conversion to an int when the object is used in a slice expression. If you define a custom numeric type that might be used in slicing, you should define \code{__index__}. 284 | \item[\code{__trunc__(self)}] 285 | Called when \code{math.trunc(self)} is called. \code{__trunc__} should return the value of \code{self} truncated to an integral type (usually a long). 286 | \item[\code{__coerce__(self, other)}] 287 | Method to implement mixed mode arithmetic. \code{__coerce__} should return \code{None} if type conversion is impossible. Otherwise, it should return a pair (2-tuple) of \code{self} and \code{other}, manipulated to have the same type. 288 | 289 | \end{description} 290 | 291 | \section{Representing your Classes} 292 | 293 | It's often useful to have a string representation of a class. In Python, there's a few methods that you can implement in your class definition to customize how built in functions that return representations of your class behave. 294 | 295 | \begin{description} 296 | 297 | \item[\code{__str__(self)}] 298 | Defines behavior for when \code{str()} is called on an instance of your class. 299 | \item[\code{__repr__(self)}] 300 | Defines behavior for when \code{repr()} is called on an instance of your class. The major difference between \code{str()} and \code{repr()} is intended audience. \code{repr()} is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereas \code{str()} is intended to be human-readable. 301 | \item[\code{__unicode__(self)}] 302 | Defines behavior for when \code{unicode()} is called on an instance of your class. \code{unicode()} is like \code{str()}, but it returns a unicode string. Be wary: if a client calls \code{str()} on an instance of your class and you've only defined \code{__unicode__()}, it won't work. You should always try to define \code{__str__()} as well in case someone doesn't have the luxury of using unicode. 303 | \item[\code{__format__(self, formatstr)}] 304 | Defines behavior for when an instance of your class is used in new-style string formatting. For instance, \code{"Hello, {0:abc}!".format(a)} would lead to the call \code{a.__format__("abc")}. This can be useful for defining your own numerical or string types that you might like to give special formatting options. 305 | \item[\code{__hash__(self)}] 306 | Defines behavior for when \code{hash()} is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing \code{__eq__} as well. Live by the following rule: \code{a == b} implies \code{hash(a) == hash(b)}. 307 | \item[\code{__nonzero__(self)}] 308 | Defines behavior for when \code{bool()} is called on an instance of your class. Should return True or False, depending on whether you would want to consider the instance to be True or False. 309 | \item[\code{__dir__(self)}] 310 | : Defines behavior for when \code{dir()} is called on an instance of your class. This method should return a list of attributes for the user. Typically, implementing \code{__dir__} is unnecessary, but it can be vitally important for interactive use of your classes if you redefine \code{__getattr__} or \code{__getattribute__} (which you will see in the next section) or are otherwise dynamically generating attributes. 311 | 312 | \end{description} 313 | 314 | \noindent 315 | We're pretty much done with the boring (and example-free) part of the magic methods guide. Now that we've covered some of the more basic magic methods, it's time to move to more advanced material. 316 | 317 | \section{Controlling Attribute Access} 318 | 319 | Many people coming to Python from other languages complain that it lacks true encapsulation for classes (e.g. no way to define private attributes and then have public getter and setters). This couldn't be farther than the truth: it just happens that Python accomplishes a great deal of encapsulation through ``magic'', instead of explicit modifiers for methods or fields. Take a look: 320 | 321 | \begin{description} 322 | 323 | \item[\code{__getattr__(self, name)}] 324 | You can define behavior for when a user attempts to access an attribute that doesn't exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing an \code{AttributeError}. It only gets called when a nonexistent attribute is accessed, however, so it isn't a true encapsulation solution. 325 | 326 | \item[\code{__setattr__(self, name, value)}] 327 | Unlike \code{__getattr__}, \code{__setattr__} is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes. However, you have to be careful with how you use \code{__setattr__}, as the example at the end of the list will show. 328 | 329 | \item[\code{__delattr__}] 330 | This is the exact same as \code{__setattr__}, but for deleting attributes instead of setting them. The same precautions need to be taken as with \code{__setattr__} as well in order to prevent infinite recursion (calling \code{del self.name} in the implementation of \code{__delattr__} would cause infinite recursion). 331 | 332 | \item[\code{__getattribute__(self, name)}] 333 | After all this, \code{__getattribute__} fits in pretty well with its companions \code{__setattr__} and \code{__delattr__}. However, I don't recommend you use it. \code{__getattribute__} can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassing \code{object}. It allows you to define rules for whenever an attribute's value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class's \code{__getattribute__} method to prevent this). It also mainly obviates the need for \code{__getattr__}, which only gets called when \code{__getattribute__} is implemented if it is called explicitly or an \code{AttributeError} is raised. This method can be used (after all, it's your choice), but I don't recommend it because it has a small use case (it's far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free. 334 | 335 | \end{description} 336 | 337 | You can easily cause a problem in your definitions of any of the methods controlling attribute access. Consider this example: 338 | 339 | \begin{lstlisting} 340 | def __setattr__(self, name, value): 341 | self.name = value 342 | # since every time an attribute is assigned, __setattr__() 343 | # is called, this is recursion. So this really means 344 | # self.__setattr__('name', value). Since the method keeps 345 | # calling itself, the recursion goes on forever causing a crash 346 | 347 | def __setattr__(self, name, value): 348 | self.__dict__[name] = value # assigning to the dict of names in the class 349 | # define custom behavior here 350 | \end{lstlisting} 351 | 352 | Again, Python's magic methods are incredibly powerful, and with great power comes great responsibility. It's important to know the proper way to use magic methods so you don't break any code. 353 | 354 | So, what have we learned about custom attribute access in Python? It's not to be used lightly. In fact, it tends to be excessively powerful and counter-intuitive. But the reason why it exists is to scratch a certain itch: Python doesn't seek to make bad things impossible, but just to make them difficult. Freedom is paramount, so you can really do whatever you want. Here's an example of some of the special attribute access methods in action (note that we use \code{super} because not all classes have an attribute \code{__dict__}): 355 | 356 | \lstinputlisting{listings/access.py} 357 | 358 | \section{Making Custom Sequences} 359 | 360 | There's a number of ways to get your Python classes to act like built in sequences (dict, tuple, list, string, etc.). These are by far my favorite magic methods in Python because of the absurd degree of control they give you and the way that they magically make a whole array of global functions work beautifully on instances of your class. But before we get down to the good stuff, a quick word on requirements. 361 | 362 | \subsection{Requirements} 363 | 364 | Now that we're talking about creating your own sequences in Python, it's time to talk about _protocols_. Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they're more like guidelines. 365 | 366 | Why are we talking about protocols now? Because implementing custom container types in Python involves using some of these protocols. First, there's the protocol for defining immutable containers: to make an immutable container, you need only define \code{__len__} and \code{__getitem__} (more on these later). The mutable container protocol requires everything that immutable containers require plus \code{__setitem__} and \code{__delitem__}. Lastly, if you want your object to be iterable, you'll have to define \code{__iter__}, which returns an iterator. That iterator must conform to an iterator protocol, which requires iterators to have methods called \code{__iter__}(returning itself) and \code{next}. 367 | 368 | 369 | \subsection{The magic behind containers} 370 | \begin{description} 371 | 372 | \item[\code{__len__(self)}] 373 | Returns the length of the container. Part of the protocol for both immutable and mutable containers. 374 | \item[\code{__getitem__(self, key)}] 375 | Defines behavior for when an item is accessed, using the notation \code{self[key]}. This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions: \code{TypeError} if the type of the key is wrong and \code{KeyError} if there is no corresponding value for the key. 376 | \item[\code{__setitem__(self, key, value)}] 377 | Defines behavior for when an item is assigned to, using the notation \code{self[key] = value}. This is part of the mutable container protocol. Again, you should raise \code{KeyError} and \code{TypeError} where appropriate. 378 | \item[\code{__delitem__(self, key)}] 379 | Defines behavior for when an item is deleted (e.g. \code{del self[key]}). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used. 380 | \item[\code{__iter__(self)}] 381 | Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by the \code{iter()} built in function and when a container is looped over using the form \code{for x in container:}. Iterators are their own objects, and they also must define an \code{__iter__} method that returns \code{self}. 382 | \item[\code{__reversed__(self)}] 383 | Called to implement behavior for the \code{reversed()} built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple. 384 | \item[\code{__contains__(self, item)}] 385 | \code{__contains__} defines behavior for membership tests using \code{in} and \code{not in}. Why isn't this part of a sequence protocol, you ask? Because when \code{__contains__} isn't defined, Python just iterates over the sequence and returns \code{True} if it comes across the item it's looking for. 386 | \item[\code{__missing__(self, key)}] 387 | \code{__missing__} is used in subclasses of \code{dict}. It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionary \code{d} and said \code{d["george"]} when \code{"george"} is not a key in the dict, \code{d.__missing__("george")} would be called). 388 | 389 | \end{description} 390 | 391 | \subsection{An example} 392 | 393 | 394 | For our example, let's look at a list that implements some functional constructs that you might be used to from other languages (Haskell, for example): 395 | 396 | \lstinputlisting{listings/list.py} 397 | 398 | \noindent 399 | There you have it, a (marginally) useful example of how to implement your own sequence. Of course, there are more useful applications of custom sequences, but quite a few of them are already implemented in the standard library (batteries included, right?), like \code{Counter}, \code{OrderedDict}, and \code{NamedTuple}. 400 | 401 | \section{Reflection} 402 | 403 | You can also control how reflection works using the built in functions \code{isinstance} and \code{issubclass()} behaves by defining magic methods. The magic methods are: 404 | 405 | \begin{description} 406 | 407 | \item[\code{__instancecheck__(self, instance)}] 408 | Checks if an instance is an instance of the class you defined (e.g. \code{isinstance(instance, class)}. 409 | \item[\code{__subclasscheck__(self, subclass)}] 410 | Checks if a class subclasses the class you defined (e.g. \code{issubclass(subclass, class)}). 411 | 412 | \end{description} 413 | 414 | The use case for these magic methods might seem small, and that may very well be true. I won't spend too much more time on reflection magic methods because they aren't very important, but they reflect something important about object-oriented programming in Python and Python in general: there is almost always an easy way to do something, even if it's rarely necessary. These magic methods might not seem useful, but if you ever need them you'll be glad that they're there (and that you read this guide!). 415 | 416 | \section{Abstract Base Classes} 417 | 418 | See http://docs.python.org/2/library/abc.html. 419 | 420 | \section{Callable Objects} 421 | 422 | As you may already know, in Python, functions are first-class objects. This means that they can be passed to functions and methods just as if they were objects of any other kind. This is an incredibly powerful feature. 423 | 424 | A special magic method in Python allows instances of your classes to behave as if they were functions, so that you can ``call'' them, pass them to functions that take functions as arguments, and so on. This is another powerful convenience feature that makes programming in Python that much sweeter. 425 | 426 | \begin{description} 427 | 428 | \item[\code{__call__(self, [args...])}] 429 | Allows an instance of a class to be called as a function. Essentially, this means that \code{x()} is the same as \code{x.__call__()}. Note that \code{__call__} takes a variable number of arguments; this means that you define \code{__call__} as you would any other function, taking however many arguments you'd like it to. 430 | 431 | \end{description} 432 | 433 | \code{__call__} can be particularly useful in classes whose instances that need to often change state. ``Calling'' the instance can be an intuitive and elegant way to change the object's state. An example might be a class representing an entity's position on a plane: 434 | 435 | \lstinputlisting{listings/call.py} 436 | 437 | \section{Context Managers} 438 | 439 | In Python 2.5, a new keyword was introduced in Python along with a new method for code reuse, the \code{with} statement. The concept of context managers was hardly new in Python (it was implemented before as a part of the library), but not until PEP 343 was accepted did it achieve status as a first class language construct. You may have seen with statements before: 440 | 441 | \begin{lstlisting} 442 | with open('foo.txt') as bar: 443 | # perform some action with bar 444 | \end{lstlisting} 445 | 446 | Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a \code{with} statement. The behavior of the context manager is determined by two magic methods: 447 | 448 | \begin{description} 449 | 450 | \item[\code{__enter__(self)}] 451 | Defines what the context manager should do at the beginning of the block created by the \code{with} statement. Note that the return value of \code{__enter__} is bound to the \emph{target} of the \code{with} statement, or the name after the \code{as}. 452 | \item[\code{__exit__(self, exception_type, exception_value, traceback)}] 453 | Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, \code{exception_type}, \code{exception_value}, and \code{traceback} will be \code{None}. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure \code{__exit__} returns \code{True} after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen. 454 | 455 | \end{description} 456 | 457 | \code{__enter__} and \code{__exit__} can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects. Here's an example: 458 | 459 | \lstinputlisting{listings/closer.py} 460 | 461 | \noindent 462 | Here's an example of \code{Closer} in action, using an FTP connection to demonstrate it (a closable socket): 463 | 464 | \begin{lstlisting} 465 | >>> from magicmethods import Closer 466 | >>> from ftplib import FTP 467 | >>> with Closer(FTP('ftp.somesite.com')) as conn: 468 | ... conn.dir() 469 | ... 470 | # output omitted for brevity 471 | >>> conn.dir() 472 | # long AttributeError message, can't use a connection that's closed 473 | >>> with Closer(int(5)) as i: 474 | ... i += 1 475 | ... 476 | Not closable. 477 | >>> i 478 | 6 479 | \end{lstlisting} 480 | 481 | \noindent 482 | See how our wrapper gracefully handled both proper and improper uses? That's the power of context managers and magic methods. Note that the Python standard library includes a module \code{contextlib} that contains a context manager, \code{contextlib.closing()}, that does approximately the same thing (without any handling of the case where an object does not have a \code{close()} method). 483 | 484 | \section{Building Descriptor Objects} 485 | 486 | Descriptors are classes which, when accessed through either getting, setting, or deleting, can also alter other objects. Descriptors aren't meant to stand alone; rather, they're meant to be held by an owner class. Descriptors can be useful when building object-oriented databases or classes that have attributes whose values are dependent on each other. Descriptors are particularly useful when representing attributes in several different units of measurement or representing computed attributes (like distance from the origin in a class to represent a point on a grid). 487 | 488 | To be a descriptor, a class must have at least one of \code{__get__}, \code{__set__}, and \code{__delete__} implemented. 489 | 490 | Let's take a look at those magic methods: 491 | 492 | \begin{description} 493 | 494 | \item[\code{__get__(self, instance, owner)}] 495 | Define behavior for when the descriptor's value is retrieved. \code{instance} is the instance of the owner object. \code{owner} is the owner class itself. 496 | 497 | \item[\code{__set__(self, instance, value)}] 498 | Define behavior for when the descriptor's value is changed. \code{instance} is the instance of the owner class and \code{value} is the value to set the descriptor to. 499 | 500 | \item[\code{__delete__(self, instance)}] 501 | Define behavior for when the descriptor's value is deleted. \code{instance} is the instance of the owner object. 502 | 503 | \end{description} 504 | 505 | \noindent 506 | Now, an example of a useful application of descriptors: unit conversions. 507 | 508 | \lstinputlisting{listings/descriptor.py} 509 | 510 | \section{Copying} 511 | 512 | Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python's \code{copy} comes into play. However (fortunately), Python modules are not sentient, so we don't have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things. 513 | 514 | \code{__copy__(self)} 515 | : Defines behavior for \code{copy.copy()} for instances of your class. \code{copy.copy()} returns a _shallow copy_ of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original). 516 | 517 | \code{__deepcopy__(self, memodict={})} 518 | : Defines behavior for \code{copy.deepcopy()} for instances of your class. \code{copy.deepcopy()} returns a _deep copy_ of your object -- the object _and_ its data are both copied. \code{memodict} is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, call \code{copy.deepcopy()} on that attribute with \code{memodict} as the first argument. 519 | 520 | What are some use cases for these magic methods? As always, in any case where you need more fine-grained control than what the default behavior gives you. For instance, if you are attempting to copy an object that stores a cache as a dictionary (which might be large), it might not make sense to copy the cache as well -- if the cache can be shared in memory between instances, then it should be. 521 | 522 | \section{Pickling} 523 | 524 | If you spend time with other Pythonistas, chances are you've at least heard of pickling. Pickling is a serialization process for Python data structures, and can be incredibly useful when you need to store an object and retrieve it later. It's also a major source of worries and confusion. 525 | 526 | Pickling is so important that it doesn't just have its own module (\code{pickle}), but its own \emph{protocol} and the magic methods to go with it. But first, a brief word on how to pickle existing types(feel free to skip it if you already know). 527 | 528 | \subsection{Pickling: A Quick Soak in the Brine} 529 | 530 | Let's dive into pickling. Say you have a dictionary that you want to store and retrieve later. You could write it's contents to a file, carefully making sure that you write correct syntax, then retrieve it using either \code{exec()} or processing the file input. But this is precarious at best: if you store important data in plain text, it could be corrupted or changed in any number of ways to make your program crash or worse run malicious code on your computer. Instead, we're going to pickle it: 531 | 532 | \begin{lstlisting} 533 | import pickle 534 | 535 | data = {'foo': [1, 2, 3], 536 | 'bar': ('Hello', 'world!'), 537 | 'baz': True} 538 | jar = open('data.pkl', 'wb') 539 | pickle.dump(data, jar) # write the pickled data to the file jar 540 | jar.close() 541 | \end{lstlisting} 542 | 543 | \noindent 544 | Now, a few hours later, we want it back. All we have to do is unpickle it: 545 | 546 | \begin{lstlisting} 547 | import pickle 548 | 549 | pkl_file = open('data.pkl', 'rb') # connect to the pickled data 550 | data = pickle.load(pkl_file) # load it into a variable 551 | print data 552 | pkl_file.close() 553 | \end{lstlisting} 554 | 555 | \noindent 556 | What happens? Exactly what you expect. It's just like we had \code{data} all along. 557 | 558 | Now, for a word of caution: pickling is not perfect. Pickle files are easily corrupted on accident and on purpose. Pickling may be more secure than using flat text files, but it still can be used to run malicious code. It's also incompatible across versions of Python, so don't expect to distribute pickled objects and expect people to be able to open them. However, it can also be a powerful tool for caching and other common serialization tasks. 559 | 560 | \subsection{Pickling your own Objects} 561 | 562 | Pickling isn't just for built-in types. It's for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it's a bit different for C extensions, but that's not in our scope): 563 | 564 | \begin{description} 565 | 566 | \item[\code{__getinitargs__(self)}] 567 | If you'd like for \code{__init__} to be called when your class is unpickled, you can define \code{__getinitargs__}, which should return a tuple of the arguments that you'd like to be passed to \code{__init__}. Note that this method will only work for old-style classes. 568 | \item[\code{__getnewargs__(self)}] 569 | For new-style classes, you can influence what arguments get passed to \code{__new__} upon unpickling. This method should also return a tuple of arguments that will then be passed to \code{__new__}. 570 | \item[\code{__getstate__(self)}] 571 | Instead of the object's \code{__dict__} attribute being stored, you can return a custom state to be stored when the object is pickled. That state will be used by \code{__setstate__} when the object is unpickled. 572 | \item[\code{__setstate__(self, state)}] 573 | When the object is unpickled, if \code{__setstate__} is defined the object's state will be passed to it instead of directly applied to the object's \code{__dict__}. This goes hand in hand with \code{__getstate__}: when both are defined, you can represent the object's pickled state however you want with whatever you want. 574 | \item[\code{__reduce__(self)}] 575 | When defining extension types (i.e., types implemented using Python's C API), you have to tell Python how to pickle them if you want them to pickle them. \code{__reduce__()} is called when an object defining it is pickled. It can either return a string representing a global name that Python will look up and pickle, or a tuple. The tuple contains between 2 and 5 elements: a callable object that is called to recreate the object, a tuple of arguments for that callable object, state to be passed to \code{__setstate__} (optional), an iterator yielding list items to be pickled (optional), and an iterator yielding dictionary items to be pickled (optional). 576 | \item[\code{__reduce_ex__(self)}] 577 | \code{__reduce_ex__} exists for compatibility. If it is defined, \code{__reduce_ex__} will be called over \code{__reduce__} on pickling. \code{__reduce__} can be defined as well for older versions of the pickling API that did not support \code{__reduce_ex__}. 578 | \end{description} 579 | 580 | \subsection{An Example} 581 | 582 | Our example is a \code{Slate}, which remembers what its values have been and when those values were written to it. However, this particular slate goes blank each time it is pickled: the current value will not be saved. 583 | 584 | \lstinputlisting{listings/slate.py} 585 | 586 | \section{Conclusion} 587 | 588 | The goal of this guide is to bring something to anyone that reads it, regardless of their experience with Python or object-oriented programming. If you're just getting started with Python, you've gained valuable knowledge of the basics of writing feature-rich, elegant, and easy-to-use classes. If you're an intermediate Python programmer, you've probably picked up some slick new concepts and strategies and some good ways to reduce the amount of code written by you and clients. If you're an expert Pythonista, you've been refreshed on some of the stuff you might have forgotten about and maybe picked up a few new tricks along the way. Whatever your experience level, I hope that this trip through Python's special methods has been truly magical (I couldn't resist the final pun). 589 | 590 | % Begin the appendix 591 | \newpage 592 | \section{Appendix 1: How to Call Magic Methods} 593 | 594 | Some of the magic methods in Python directly map to built-in functions; in this case, how to invoke them is fairly obvious. However, in other cases, the invocation is far less obvious. This appendix is devoted to exposing non-obvious syntax that leads to magic methods getting called. 595 | \begin{center} 596 | \begin{tabular}{| p{5cm} | p{5cm} | p{5cm} |} 597 | \hline 598 | $Magic\ Method$ & $When\ it\ gets\ invoked$ & $Explanation$\\ 599 | \hline 600 | \code{__new__(cls [,...])} & \code{instance = MyClass(arg1, arg2)} & \code{__new__} is called on instance creation\\ 601 | \hline 602 | \code{__init__(self [,...])} & \code{instance = MyClass(arg1, arg2)} & \code{__init__} is called on instance creation\\ 603 | \hline 604 | \code{__cmp__(self, other)} & \code{self == other}, \code{self > other}, etc. & Called for any comparison\\ 605 | \hline 606 | \code{__pos__(self)} & \code{+self} & Unary plus sign\\ 607 | \hline 608 | \code{__neg__(self)} & \code{-self} & Unary minus sign\\ 609 | \hline 610 | \code{__invert__(self)} & \code{~self} & Bitwise inversion\\ 611 | \hline 612 | \code{__index__(self)} & \code{x[self]} & Conversion when object is used as index\\ 613 | \hline 614 | \code{__nonzero__(self)} & \code{bool(self)} & Boolean value of the object\\ 615 | \hline 616 | \code{__getattr__(self, name)} & \code{self.name \# name doesn't exist} & Accessing nonexistent attribute\\ 617 | \hline 618 | \code{__setattr__(self, name, val)} & \code{self.name = val} & Assigning to an attribute\\ 619 | \hline 620 | \code{__delattr__(self, name)} & \code{del self.name} & Deleting an attribute\\ 621 | \hline 622 | \code{__getattribute__(self, name)} & \code{self.name} & Accessing any attribute\\ 623 | \hline 624 | \code{__getitem__(self, key)} & \code{self[key]} & Accessing an item using an index\\ 625 | \hline 626 | \code{__setitem__(self, key, val)} & \code{self[key] = val} & Assigning to an item using an index\\ 627 | \hline 628 | \code{__delitem__(self, key)} & \code{del self[key]} & Deleting an item using an index\\ 629 | \hline 630 | \code{__iter__(self)} & \code{for x in self} & Iteration\\ 631 | \hline 632 | \code{__contains__(self, value)} & \code{value in self}, \code{value not in self} & Membership tests using \code{in}\\ 633 | \hline 634 | \code{__call__(self [,...])} & \code{self(args)} & "Calling" an instance\\ 635 | \hline 636 | \code{__enter__(self)} & \code{with self as x:} & \code{with} statement context managers\\ 637 | \hline 638 | \code{__exit__(self, exc, val, trace)} & \code{with self as x:} & \code{with} statement context managers\\ 639 | \hline 640 | \code{__getstate__(self)} & \code{pickle.dump(pkl_file, self)} & Pickling\\ 641 | \hline 642 | \code{__setstate__(self)} & \code{data = pickle.load(pkl_file)} & Pickling\\ 643 | \hline 644 | \end{tabular} 645 | \end{center} 646 | 647 | \section{Appendix 2: Changes in Python 3} 648 | 649 | Here, we document a few major places where Python 3 differs from 2.x in terms of its object model: 650 | 651 | \begin{itemize} 652 | \item Since the distinction between string and unicode has been done away with in Python 3, \code{__unicode__} is gone and \code{__bytes__} (which behaves similarly to \code{__str__} and \code{__unicode__} in 2.7) exists for a new built-in for constructing byte arrays. 653 | \item Since division defaults to true division in Python 3, \code{__div__} is gone in Python 3 654 | \item \code{__coerce__} is gone due to redundancy with other magic methods and confusing behavior 655 | \item \code{__cmp__} is gone due to redundancy with other magic methods 656 | \item \code{__nonzero__} has been renamed to \code{__bool__} 657 | \end{itemize} 658 | 659 | \end{document} 660 | -------------------------------------------------------------------------------- /style.css: -------------------------------------------------------------------------------- 1 | p, h1, h2, h3, h4, h5, h6, ul, ol, dl, pre, table { 2 | width: 800px; 3 | margin-left: auto; 4 | margin-right: auto; 5 | } 6 | 7 | table { border: 1px solid #ccc; 8 | background-color: #f5f5f5; 9 | padding: 5px 5px; 10 | width: 900px; 11 | } 12 | 13 | th, td { border: 1px solid #ccc;} 14 | 15 | h1, h2, h3, h4, h5, h6 { 16 | font-family: "Hoefler Text", serif; 17 | } 18 | 19 | body { 20 | background-color: #fafafa; 21 | color: #111; 22 | font-family: Georgia, serif; 23 | font-size: 17px; 24 | line-height: 23px; 25 | margin-top: 68px; /* 4em */ 26 | } 27 | 28 | pre,code { 29 | font: normal 15px/20px Menlo, consolas, mono; 30 | } 31 | 32 | pre { 33 | border: 1px solid #ddd; 34 | background-color: #f5f5f5; 35 | padding: 15px 20px; 36 | overflow-x: auto; 37 | } 38 | 39 | code { 40 | background-color: #f5f5f5; 41 | padding: 0px 4px; 42 | } 43 | 44 | a { 45 | text-decoration: none; 46 | color: #D63500; 47 | } 48 | 49 | h1 a, h2 a, h3 a, h4 a, h5 a, h6 a { 50 | color: inherit; 51 | } 52 | 53 | h1 a:hover, h2 a:hover, h3 a:hover, h4 a:hover, h5 a:hover, h6 a:hover { 54 | text-decoration: none; 55 | color: #D63500; 56 | } 57 | 58 | a:hover { 59 | text-decoration: underline; 60 | } 61 | 62 | dd { 63 | margin-bottom: 10px; 64 | } 65 | 66 | dl:first-of-type dd { 67 | margin-bottom: 0; 68 | } 69 | 70 | /* pygments CSS, adapted from Richard Leland's tango.css, found on github 71 | /* repo richleland/pygments-css */ 72 | 73 | .codehilite .hll { background-color: #ffffcc } 74 | .codehilite .c { color: #8f5902; font-style: italic } /* Comment */ 75 | .codehilite .err { color: #a40000; border: 1px solid #ef2929 } /* Error */ 76 | .codehilite .g { color: #000000 } /* Generic */ 77 | .codehilite .k { color: #204a87; font-weight: bold } /* Keyword */ 78 | .codehilite .l { color: #000000 } /* Literal */ 79 | .codehilite .n { color: #000000 } /* Name */ 80 | .codehilite .o { color: #ce5c00; font-weight: bold } /* Operator */ 81 | .codehilite .x { color: #000000 } /* Other */ 82 | .codehilite .p { color: #000000; font-weight: bold } /* Punctuation */ 83 | .codehilite .cm { color: #8f5902; font-style: italic } /* Comment.Multiline */ 84 | .codehilite .cp { color: #8f5902; font-style: italic } /* Comment.Preproc */ 85 | .codehilite .c1 { color: #8f5902; font-style: italic } /* Comment.Single */ 86 | .codehilite .cs { color: #8f5902; font-style: italic } /* Comment.Special */ 87 | .codehilite .gd { color: #a40000 } /* Generic.Deleted */ 88 | .codehilite .ge { color: #000000; font-style: italic } /* Generic.Emph */ 89 | .codehilite .gr { color: #ef2929 } /* Generic.Error */ 90 | .codehilite .gh { color: #000080; font-weight: bold } /* Generic.Heading */ 91 | .codehilite .gi { color: #00A000 } /* Generic.Inserted */ 92 | .codehilite .go { color: #000000; font-style: italic } /* Generic.Output */ 93 | .codehilite .gp { color: #8f5902 } /* Generic.Prompt */ 94 | .codehilite .gs { color: #000000; font-weight: bold } /* Generic.Strong */ 95 | .codehilite .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ 96 | .codehilite .gt { color: #a40000; font-weight: bold } /* Generic.Traceback */ 97 | .codehilite .kc { color: #204a87; font-weight: bold } /* Keyword.Constant */ 98 | .codehilite .kd { color: #204a87; font-weight: bold } /* Keyword.Declaration */ 99 | .codehilite .kn { color: #204a87; font-weight: bold } /* Keyword.Namespace */ 100 | .codehilite .kp { color: #204a87; font-weight: bold } /* Keyword.Pseudo */ 101 | .codehilite .kr { color: #204a87; font-weight: bold } /* Keyword.Reserved */ 102 | .codehilite .kt { color: #204a87; font-weight: bold } /* Keyword.Type */ 103 | .codehilite .ld { color: #000000 } /* Literal.Date */ 104 | .codehilite .m { color: #0000cf; font-weight: bold } /* Literal.Number */ 105 | .codehilite .s { color: #4e9a06 } /* Literal.String */ 106 | .codehilite .na { color: #c4a000 } /* Name.Attribute */ 107 | .codehilite .nb { color: #204a87 } /* Name.Builtin */ 108 | .codehilite .nc { color: #000000 } /* Name.Class */ 109 | .codehilite .no { color: #000000 } /* Name.Constant */ 110 | .codehilite .nd { color: #5c35cc; font-weight: bold } /* Name.Decorator */ 111 | .codehilite .ni { color: #ce5c00 } /* Name.Entity */ 112 | .codehilite .ne { color: #cc0000; font-weight: bold } /* Name.Exception */ 113 | .codehilite .nf { color: #000000 } /* Name.Function */ 114 | .codehilite .nl { color: #f57900 } /* Name.Label */ 115 | .codehilite .nn { color: #000000 } /* Name.Namespace */ 116 | .codehilite .nx { color: #000000 } /* Name.Other */ 117 | .codehilite .py { color: #000000 } /* Name.Property */ 118 | .codehilite .nt { color: #204a87; font-weight: bold } /* Name.Tag */ 119 | .codehilite .nv { color: #000000 } /* Name.Variable */ 120 | .codehilite .ow { color: #204a87; font-weight: bold } /* Operator.Word */ 121 | .codehilite .w { color: #f8f8f8; text-decoration: underline } /* Text.Whitespace */ 122 | .codehilite .mf { color: #0000cf; font-weight: bold } /* Literal.Number.Float */ 123 | .codehilite .mh { color: #0000cf; font-weight: bold } /* Literal.Number.Hex */ 124 | .codehilite .mi { color: #0000cf; font-weight: bold } /* Literal.Number.Integer */ 125 | .codehilite .mo { color: #0000cf; font-weight: bold } /* Literal.Number.Oct */ 126 | .codehilite .sb { color: #4e9a06 } /* Literal.String.Backtick */ 127 | .codehilite .sc { color: #4e9a06 } /* Literal.String.Char */ 128 | .codehilite .sd { color: #8f5902; font-style: italic } /* Literal.String.Doc */ 129 | .codehilite .s2 { color: #4e9a06 } /* Literal.String.Double */ 130 | .codehilite .se { color: #4e9a06 } /* Literal.String.Escape */ 131 | .codehilite .sh { color: #4e9a06 } /* Literal.String.Heredoc */ 132 | .codehilite .si { color: #4e9a06 } /* Literal.String.Interpol */ 133 | .codehilite .sx { color: #4e9a06 } /* Literal.String.Other */ 134 | .codehilite .sr { color: #4e9a06 } /* Literal.String.Regex */ 135 | .codehilite .s1 { color: #4e9a06 } /* Literal.String.Single */ 136 | .codehilite .ss { color: #4e9a06 } /* Literal.String.Symbol */ 137 | .codehilite .bp { color: #3465a4 } /* Name.Builtin.Pseudo */ 138 | .codehilite .vc { color: #000000 } /* Name.Variable.Class */ 139 | .codehilite .vg { color: #000000 } /* Name.Variable.Global */ 140 | .codehilite .vi { color: #000000 } /* Name.Variable.Instance */ 141 | .codehilite .il { color: #0000cf; font-weight: bold } /* Literal.Number.Integer.Long */ 142 | 143 | 144 | -------------------------------------------------------------------------------- /table.markdown: -------------------------------------------------------------------------------- 1 | # Python 的神奇方法指南 # 2 | 3 | ###Rafe Kettler### 4 | 5 | Copyright © 2012 Rafe Kettler 6 | 7 | Version 1.17 8 | 9 | PDF 版本正在制作中。 10 | 英文版可以从 [作者网站](http://www.rafekettler.com/magicmethods.pdf) 或者 [Github](https://github.com/justjavac/magicmethods-zh_CN/raw/master/magicmethods.pdf) 下载。 11 | 12 | **目录** 13 | 14 | 1. [介绍](./magicmethods.markdown#1) 15 | 2. [构建和初始化](./magicmethods.markdown#2) 16 | 3. [使操作符在自定义类内工作](./magicmethods.markdown#3) 17 | * [神奇方法——比较](./magicmethods.markdown#31-) 18 | * [神奇方法——数字](./magicmethods.markdown#32-) 19 | 4. [描述你的类](./magicmethods.markdown#4) 20 | 5. [属性访问控制](./magicmethods.markdown#5) 21 | 6. [制作自定义序列](./magicmethods.markdown#6) 22 | 7. [反射](./magicmethods.markdown#7) 23 | 8. [可调用对象](./magicmethods.markdown#8) 24 | 9. [上下文管理](./magicmethods.markdown#9) 25 | 10. [构建描述符对象](./magicmethods.markdown#10) 26 | 11. [Pickling 你的对象](./magicmethods.markdown#11pickling-) 27 | 12. [总结](./magicmethods.markdown#12) 28 | 13. [附录:如何调用神奇方法](./magicmethods.markdown#13) 29 | --------------------------------------------------------------------------------