├── .gitignore ├── LICENSE ├── README.md ├── _config.yml └── debugging ├── code ├── main.py └── multiprocess_main.py ├── images ├── debug1.png ├── multiprocess_debugging.png ├── no_set_trace.png ├── remote_debugging.png ├── set_trace.png ├── variables.png ├── variables1.png └── variables_config.png ├── pudb.md └── pysnooper.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 westdoorblowcola 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # python-Debugging-skills 2 | 整理一些python调试相关的方法, 列举出一些各个场景下使用的例子。 3 | 4 | ## 调试方法 5 | 6 | 7 | #### pdb类调试 8 | 9 | | 名称 | 描述 | 详情 | 10 | | -------- | --------------------------------------- | ------------------------- | 11 | | **pudb** | 带界面的调试器. 调试脚本非常方便~ 推荐~ | [链接](debugging/pudb.md) | 12 | 13 | #### 日志类 14 | 15 | | 名称 | 描述 | 详情 | 16 | | --------------------- | --------------------------------------------------------- | ------------------------------------------------------------ | 17 | | **better-exceptions** | 异常时可以将变量都打印出来，直接集成到日志里,写日志必备品 | [链接](https://github.com/510908220/better-exceptions-examples) | 18 | | | | | 19 | 20 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-leap-day -------------------------------------------------------------------------------- /debugging/code/main.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | 3 | 4 | def generate_list(number, number_list=[]): 5 | number_list.append(number) 6 | return number_list 7 | 8 | 9 | def main(): 10 | num = 1 11 | list1 = generate_list(num) 12 | print('list1:', list1) 13 | assert list1[0] == 1, 'bad list 1' 14 | num = 2 15 | list2 = generate_list(num) 16 | print('list2:', list2) 17 | assert list2[0] == 2, 'bad list 2' 18 | 19 | 20 | if __name__ == "__main__": 21 | main() 22 | -------------------------------------------------------------------------------- /debugging/code/multiprocess_main.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | 3 | import multiprocessing as mp 4 | import time 5 | from pudb.remote import set_trace 6 | 7 | 8 | def worker(worker_id): 9 | """ Simple worker process""" 10 | i = 0 11 | while i < 10: 12 | if worker_id == 1: # debug process with id 1 13 | set_trace(term_size=(80, 24)) 14 | time.sleep(1) # represents some work 15 | print('In Process {}, i:{}'.format(worker_id, i)) 16 | i = i + 1 17 | 18 | 19 | if __name__ == '__main__': 20 | processes = [] 21 | for p_id in range(2): # 2 worker processes 22 | p = mp.Process(target=worker, args=(p_id,)) 23 | p.start() 24 | processes.append(p) 25 | 26 | for p in processes: 27 | p.join() 28 | -------------------------------------------------------------------------------- /debugging/images/debug1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/debug1.png -------------------------------------------------------------------------------- /debugging/images/multiprocess_debugging.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/multiprocess_debugging.png -------------------------------------------------------------------------------- /debugging/images/no_set_trace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/no_set_trace.png -------------------------------------------------------------------------------- /debugging/images/remote_debugging.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/remote_debugging.png -------------------------------------------------------------------------------- /debugging/images/set_trace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/set_trace.png -------------------------------------------------------------------------------- /debugging/images/variables.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/variables.png -------------------------------------------------------------------------------- /debugging/images/variables1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/variables1.png -------------------------------------------------------------------------------- /debugging/images/variables_config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/510908220/python-debugging-skills/be5ce2fe75c123739a6b38fe983aca5560e85929/debugging/images/variables_config.png -------------------------------------------------------------------------------- /debugging/pudb.md: -------------------------------------------------------------------------------- 1 | # pudb 2 | 3 | PuDB is a full-screen, console-based visual debugger for Python. 相对于pdb更加容易使用. 下面整理一下pudb的使用，在需要用的时候当做一个参考. 4 | 5 | 6 | ## 安装 7 | 8 | ``` 9 | pip install pudb 10 | ``` 11 | 12 | ## 代码 13 | 14 | 下面例子使用的代码 15 | 16 | | 文件名 | 描述 | 路径 | 17 | | -------------------- | ------------------ | ---------------------------------------------- | 18 | | main.py | 测试调试 | [main](code/main.py) | 19 | | multiprocess_main.py | 测试pudb多进程调试 | [multiprocess_main](code/multiprocess_main.py) | 20 | | | | | 21 | 22 | ## 基本使用 23 | 24 | #### 不设置断点方式 25 | 26 | ``` 27 | python -m pudb main.py 28 | ``` 29 | 30 | 如图,光标指向了第一行. 因为没有设置断点,直接从第一行开始调试运行. 31 | 32 | ![](images/no_set_trace.png) 33 | 34 | 从界面上可以看到几个信息: 35 | 36 | - **n**: 往下执行 37 | - **s**:进入,比如进入函数 38 | - **b**: 设置断点 39 | - **!**:进入python交互模式,比如可以打印当前上下文的变量等. `ctrl+d`返回到调试模式. 40 | - **Variables**: 当前堆栈变量 41 | - **Stack**: 调用栈 42 | - **Breakpoints**:断点 43 | - **q**: 退出, 出现退出界面再按一次`q`就退出了. 44 | 45 | 46 | 47 | 例子中代码如下: 48 | 49 | ```python 50 | def generate_list(number, number_list=[]): 51 | number_list.append(number) 52 | return number_list 53 | 54 | def main(): 55 | num = 1 56 | list1 = generate_list(num) 57 | print('list1:', list1) 58 | assert list1[0] == 1, 'bad list 1' 59 | num = 2 60 | list2 = generate_list(num) 61 | print('list2:', list2) 62 | assert list2[0] == 2, 'bad list 2' 63 | 64 | if __name__ == "__main__": 65 | main() 66 | 67 | ``` 68 | 69 | 函数`generate_list`会根据传入的参数返回一个包含着参数的列表. 下面运行一下看看效果: 70 | 71 | ```shell 72 | (pudb_demo-cp5XedMM) root@env-monitor-5-2-5:/opt/pudb_demo# python main.py 73 | list1: [1] 74 | list2: [1, 2] 75 | Traceback (most recent call last): 76 | File "main.py", line 21, in 77 | main() 78 | File "main.py", line 17, in main 79 | assert list2[0] == 2, 'bad list 2' 80 | AssertionError: bad list 2 81 | 82 | ``` 83 | 84 | 竟然出错了。。。因为缺少上下文,所以直接看不出错误. 这下改我们的调试帮手`pudb`上手了. 85 | 86 | 运行`python -m pudb main.py `,会出现上面图中那样. 我们输入`n`、`s`等命令一步步看在这里停下来: 87 | 88 | ```python 89 | list2 = generate_list(num) 90 | ``` 91 | 92 | 输入`s`进入函数,如图: 93 | 94 | ![](images/debug1.png) 95 | 96 | 奇怪,`number_list.append(number)`这一行还未执行, `number_list`竟然已经有一个值了. 明明参数上通过`number_list=[]`设置空了. 到目前为止,根据调试找到了错误的位置. 现在一共有两个问题要解决: 97 | 98 | 1. 右边`Variables`只显示了`number_list`参数个数,要是能显示内容就好了. 99 | 2. `number_list=[]`为啥不生效. 100 | 101 | 下面来解答这两个问题: 102 | 103 | 1. 通过键盘`->`键可以将光标移到`Variables`,再上下选择`number_list`变量. 如图: 104 | 105 | ![](images/variables.png)输入回车,可以看到对变量展示的一个配置![](images/variables_config.png)给`Expanded`打钩,然后保存.如图: 106 | 107 | ![](images/variables1.png)现在已经看到`number_list`具体值了. 108 | 109 | 2. 在python里,如果将字典或列表作为函数的参数, 函数第一次调用的时候,python会创建一个持久对象(persistent object ),后面再次调用的时候使用的是同一个对象. 打印对象id可以看出`list1`和`list2`是一样的. 110 | 111 | #### 设置断点方式 112 | 113 | 可以直接在错误的地方加断点. 114 | 115 | ``` 116 | from pudb import set_trace; set_trace() 117 | or 118 | import pudb; pu.db 119 | ``` 120 | 121 | 这里在`list2 = generate_list(num)`上一行加断点. 有断点就不需要`python -m pudb`方式了, 直接`python main.py`就可以了,如图: 122 | 123 | ![](images/set_trace.png) 124 | 125 | 然后就可以像上面那样调试了. 126 | 127 | ## 分离调试窗口 128 | 129 | 当我们在一个窗口里调试时, 是看不到程序的输出的. `pudb`可以通过如下方式实现在新的窗口显示调试时程序的输出. 假设现在两个窗口:窗口A和窗口B. 我们要进行调试的窗口是窗口 B. 130 | 131 | - 查看窗口B的tty, 输入`tty`命令,例如我的是`/dev/pts/4` 132 | - 在窗口 B执行`perl -MPOSIX -e pause`,目的是不让`pudb`有任何输入,要不调试时会错乱. 133 | - 在窗口A执行`PUDB_TTY=/dev/pts/4 python -m pudb main.py` 134 | - 切换到窗口B进行实际的调试. 调试过程可以看到输出语句会打印在窗口A里. 135 | 136 | ## 远程调试 137 | 138 | 首先需要在代码里加上 139 | 140 | ```python 141 | from pudb.remote import set_trace 142 | set_trace(term_size=(80, 24)) 143 | ``` 144 | 145 | 比如现在有窗口A和窗口 B. 146 | 147 | - 在窗口A启动,例如: 148 | 149 | ```shell 150 | (pudb_demo-cp5XedMM) root@env-monitor-5-2-5:/opt/pudb_demo# python main.py 151 | pudb:6899: Please telnet into 127.0.0.1 6899. 152 | pudb:6899: Waiting for client... 153 | ``` 154 | 155 | - 在窗口B执行 156 | 157 | ```shell 158 | telnet 127.0.0.1 6899 159 | ``` 160 | 161 | 如图:![](images/remote_debugging.png) 162 | 163 | 如果需要在其他机器访问,需要这样启动: 164 | 165 | ```shell 166 | PUDB_RDB_HOST=0.0.0.0 python main.py 167 | ``` 168 | 169 | ## 多进程调试 170 | 171 | 有时我们的程序会启动进程去做一些事情. 在多进程模式下,如果普通方式设置断点，例如: 172 | 173 | ```python 174 | from pudb import set_trace; set_trace() 175 | ``` 176 | 177 | 启动时会出现如下错误: 178 | 179 | ```python 180 | return self._getch(0) 181 | File "/root/.local/share/virtualenvs/pudb_demo-cp5XedMM/lib/python3.6/site-packages/urwid/raw_display.py", line 554, in _getch 182 | return ord(os.read(self._term_input_file.fileno(), 1)) 183 | TypeError: ord() expected a character, but string of length 0 found 184 | ``` 185 | 186 | 正确的方式是利用远程调试. 执行如下命令: 187 | 188 | ```shell 189 | PUDB_RDB_HOST=0.0.0.0 python multiprocess_main.py 190 | ``` 191 | 192 | 在远程机器运行: 193 | 194 | ```shell 195 | telnet x.x.x.x 6899 196 | ``` 197 | 198 | 如图: 199 | 200 | ![](images/multiprocess_debugging.png) 201 | 202 | 就好像在一个进程里调试一样,剩余的进程该做什么做什么，互不影响. 203 | 204 | 205 | 206 | ## 参考 207 | 208 | - [**官方文档 **](https://documen.tician.de) 209 | - [Use puDB to Debug Python Multiprocessing Code](https://medium.com/@auro_227/use-pudb-to-debug-python-multiprocessing-code-c0c5551d010c) 210 | - [Using a mutable default value as an argument](https://docs.quantifiedcode.com/python-anti-patterns/correctness/mutable_default_value_as_argument.html) 211 | 212 | -------------------------------------------------------------------------------- /debugging/pysnooper.md: -------------------------------------------------------------------------------- 1 | # PySnooper 2 | 3 | 4 | 5 | https://github.com/cool-RR/PySnooper 6 | 7 | 待整理... --------------------------------------------------------------------------------