├── .gitignore ├── LICENSE ├── README.md ├── docs ├── DevelopmentGuide.md ├── InstallGuide.md ├── SupportedVersions.md └── UserGuide.md ├── heap.png ├── libptmalloc ├── __init__.py ├── frontend │ ├── __init__.py │ ├── commands │ │ ├── __init__.py │ │ └── gdb │ │ │ ├── __init__.py │ │ │ ├── ptarena.py │ │ │ ├── ptbin.py │ │ │ ├── ptchunk.py │ │ │ ├── ptcmd.py │ │ │ ├── ptconfig.py │ │ │ ├── ptfast.py │ │ │ ├── ptfree.py │ │ │ ├── pthelp.py │ │ │ ├── ptlist.py │ │ │ ├── ptmeta.py │ │ │ ├── ptparam.py │ │ │ ├── ptstats.py │ │ │ └── pttcache.py │ ├── frontend_gdb.py │ ├── helpers.py │ └── printutils.py ├── libptmalloc.cfg ├── logger.py ├── ptmalloc │ ├── __init__.py │ ├── cache.py │ ├── heap_structure.py │ ├── malloc_chunk.py │ ├── malloc_par.py │ ├── malloc_state.py │ ├── ptmalloc.py │ └── tcache_perthread.py ├── pydbg │ ├── __init__.py │ ├── debugger.py │ └── pygdbpython.py └── pyptmalloc.py ├── pyptmalloc-dev.py ├── reload.sh ├── requirements.txt ├── setup.py └── test ├── Makefile ├── debug.gdb ├── debug.sh ├── doc.gdb ├── doc.py ├── doc.sh ├── doc2.gdb ├── generate_bins.py ├── sizes.c ├── test.c ├── test.gdb ├── test.py ├── test.sh └── test2.gdb /.gitignore: -------------------------------------------------------------------------------- 1 | tags 2 | __pycache__ 3 | test/build/* 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 cloudburst 4 | Copyright (c) 2021 NCC Group 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # libptmalloc 2 | 3 | libptmalloc is a python library to examine ptmalloc (the glibc userland heap implementation). It is currently designed for use 4 | with GDB but could easily be adapted to work with other debuggers. 5 | 6 | # Installation 7 | 8 | Please refer to the [Install Guide](docs/InstallGuide.md). 9 | 10 | # Usage 11 | 12 | Please refer to the [User Guide](docs/UserGuide.md). 13 | 14 | # Supported versions 15 | 16 | Please refer to the [Supported Versions](docs/SupportedVersions.md). 17 | 18 | # Development 19 | 20 | Please refer to the [Development Guide](docs/DevelopmentGuide.md). -------------------------------------------------------------------------------- /docs/DevelopmentGuide.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | * [Design](#design) 4 | * [pyptmalloc-dev.py](#pyptmalloc-devpy) 5 | * [dev branch](#dev-branch) 6 | * [Python namespaces limitation](#python-namespaces-limitation) 7 | * [Do not use `from XXX import YYY`](#do-not-use-from-xxx-import-yyy) 8 | * [glibc references](#glibc-references) 9 | * [glibc recommended versions](#glibc-recommended-versions) 10 | 11 | 12 | 13 | 14 | # Design 15 | 16 | The following design abstracting the debugger was taken from [libheap](https://github.com/cloudburst/libheap): 17 | 18 | ``` 19 | ----------------------------------------------------------------------- 20 | debugger frontend (commands and prettyprinters) 21 | libptmalloc/frontend 22 | 23 | +-----+ 24 | | | 25 | | gdb | 26 | | | 27 | +--+--+ 28 | | 29 | ------------------------+---------------------------------------------- 30 | | core logic (debugger-agnostic) 31 | | libptmalloc/ptmalloc 32 | +----+-----+ 33 | | | 34 | | ptmalloc | 35 | | | 36 | +----+-----+ 37 | | 38 | ------------------------+---------------------------------------------- 39 | | debugger-dependent APIs 40 | | libptmalloc/pydbg 41 | +--------------+-----+---------+-------------+ 42 | | | | | 43 | +--+---+ +------+------+ +----+----+ +----+---+ 44 | | | | | | | | | 45 | | lldb | | pygdbpython | | pygdbmi | | r2pipe | 46 | | TODO | | | | TODO | | TODO | 47 | | | | | | | | | 48 | +---+--+ +-------+-----+ +---+-----+ +----+---+ 49 | | | | | 50 | | | | +---------+ 51 | | | | | 52 | ----+--------------+-------------+----+-------------------------------- 53 | | | | | debugger-provided backend 54 | | | | +--+ 55 | | | +--------+ | 56 | +--+---+ +--+--+ | +------+-+ 57 | | | | | | | | 58 | | lldb | | gdb +-+ | ptrace | 59 | | | | | | | 60 | +------+ +-----+ +--------+ 61 | ----------------------------------------------------------------------- 62 | ``` 63 | 64 | # pyptmalloc-dev.py 65 | 66 | The normal way to use libptmalloc is to install it in Python libraries with `setup.py` but during development it is easier to use `pyptmalloc-dev.py` that will import libptmalloc after adding the root folder in the Python path. 67 | 68 | # dev branch 69 | 70 | The `dev` branch only supports Python >= 3.7. This is for commodity reasons, as detailed below and in the following [post](https://stackoverflow.com/questions/62524794/python-submodule-importing-correctly-in-python-3-7-but-not-3-6). 71 | 72 | ## Python namespaces limitation 73 | 74 | One quirk of Python namespaces and tools like gdb which allows importing Python files is that it won't reload files that have been already imported, except if you especially request it. So let's consider a scenario where you source `A.py` which imports `B.py` (using `import B` or `from B import *`), it will import `B`. Now you modify `B.py` and re-source `A.py` in your debugger to test your changes. Unfortunately the changes made to `B.py` won't be taken into account. The only solution will be to reload gdb entirely before reloading `A.py`. 75 | 76 | To work around that limitation, we use `importlib.reload()` in the dev branch for all the imported modules. This slows down significantly reloading libptmalloc but it is still faster than reloading gdb :) 77 | 78 | ## Do not use `from XXX import YYY` 79 | 80 | When modifying libptmalloc's source code, it is handy to be able to re-import libptmalloc in gdb without having to restart gdb itself. 81 | 82 | In the `master` branch, we use: `from libptmalloc import *`. 83 | 84 | In the `dev` branch, we don't. We need to use `importlib.reload()` for all imported sub modules, hence we never use `from XXX import YYY` but instead always use `import XXX` so we can then use `importlib.reload(XXX)`. 85 | 86 | # glibc references 87 | 88 | The main reference to use when modifying libptmalloc is the different versions of the glibc source code from [here](http://ftp.gnu.org/gnu/glibc/). 89 | E.g. comparing glibc 2.25 and 2.26 shows the introduction of tcache (`USE_TCACHE`). 90 | 91 | ## glibc recommended versions 92 | 93 | In particular we recommend having at least the following versions: 94 | 95 | * https://ftp.gnu.org/gnu/glibc/glibc-2.22.tar.gz 96 | * https://ftp.gnu.org/gnu/glibc/glibc-2.23.tar.gz (+ malloc_state.attached_threads) 97 | * https://ftp.gnu.org/gnu/glibc/glibc-2.25.tar.gz 98 | * https://ftp.gnu.org/gnu/glibc/glibc-2.26.tar.gz (+ USE_TCACHE) 99 | * https://ftp.gnu.org/gnu/glibc/glibc-2.27.tar.gz (+ malloc_state.have_fastchunks) 100 | 101 | There are important changes between 2 versions that follow each other above. -------------------------------------------------------------------------------- /docs/InstallGuide.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | * [Requirements](#requirements) 4 | * [Debugger/Python](#debuggerpython) 5 | * [Glibc debug symbols](#glibc-debug-symbols) 6 | * [Ubuntu](#ubuntu) 7 | * [Fedora](#fedora) 8 | * [CentOS](#centos) 9 | * [PhotonOS](#photonos) 10 | * [libptmalloc installation](#libptmalloc-installation) 11 | * [General installation](#general-installation) 12 | * [Use without installation](#use-without-installation) 13 | 14 | 15 | 16 | # Requirements 17 | 18 | ## Debugger/Python 19 | 20 | libptmalloc currently works with any gdb version that supports Python >= 3.5. 21 | 22 | libptmalloc code attempts to abstract the debugger so could theoretically be ported to any debugger with Python support. 23 | 24 | ## Glibc debug symbols 25 | 26 | Although libptmalloc may not require a glibc compiled with gdb debugging support and symbols, it functions best if you do use one. Without debug symbols you will need to supply the address of `main_arena`, `mp_` and optionally `tcache` yourself. 27 | 28 | ### Ubuntu 29 | 30 | ``` 31 | apt-get install libc6-dbg 32 | ``` 33 | 34 | ### Fedora 35 | 36 | ``` 37 | yum install yum-utils 38 | debuginfo-install glibc 39 | ``` 40 | 41 | or 42 | 43 | ``` 44 | dnf install dnf-plugins-core 45 | dnf debuginfo-install glibc 46 | ``` 47 | 48 | ### CentOS 49 | 50 | ``` 51 | yum install glibc-debuginfo 52 | ``` 53 | 54 | ### PhotonOS 55 | 56 | On PhotonOS 1.0, the normal `glibc` library already includes symbols. On PhotonOS 3.0, you can get them from: 57 | 58 | ``` 59 | tdnf install glibc-debuginfo 60 | ``` 61 | 62 | # libptmalloc installation 63 | 64 | Clone the repo: 65 | 66 | ``` 67 | git clone https://github.com/nccgroup/libptmalloc 68 | ``` 69 | 70 | Install the Python packages. Note that [future-fstrings](https://pypi.org/project/future-fstrings/) is only required for Python < 3.7 (so effectively for Python 3.5 and 3.6): 71 | 72 | ``` 73 | pip3 install -r libptmalloc/requirements.txt 74 | ``` 75 | 76 | ## General installation 77 | 78 | Then install it globally: 79 | 80 | ``` 81 | sudo pip3 install ./libptmalloc/ 82 | ``` 83 | 84 | or 85 | 86 | ``` 87 | cd libptmalloc 88 | sudo python3 setup.py install 89 | ``` 90 | 91 | Then you can load it into gdb: 92 | 93 | ``` 94 | (gdb) python from libptmalloc import * 95 | ``` 96 | 97 | Note: you can add this command to your gdbinit: 98 | 99 | ``` 100 | echo "python from libptmalloc import *" >> ~/.gdbinit 101 | ``` 102 | 103 | ## Use without installation 104 | 105 | If you don't want to install it globally, you can just source this file: 106 | 107 | ``` 108 | (gdb) source libptmalloc/pyptmalloc-dev.py 109 | ``` 110 | 111 | If you need to modify the libptmalloc and reload it in your debugger, please refer to [DevelopmentGuide.md](DevelopmentGuide.md). -------------------------------------------------------------------------------- /docs/SupportedVersions.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | * [gdb/Python versions](#gdbpython-versions) 4 | * [glibc/ptmalloc versions](#glibcptmalloc-versions) 5 | 6 | 7 | 8 | # gdb/Python versions 9 | 10 | libptmalloc currently works with any gdb version that supports Python >= 3.5. Note that if you use the `dev` branch, you'll need Python >= 3.7, see [DevelopmentGuide.md](DevelopmentGuide.md) for more information. 11 | 12 | # glibc/ptmalloc versions 13 | 14 | The goal of libptmalloc is to support all ptmalloc and glibc versions. 15 | 16 | That being said, it has only been tested extensively on a limited number of versions. If you encounter an error when using it, please create an issue or do a pull request. 17 | 18 | We have used it extensively on the following versions: 19 | 20 | | Linux distribution | Binary/libc architecture | glibc version | Package | tcache | 21 | | -- | -- | -- | -- | -- | 22 | | Centos 7 x64 | x64 | 2.17 | glibc-2.17-322.el7_9 | No | 23 | | Photon 1.0 x64 | x64 | 2.22 | glibc-2.22-26.ph1 | No | 24 | | Ubuntu 18.04 x64 | x86 | 2.27 | libc6-i386-2.27-3ubuntu1.4 | Yes | 25 | | Ubuntu 18.04 x64 | x64 | 2.27 | libc6-2.27-3ubuntu1.4 | Yes | 26 | | Photon 3.0 x64 | x64 | 2.28 | glibc-2.28-13.ph3 | No (disabled) | 27 | 28 | The above list will be updated once we test more versions. Feel free to report 29 | any additional working version so we add it to the list. -------------------------------------------------------------------------------- /heap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nccgroup/libptmalloc/e9011393db1ea79b769dcf5f52bd1170a367b304/heap.png -------------------------------------------------------------------------------- /libptmalloc/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import sys 3 | import logging 4 | 5 | from libptmalloc import logger 6 | from libptmalloc import pyptmalloc as pyp 7 | 8 | try: 9 | log 10 | except: 11 | log = logging.getLogger("libptmalloc") 12 | handler = logging.StreamHandler(sys.stdout) 13 | handler.setFormatter(logger.MyFormatter(datefmt="%H:%M:%S")) 14 | log.addHandler(handler) 15 | 16 | # This allows changing the log level and reloading in gdb even if the logger was already defined 17 | # XXX - however this file is not reloaded early when we reload in gdb, so we need to re-source in gdb 2x 18 | # for the logger level to be changed atm 19 | #log.setLevel(logging.TRACE) # use for debugging reloading .py files only 20 | #log.setLevel(logging.DEBUG) # all other types of debugging 21 | log.setLevel(logging.NOTSET) 22 | 23 | if log.isEnabledFor(logging.TRACE): 24 | log.warning(f"logging TRACE enabled") 25 | elif log.isEnabledFor(logging.DEBUG): 26 | log.warning(f"logging DEBUG enabled") 27 | # elif log.isEnabledFor(logging.INFO): 28 | # log.warning(f"logging INFO enabled") 29 | # elif log.isEnabledFor(logging.WARNING): 30 | # log.warning(f"logging WARNING enabled") 31 | 32 | log.trace("libptmalloc/__init__.py") 33 | 34 | pyp.pyptmalloc() 35 | -------------------------------------------------------------------------------- /libptmalloc/frontend/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | 4 | log = logging.getLogger("libptmalloc") 5 | log.trace("libptmalloc/frontend/__init__.py") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nccgroup/libptmalloc/e9011393db1ea79b769dcf5f52bd1170a367b304/libptmalloc/frontend/commands/__init__.py -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nccgroup/libptmalloc/e9011393db1ea79b769dcf5f52bd1170a367b304/libptmalloc/frontend/commands/gdb/__init__.py -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptarena.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import binascii 6 | import struct 7 | import sys 8 | import logging 9 | 10 | from libptmalloc.frontend import printutils as pu 11 | from libptmalloc.ptmalloc import malloc_chunk as mc 12 | from libptmalloc.ptmalloc import malloc_state as ms 13 | from libptmalloc.ptmalloc import ptmalloc as pt 14 | from libptmalloc.frontend import helpers as h 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("ptarena.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | class ptarena(ptcmd.ptcmd): 27 | """Command to print information about arena(s) represented by the malloc_state structure 28 | """ 29 | 30 | def __init__(self, ptm): 31 | log.debug("ptarena.__init__()") 32 | super(ptarena, self).__init__(ptm, "ptarena") 33 | 34 | self.parser = argparse.ArgumentParser( 35 | description="""Print arena(s) information 36 | 37 | An arena is also known as an mstate. 38 | Analyze the malloc_state structure's fields.""", 39 | add_help=False, 40 | formatter_class=argparse.RawTextHelpFormatter, 41 | epilog='NOTE: Last defined mstate will be cached for future use') 42 | self.parser.add_argument( 43 | "-v", "--verbose", dest="verbose", action="count", default=0, 44 | help="Use verbose output (multiple for more verbosity)" 45 | ) 46 | self.parser.add_argument( 47 | "-h", "--help", dest="help", action="store_true", default=False, 48 | help="Show this help" 49 | ) 50 | self.parser.add_argument( 51 | "-l", dest="list", action="store_true", default=False, 52 | help="List the arenas addresses only" 53 | ) 54 | self.parser.add_argument( 55 | "--use-cache", dest="use_cache", action="store_true", default=False, 56 | help="Do not fetch mstate data if you know they haven't changed since last time they were cached" 57 | ) 58 | self.parser.add_argument( 59 | "address", default=None, nargs="?", type=h.string_to_int, 60 | help="A malloc_mstate struct address. Optional with cached mstate" 61 | ) 62 | # allows to enable a different log level during development/debugging 63 | self.parser.add_argument( 64 | "--loglevel", dest="loglevel", default=None, 65 | help=argparse.SUPPRESS 66 | ) 67 | 68 | @h.catch_exceptions 69 | @ptcmd.ptcmd.init_and_cleanup 70 | def invoke(self, arg, from_tty): 71 | """Inherited from gdb.Command 72 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 73 | """ 74 | 75 | log.debug("ptarena.invoke()") 76 | 77 | self.cache.update_arena(self.args.address, show_status=True, use_cache=self.args.use_cache) 78 | 79 | if self.args.list: 80 | self.list_arenas() 81 | return 82 | 83 | if self.args.verbose == 0: 84 | print(self.cache.mstate) 85 | elif self.args.verbose >= 1: 86 | print(self.cache.mstate.to_string(self.args.verbose)) 87 | 88 | def list_arenas(self): 89 | """List the arena addresses only 90 | """ 91 | 92 | mstate = self.cache.mstate 93 | 94 | if mstate.next == 0: 95 | print("No arenas could be correctly guessed. Wrong glibc version configured?") 96 | print("Nothing was found at {0:#x}".format(mstate.address)) 97 | return 98 | 99 | print("Arena(s) found:", end="\n") 100 | print(" arena @ ", end="") 101 | pu.print_header("{:#x}".format(int(mstate.address)), end="\n") 102 | 103 | if mstate.address != mstate.next: 104 | # we have more than one arena 105 | 106 | curr_arena = ms.malloc_state( 107 | self.ptm, mstate.next, debugger=self.dbg, version=self.version 108 | ) 109 | 110 | while mstate.address != curr_arena.address: 111 | print(" arena @ ", end="") 112 | pu.print_header("{:#x}".format(int(curr_arena.address)), end="\n") 113 | curr_arena = ms.malloc_state( 114 | self.ptm, curr_arena.next, debugger=self.dbg, version=self.version 115 | ) 116 | 117 | if curr_arena.address == 0: 118 | pu.print_error("No arenas could be correctly found.") 119 | break # breaking infinite loop -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptbin.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import struct 6 | import sys 7 | import logging 8 | 9 | from libptmalloc.frontend import printutils as pu 10 | from libptmalloc.ptmalloc import malloc_chunk as mc 11 | from libptmalloc.ptmalloc import ptmalloc as pt 12 | from libptmalloc.frontend import helpers as h 13 | from libptmalloc.frontend.commands.gdb import ptfree 14 | from libptmalloc.frontend.commands.gdb import ptchunk 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("ptbin.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | class ptbin(ptcmd.ptcmd): 27 | """Command to walk and print the unsorted/small/large bins 28 | 29 | Also see ptchunk description""" 30 | 31 | def __init__(self, ptm): 32 | log.debug("ptbin.__init__()") 33 | super(ptbin, self).__init__(ptm, "ptbin") 34 | 35 | self.parser = argparse.ArgumentParser( 36 | description="""Print unsorted/small/large bins information 37 | 38 | All these bins are implemented in the malloc_state.bins[] member. 39 | The unsorted bin is index 0, the small bins are indexes 1-62 and above 63 are large bins.""", 40 | add_help=False, 41 | formatter_class=argparse.RawTextHelpFormatter, 42 | ) 43 | self.parser.add_argument( 44 | "-i", "--index", dest="index", default=None, type=int, 45 | help="Index to the bin to show (0 to 126)" 46 | ) 47 | self.parser.add_argument( 48 | "-b", "--bin-size", dest="size", default=None, type=h.string_to_int, 49 | help="Small/large bin size to show" 50 | ) 51 | # "ptchunk" also has this argument but default and help is different 52 | self.parser.add_argument( 53 | "-c", "--count", dest="count", type=h.check_positive, default=None, 54 | help="Maximum number of chunks to print in each bin" 55 | ) 56 | # other arguments are implemented in the "ptchunk" command 57 | # and will be shown after the above 58 | ptchunk.ptchunk.add_arguments(self) 59 | 60 | @h.catch_exceptions 61 | @ptcmd.ptcmd.init_and_cleanup 62 | def invoke(self, arg, from_tty): 63 | """Inherited from gdb.Command 64 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 65 | """ 66 | 67 | log.debug("ptbin.invoke()") 68 | 69 | self.cache.update_arena(show_status=self.args.debug) 70 | mstate = self.cache.mstate 71 | # This is required by ptchunk.parse_many() 72 | self.cache.update_param(show_status=self.args.debug, use_cache=self.args.use_cache) 73 | 74 | # This is required by show_one_bin(), see description 75 | self.args.real_count = self.args.count 76 | 77 | if self.args.index != None and self.args.size != None: 78 | pu.print_error("Only one of -i and -s can be provided") 79 | return 80 | 81 | if self.args.index != None or self.args.size != None: 82 | ptfree.ptfree.show_one_bin(self, "regular", index=self.args.index, size=self.args.size, use_cache=self.args.use_cache) 83 | else: 84 | self.show_bins(mstate) 85 | 86 | def show_bins(self, mstate, use_cache=False): 87 | """Browse the malloc_state.bins[] fd/bk entries and show how many chunks there is. 88 | It does not show the actual chunks in each bin though 89 | """ 90 | 91 | # We update the cache here so we can see the status before we print 92 | # the title below. Hence we pass use_cache=False on bins_to_string() call 93 | self.ptm.cache.update_bins(show_status=self.args.debug, use_cache=use_cache) 94 | 95 | verbose = self.args.verbose 96 | sb_base = mstate.address + mstate.bins_offset 97 | 98 | pu.print_title("Unsorted/small/large bins in malloc_state @ {:#x}".format(mstate.address), end="") 99 | txt = mstate.bins_to_string(verbose=self.args.verbose+1, use_cache=False) 100 | print(txt) -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptcmd.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | import shlex 4 | from functools import wraps 5 | 6 | from libptmalloc.frontend import printutils as pu 7 | 8 | log = logging.getLogger("libptmalloc") 9 | log.trace("ptcmd.py") 10 | 11 | try: 12 | import gdb 13 | except ImportError: 14 | print("Not running inside of GDB, exiting...") 15 | raise Exception("sys.exit()") 16 | 17 | class ptcmd(gdb.Command): 18 | """This is a super class with convenience methods shared by all the commands to: 19 | - parse the command's arguments/options 20 | - set/reset a logging level (debugging only) 21 | """ 22 | 23 | def __init__(self, ptm, name): 24 | self.ptm = ptm 25 | 26 | if self.ptm.dbg is None: 27 | pu.print_error("Please specify a debugger") 28 | raise Exception("sys.exit()") 29 | 30 | self.name = name 31 | self.old_level = None 32 | self.parser = None # ArgumentParser 33 | self.description = None # Only use if not in the parser 34 | 35 | super(ptcmd, self).__init__(name, gdb.COMMAND_DATA, gdb.COMPLETE_NONE) 36 | 37 | @property 38 | def version(self): 39 | """Easily access the version string without going through the ptmalloc object""" 40 | return self.ptm.version 41 | 42 | @property 43 | def dbg(self): 44 | """Easily access the pydbg object without going through the ptmalloc object""" 45 | return self.ptm.dbg 46 | 47 | @property 48 | def cache(self): 49 | """Easily access the cache object without going through the ptmalloc object""" 50 | return self.ptm.cache 51 | 52 | def set_loglevel(self, loglevel): 53 | """Change the logging level. This is changed temporarily for the duration 54 | of the command since reset_loglevel() is called at the end after the command is executed 55 | """ 56 | if loglevel != None: 57 | numeric_level = getattr(logging, loglevel.upper(), None) 58 | if not isinstance(numeric_level, int): 59 | print("WARNING: Invalid log level: %s" % loglevel) 60 | return 61 | self.old_level = log.getEffectiveLevel() 62 | #print("old loglevel: %d" % self.old_level) 63 | #print("new loglevel: %d" % numeric_level) 64 | log.setLevel(numeric_level) 65 | 66 | def reset_loglevel(self): 67 | """Reset the logging level to the previous one""" 68 | if self.old_level != None: 69 | #print("restore loglevel: %d" % self.old_level) 70 | log.setLevel(self.old_level) 71 | self.old_level = None 72 | 73 | def init_and_cleanup(f): 74 | """Decorator for a command's invoke() method 75 | 76 | This allows: 77 | - not having to duplicate the argument parsing in all commands 78 | - not having to reset the log level before each of the "return" 79 | in the invoke() of each command 80 | """ 81 | 82 | @wraps(f) 83 | def _init_and_cleanup(self, arg, from_tty): 84 | try: 85 | self.args = self.parser.parse_args(shlex.split(arg)) 86 | except SystemExit as e: 87 | # If we specified an unsupported argument/option, argparse will try to call sys.exit() 88 | # which will trigger such an exception, so we can safely catch it to avoid error messages 89 | # in gdb 90 | #h.show_last_exception() 91 | #raise e 92 | return 93 | if self.args.help: 94 | self.parser.print_help() 95 | return 96 | self.set_loglevel(self.args.loglevel) 97 | f(self, arg, from_tty) # Call actual invoke() 98 | self.reset_loglevel() 99 | return _init_and_cleanup -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptconfig.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import binascii 6 | import struct 7 | import sys 8 | import logging 9 | import pprint 10 | import re 11 | 12 | from libptmalloc.frontend import printutils as pu 13 | from libptmalloc.frontend import helpers as h 14 | from libptmalloc.frontend.commands.gdb import ptcmd 15 | 16 | log = logging.getLogger("libptmalloc") 17 | log.trace("ptconfig.py") 18 | 19 | try: 20 | import gdb 21 | except ImportError: 22 | print("Not running inside of GDB, exiting...") 23 | raise Exception("sys.exit()") 24 | 25 | class ptconfig(ptcmd.ptcmd): 26 | """Command to manage ptmalloc configuration""" 27 | 28 | def __init__(self, ptm): 29 | log.debug("ptconfig.__init__()") 30 | super(ptconfig, self).__init__(ptm, "ptconfig") 31 | 32 | self.parser = argparse.ArgumentParser( 33 | description="""Show/change ptmalloc configuration""", 34 | formatter_class=argparse.RawTextHelpFormatter, 35 | add_help=False, 36 | epilog="""E.g. 37 | ptconfig 38 | ptconfig -v 2.27 39 | ptconfig -t off""") 40 | self.parser.add_argument( 41 | "-h", "--help", dest="help", action="store_true", default=False, 42 | help="Show this help" 43 | ) 44 | self.parser.add_argument( 45 | "-v", "--version", dest="version", type=float, default=None, 46 | help="Change the glibc version manually (e.g. 2.27)" 47 | ) 48 | self.parser.add_argument( 49 | "-t", "--tcache", dest="tcache", type=str, default=None, 50 | help="Enable or disable tcache (on/off)" 51 | ) 52 | self.parser.add_argument( 53 | "-o", "--distribution", dest="distribution", type=str, default=None, 54 | help="Target OS distribution (e.g. debian, ubuntu, centos, photon)" 55 | ) 56 | self.parser.add_argument( 57 | "-r", "--release", dest="release", type=str, default=None, 58 | help="Target OS release version (e.g. 10 for debian, 18.04 for ubuntu, 8 for centos, 3.0 for photon)" 59 | ) 60 | # allows to enable a different log level during development/debugging 61 | self.parser.add_argument( 62 | "--loglevel", dest="loglevel", default=None, 63 | help=argparse.SUPPRESS 64 | ) 65 | 66 | @staticmethod 67 | def set_distribution(ptm, distribution): 68 | if distribution != "photon": 69 | print("Distribution has default glibc settings, ignoring") 70 | return 71 | ptm.distribution = distribution 72 | 73 | @staticmethod 74 | def set_release(ptm, release): 75 | if ptm.distribution == "photon": 76 | if release != "3.0": 77 | print("Release has default glibc settings for Photon OS, ignoring") 78 | return 79 | else: 80 | print("Unsupported distribution or has default glibc setttings, ignoring") 81 | return 82 | ptm.release = release 83 | 84 | @h.catch_exceptions 85 | @ptcmd.ptcmd.init_and_cleanup 86 | def invoke(self, arg, from_tty): 87 | """Inherited from gdb.Command 88 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 89 | """ 90 | 91 | log.debug("ptconfig.invoke()") 92 | 93 | updated = False 94 | 95 | if self.args.version != None: 96 | self.ptm.version = self.args.version 97 | # Resetting it 98 | if self.ptm.version >= 2.26: 99 | self.ptm.tcache_enabled = True 100 | else: 101 | self.ptm.tcache_enabled = False 102 | updated = True 103 | 104 | if self.args.tcache != None: 105 | if self.args.tcache == "on": 106 | self.ptm.tcache_enabled = True 107 | elif self.args.tcache == "off": 108 | self.ptm.tcache_enabled = False 109 | else: 110 | print("Unsupported tcache value, only \"on\" and \"off\" are supported, ignoring") 111 | updated = True 112 | 113 | if self.args.distribution != None: 114 | ptconfig.set_distribution(self.ptm, self.args.distribution) 115 | updated = True 116 | 117 | if self.args.release != None: 118 | ptconfig.set_release(self.ptm, self.args.release) 119 | updated = True 120 | 121 | if updated: 122 | # Resetting some cached info 123 | self.ptm.cache.mstate = None 124 | return 125 | 126 | # no argument specified 127 | d = {} 128 | d["glibc version"] = self.ptm.version 129 | if self.ptm.tcache_enabled is True: 130 | d["tcache"] = "enabled" 131 | elif self.ptm.tcache_enabled is False: 132 | d["tcache"] = "disabled" 133 | if self.ptm.distribution is not None: 134 | d["distribution"] = self.ptm.distribution 135 | if self.ptm.release is not None: 136 | d["release"] = self.ptm.release 137 | 138 | for k,v in d.items(): 139 | pu.print_header("{:<20}".format(k), end="") 140 | print(v) 141 | 142 | 143 | -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptfast.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import struct 6 | import sys 7 | import logging 8 | 9 | from libptmalloc.frontend import printutils as pu 10 | from libptmalloc.ptmalloc import malloc_chunk as mc 11 | from libptmalloc.ptmalloc import malloc_state as ms 12 | from libptmalloc.ptmalloc import ptmalloc as pt 13 | from libptmalloc.frontend import helpers as h 14 | from libptmalloc.frontend.commands.gdb import ptchunk 15 | from libptmalloc.frontend.commands.gdb import ptfree 16 | from libptmalloc.frontend.commands.gdb import ptcmd 17 | 18 | log = logging.getLogger("libptmalloc") 19 | log.trace("ptfast.py") 20 | 21 | try: 22 | import gdb 23 | except ImportError: 24 | print("Not running inside of GDB, exiting...") 25 | raise Exception("sys.exit()") 26 | 27 | class ptfast(ptcmd.ptcmd): 28 | """Command to walk and print the fast bins 29 | 30 | Also see ptchunk description""" 31 | 32 | def __init__(self, ptm): 33 | log.debug("ptfast.__init__()") 34 | super(ptfast, self).__init__(ptm, "ptfast") 35 | 36 | self.parser = argparse.ArgumentParser( 37 | description="""Print fast bins information 38 | 39 | They are implemented in the malloc_state.fastbinsY[] member.""", 40 | add_help=False, 41 | formatter_class=argparse.RawTextHelpFormatter, 42 | ) 43 | self.parser.add_argument( 44 | "address", default=None, nargs="?", type=h.string_to_int, 45 | help="An optional arena address" 46 | ) 47 | self.parser.add_argument( 48 | "-i", "--index", dest="index", default=None, type=int, 49 | help="Index to the fast bin to show (0 to 9)" 50 | ) 51 | self.parser.add_argument( 52 | "-b", "--bin-size", dest="size", default=None, type=h.string_to_int, 53 | help="Fast bin size to show" 54 | ) 55 | # "ptchunk" also has this argument but default and help is different 56 | self.parser.add_argument( 57 | "-c", "--count", dest="count", type=h.check_positive, default=None, 58 | help="Maximum number of chunks to print in each bin" 59 | ) 60 | # other arguments are implemented in the "ptchunk" command 61 | # and will be shown after the above 62 | ptchunk.ptchunk.add_arguments(self) 63 | 64 | @h.catch_exceptions 65 | @ptcmd.ptcmd.init_and_cleanup 66 | def invoke(self, arg, from_tty): 67 | """Inherited from gdb.Command 68 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 69 | """ 70 | 71 | log.debug("ptfast.invoke()") 72 | 73 | self.cache.update_arena(self.args.address, show_status=self.args.debug, use_cache=self.args.use_cache) 74 | mstate = self.cache.mstate 75 | # This is required by ptchunk.parse_many() 76 | self.cache.update_param(show_status=self.args.debug, use_cache=self.args.use_cache) 77 | 78 | # This is required by show_one_bin(), see description 79 | self.args.real_count = self.args.count 80 | 81 | if self.args.index != None and self.args.size != None: 82 | pu.print_error("Only one of -i and -s can be provided") 83 | return 84 | 85 | if self.args.index != None or self.args.size != None: 86 | ptfree.ptfree.show_one_bin(self, "fast", index=self.args.index, size=self.args.size, use_cache=self.args.use_cache) 87 | else: 88 | self.show_fastbins(mstate, use_cache=self.args.use_cache) 89 | 90 | def show_fastbins(self, mstate, use_cache=False): 91 | """Browse the malloc_state.fastbinsY[] fd entries and show how many chunks there is. 92 | It does not show the actual chunks in each bin though 93 | """ 94 | 95 | # We update the cache here so we can see the status before we print 96 | # the title below. Hence we pass use_cache=False on fastbins_to_string() call 97 | self.ptm.cache.update_fast_bins(show_status=self.args.debug, use_cache=use_cache) 98 | 99 | pu.print_title("Fast bins in malloc_state @ {:#x}".format(mstate.address), end="") 100 | txt = mstate.fastbins_to_string(verbose=self.args.verbose+1, use_cache=False) 101 | print(txt) 102 | 103 | # XXX - support the "size" argument if needed 104 | @staticmethod 105 | def is_in_fastbin(address, ptm, dbg=None, size=None, index=None, use_cache=False): 106 | """Check if a particular chunk's address is in one or all fast bins""" 107 | 108 | if index != None: 109 | ptm.cache.update_fast_bins(use_cache=use_cache, bins_list=[index]) 110 | if address in ptm.cache.fast_bins[index]: 111 | return True 112 | else: 113 | return False 114 | else: 115 | ptm.cache.update_fast_bins(use_cache=use_cache) 116 | for i in range(0, ptm.NFASTBINS): 117 | if address in ptm.cache.fast_bins[i]: 118 | return True 119 | return False -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptfree.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import struct 6 | import sys 7 | import logging 8 | 9 | from libptmalloc.frontend import printutils as pu 10 | from libptmalloc.ptmalloc import malloc_chunk as mc 11 | from libptmalloc.ptmalloc import malloc_state as ms 12 | from libptmalloc.ptmalloc import ptmalloc as pt 13 | from libptmalloc.frontend import helpers as h 14 | from libptmalloc.frontend.commands.gdb import ptchunk 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("ptfree.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | class ptfree(ptcmd.ptcmd): 27 | """Command to walk and print all bins 28 | 29 | Also see ptchunk description""" 30 | 31 | def __init__(self, ptm): 32 | log.debug("ptfree.__init__()") 33 | super(ptfree, self).__init__(ptm, "ptfree") 34 | 35 | self.parser = argparse.ArgumentParser( 36 | description="""Print all bins information 37 | 38 | Browse fast bins, tcache bins, unsorted/small/large bins. 39 | Effectively calls into 'ptfast', 'pttcache' and 'ptbin' commands""", 40 | formatter_class=argparse.RawTextHelpFormatter, 41 | add_help=False) 42 | # "ptchunk" also has this argument but default and help is different 43 | self.parser.add_argument( 44 | "-c", "--count", dest="count", type=h.check_positive, default=None, 45 | help="Maximum number of chunks to print in each bin" 46 | ) 47 | # other arguments are implemented in the "ptchunk" command 48 | # and will be shown after the above 49 | ptchunk.ptchunk.add_arguments(self) 50 | 51 | @h.catch_exceptions 52 | @ptcmd.ptcmd.init_and_cleanup 53 | def invoke(self, arg, from_tty): 54 | """Inherited from gdb.Command 55 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 56 | """ 57 | 58 | log.debug("ptfree.invoke()") 59 | 60 | self.cache.update_arena(show_status=self.args.debug) 61 | mstate = self.cache.mstate 62 | 63 | self.cache.update_tcache(show_status=self.args.debug) 64 | # This is required by ptchunk.parse_many() 65 | self.cache.update_param(show_status=self.args.debug) 66 | 67 | # We don't update the tcache bins, fast bins and unsorted/small/large bins 68 | # in the cache because it will automatically be done in show_all_bins() 69 | # The idea is we'd rather have things written over time than fetching all the cache 70 | # at the beginning without having any output for the user 71 | self.show_all_bins(mstate) 72 | 73 | @staticmethod 74 | def bin_size2index(ptm, name, size): 75 | """Convert a chunk size into an index in one of the bin array: 76 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 77 | 78 | :param ptm: ptmalloc object 79 | :param name: "tcache", "fast" or "regular" string 80 | :param size: the chunk size queried 81 | :return: the matching index in the corresponding array 82 | """ 83 | 84 | if name == "tcache": 85 | return ptm.tcache_bin_index(size) 86 | elif name == "fast": 87 | return ptm.fast_bin_index(size) 88 | elif name == "regular": 89 | # XXX: -1 is because the index 0 is for unsorted bin? 90 | return ptm.bin_index(size)-1 91 | else: 92 | raise Exception("Wrong name in bin_size2index()") 93 | 94 | @staticmethod 95 | def update_bins_in_cache(ptm, 96 | name, 97 | show_status=False, 98 | use_cache=False, 99 | bins_list=[] 100 | ): 101 | """Update the bins in the cache for one of the bin array: 102 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 103 | 104 | :param ptm: ptmalloc object 105 | :param name: "tcache", "fast" or "regular" string 106 | :param show_status: True to print cache status, False to not print it 107 | :param use_cache: True to avoid fetching structures and bins. False to 108 | fetch them again and update the cache 109 | :param bins_list: If non-empty, contains a list of indexes into the bin array 110 | that we update. It means the others won't be modified. It 111 | serves as an optimization so we can update only certain bins 112 | 113 | :return: the matching index in the corresponding array 114 | """ 115 | 116 | if name == "tcache": 117 | ptm.cache.update_tcache_bins(show_status=show_status, use_cache=use_cache, bins_list=bins_list) 118 | elif name == "fast": 119 | ptm.cache.update_fast_bins(show_status=show_status, use_cache=use_cache, bins_list=bins_list) 120 | elif name == "regular": 121 | ptm.cache.update_bins(show_status=show_status, use_cache=use_cache, bins_list=bins_list) 122 | else: 123 | raise Exception("Wrong name in update_bins_in_cache()") 124 | 125 | @staticmethod 126 | def get_chunks_addresses_in_bin(ptm, 127 | name, 128 | index 129 | ): 130 | """Get the list of chunks addresses in in one of the bin array: 131 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 132 | for a given index 133 | 134 | :param ptm: ptmalloc object 135 | :param name: "tcache", "fast" or "regular" string 136 | :param index: the index in the particular bin 137 | :return: the list of chunks' addresses (list of integers) 138 | """ 139 | 140 | if name == "tcache": 141 | return ptm.cache.tcache_bins[index] 142 | elif name == "fast": 143 | return ptm.cache.fast_bins[index] 144 | elif name == "regular": 145 | return ptm.cache.bins[index] 146 | else: 147 | raise Exception("Wrong name in get_chunks_addresses_in_bin()") 148 | 149 | @staticmethod 150 | def get_bin_header(ptm, 151 | name, 152 | index, 153 | empty=False 154 | ): 155 | """Get the string header shown before printing a given bin array: 156 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 157 | for a given index 158 | 159 | :param ptm: ptmalloc object 160 | :param name: "tcache", "fast" or "regular" string 161 | :param index: the index in the particular bin 162 | :param empty: True if the bin for that array and index does not have 163 | any chunk. 164 | :return: the string to show before printing the actual chunks in a bin 165 | """ 166 | 167 | if name == "tcache": 168 | header = pu.color_header("{} bin {}".format(name, index)) 169 | header += " (sz {:#x})".format(ptm.tcache_bin_size(index)) 170 | elif name == "fast": 171 | header = pu.color_header("{} bin {}".format(name, index)) 172 | header += " (sz {:#x})".format(ptm.fast_bin_size(index)) 173 | elif name == "regular": 174 | if index == ptm.bin_index_unsorted: 175 | header = pu.color_header("unsorted bin {}".format(index)) 176 | header += " (various sz)" 177 | elif index <= ptm.bin_index_small_max: 178 | header = pu.color_header("small bin {}".format(index)) 179 | header += " (sz {:#x})".format(ptm.bin_size(index)) 180 | elif index <= ptm.bin_index_large_max: 181 | header = pu.color_header("large bin {}".format(index)) 182 | header += " (sz {:#x})".format(ptm.bin_size(index)) 183 | elif index == ptm.bin_index_uncategorized: 184 | header = pu.color_header("large bin uncategorized {}".format(index)) 185 | header += " (sz > {:#x})".format(ptm.bin_size(127)) 186 | else: 187 | raise Exception("Wrong name in get_bin_header()") 188 | if empty: 189 | header += " is empty" 190 | return header 191 | 192 | @staticmethod 193 | def get_bin_footer(self, 194 | name, 195 | index, 196 | printed_count, 197 | max_count=None 198 | ): 199 | """Get the string footer shown after printing a given bin array: 200 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 201 | for a given index 202 | 203 | :param self: ptfree object 204 | :param name: "tcache", "fast" or "regular" string 205 | :param index: the index in the particular bin 206 | :param printed_count: How many chunks were already printed for that bin 207 | :param max_count: How many chunks were requested to be printed by the user 208 | or None if no limit. 209 | :return: the string to show after printing the actual chunks in a bin 210 | """ 211 | 212 | if name == "tcache": 213 | footer = pu.color_footer("{} bin {}".format(name, index)) 214 | elif name == "fast": 215 | footer = pu.color_footer("{} bin {}".format(name, index)) 216 | elif name == "regular": 217 | if index == self.ptm.bin_index_unsorted: 218 | footer = pu.color_footer("unsorted bin {}".format(index)) 219 | elif index <= self.ptm.bin_index_small_max: 220 | footer = pu.color_footer("small bin {}".format(index)) 221 | elif index <= self.ptm.bin_index_large_max: 222 | footer = pu.color_footer("large bin {}".format(index)) 223 | elif index == self.ptm.bin_index_uncategorized: 224 | footer = pu.color_footer("large uncategorized bin {}".format(index)) 225 | else: 226 | raise Exception("Wrong name in get_bin_footer()") 227 | if max_count == None or printed_count < max_count: 228 | footer += f": total of {printed_count} chunks" 229 | else: 230 | footer += f": total of {printed_count}+ chunks" 231 | return footer 232 | 233 | @staticmethod 234 | def get_count_bins(ptm, name): 235 | """Retrieve how many bins there is for given bin array: 236 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 237 | 238 | :param ptm: ptmalloc object 239 | :param name: "tcache", "fast" or "regular" string 240 | :return: the size of the bin array 241 | """ 242 | 243 | if name == "tcache": 244 | return ptm.TCACHE_MAX_BINS 245 | elif name == "fast": 246 | return ptm.NFASTBINS 247 | elif name == "regular": 248 | return ptm.NBINS-1 249 | else: 250 | raise Exception("Wrong name in get_count_bins()") 251 | 252 | @staticmethod 253 | def show_one_bin( 254 | self, 255 | name, 256 | index=None, 257 | size=None, 258 | print_empty=True, 259 | show_status=False, 260 | use_cache=False, 261 | 262 | ): 263 | """Browse a given index for a given bin array: 264 | tcache.entries[], malloc.fastbinsY[] or malloc.bins[] 265 | and show the actual chunks in that particular bin 266 | 267 | :param ptm: ptmalloc object 268 | :param name: "tcache", "fast" or "regular" string 269 | :param index: the index in the particular bin (instead of size) or None 270 | :param size: the chunk size in the particular bin (instead of index) or None 271 | :param print_empty: True if we want to show empty bins. False otherwise 272 | :param show_status: True to print cache status, False to not print it 273 | :param use_cache: True to avoid fetching structures and bins. False to 274 | fetch them again and update the cache 275 | 276 | Note that this function assumes self.args.real_count exists and equals 277 | to the number of chunks requested to be printed by the user (using -c i.e. 278 | initially equal to self.args.count). 279 | This is because show_one_bin() may be called several times but sometimes 280 | we need to override self.args.count to be equal to 1 before we can call into 281 | ptchunk.parse_many2(), but still passing the real_count as count_printed 282 | argument 283 | 284 | Note that it is a static method but it has self as a first 285 | argument to make it easier to read its implementation 286 | """ 287 | 288 | ptm = self.ptm 289 | dbg = self.dbg 290 | 291 | if index == None and size == None: 292 | raise Exception("show_one_bin requires an index or size") 293 | if size != None and index != None: 294 | raise Exception("show_one_bin requires either an index or a size, not both") 295 | if index == None: 296 | index = ptfree.bin_size2index(ptm, name, size) 297 | 298 | if index < 0 or index >= ptfree.get_count_bins(ptm, name): 299 | raise Exception("index out of range in bin") 300 | 301 | # Prepare arguments passed to malloc_chunk() for all the chunks 302 | # in the bin 303 | tcache = None 304 | fast = None 305 | inuse = None 306 | if name == "tcache": 307 | tcache = True 308 | elif name == "fast": 309 | fast = True 310 | elif name == "regular": 311 | inuse = False 312 | tcache = False 313 | fast = False 314 | else: 315 | raise Exception("Wrong name in show_one_bin()") 316 | 317 | ptfree.update_bins_in_cache(ptm, name, show_status=show_status, use_cache=use_cache, bins_list=[index]) 318 | bin_ = [f"{addr:#x}" for addr in ptfree.get_chunks_addresses_in_bin(self, name, index)] 319 | 320 | if len(bin_) == 0: 321 | if print_empty: 322 | print(ptfree.get_bin_header(ptm, name, index, empty=True)) 323 | return 0 324 | 325 | # Prepare arguments for "ptchunk" format 326 | # i.e. the chunks to print are from the bin 327 | # The amount of printed addresses will be limited by 328 | # parse_many2()'s count_printed argument 329 | self.args.addresses = bin_ 330 | self.args.no_newline = False 331 | # Quirk of parse_many2() since we only want to print 1 chunk linearly 332 | # in memory for every chunk in the bin and the real count of chunks is 333 | # set above in self.args.addresses 334 | self.args.count = 1 335 | 336 | header_once = ptfree.get_bin_header(ptm, name, index) 337 | chunks = ptchunk.ptchunk.parse_many2( 338 | self, 339 | inuse=inuse, 340 | tcache=tcache, 341 | fast=fast, 342 | allow_invalid=True, 343 | separate_addresses_non_verbose=False, 344 | header_once=header_once, 345 | count_handle=1, 346 | count_printed=self.args.real_count 347 | ) 348 | 349 | if print_empty or len(chunks) > 0: 350 | print(ptfree.get_bin_footer(self, name, index, len(chunks), self.args.real_count)) 351 | 352 | return len(chunks) 353 | 354 | def show_all_bins(self, mstate): 355 | """Calls into pttcache, ptfast and ptbin to browse the bins and show how many chunk there is. 356 | It does not show the actual chunks in each bin though 357 | """ 358 | 359 | ptm = self.ptm 360 | 361 | # As you can see below, we don't pass any use_cache=True because we want it to re-fetch 362 | # data, as pointed out in the comment above in invoke() 363 | 364 | # Save old count since we will override it later 365 | # when we call into ptchunk but we need to reset it 366 | # for every show_one_*() calls in the 3 loops below 367 | self.args.real_count = self.args.count 368 | 369 | if self.ptm.is_tcache_enabled() and self.ptm.tcache_available: 370 | for i in range(ptfree.get_count_bins(ptm, "tcache")): 371 | count = ptfree.show_one_bin(self, "tcache", index=i, print_empty=False) 372 | if count > 0: 373 | print("---") 374 | 375 | for i in range(ptfree.get_count_bins(ptm, "fast")): 376 | count = ptfree.show_one_bin(self, "fast", index=i, print_empty=False) 377 | if count > 0: 378 | print("---") 379 | 380 | for i in range(ptfree.get_count_bins(ptm, "regular")): 381 | count = ptfree.show_one_bin(self, "regular", index=i, print_empty=False) 382 | if count > 0: 383 | print("---") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/pthelp.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import sys 5 | import logging 6 | 7 | from libptmalloc.frontend import printutils as pu 8 | from libptmalloc.ptmalloc import malloc_state as ms 9 | from libptmalloc.ptmalloc import ptmalloc as pt 10 | from libptmalloc.frontend import helpers as h 11 | from libptmalloc.frontend.commands.gdb import ptcmd 12 | 13 | log = logging.getLogger("libptmalloc") 14 | log.trace("pthelp.py") 15 | 16 | try: 17 | import gdb 18 | except ImportError: 19 | print("Not running inside of GDB, exiting...") 20 | raise Exception("sys.exit()") 21 | 22 | class pthelp(ptcmd.ptcmd): 23 | """Command to list all available commands""" 24 | 25 | def __init__(self, ptm, commands=[]): 26 | log.debug("pthelp.__init__()") 27 | super(pthelp, self).__init__(ptm, "pthelp") 28 | 29 | self.cmds = commands 30 | 31 | @h.catch_exceptions 32 | def invoke(self, arg, from_tty): 33 | """Inherited from gdb.Command 34 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 35 | 36 | Print the usage of all the commands 37 | """ 38 | 39 | pu.print_header("{:<20}".format("pthelp"), end="") 40 | print("List all libptmalloc commands") 41 | for cmd in self.cmds: 42 | if cmd.parser != None: 43 | # Only keep the first line of the description which should be short 44 | description = cmd.parser.description.split("\n")[0] 45 | elif cmd.description != None: 46 | description = cmd.description 47 | else: 48 | description = "Unknown" 49 | pu.print_header("{:<20}".format(cmd.name), end="") 50 | print(description) 51 | print("Note: Use a command name with -h to get additional help") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptlist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import sys 6 | import logging 7 | 8 | from libptmalloc.frontend import printutils as pu 9 | from libptmalloc.ptmalloc import malloc_chunk as mc 10 | from libptmalloc.ptmalloc import malloc_par as mp 11 | from libptmalloc.ptmalloc import malloc_state as ms 12 | from libptmalloc.ptmalloc import ptmalloc as pt 13 | from libptmalloc.frontend import helpers as h 14 | from libptmalloc.frontend.commands.gdb import ptchunk 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("ptlist.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | class ptlist(ptcmd.ptcmd): 27 | """Command to print a flat listing of all the chunks in an arena 28 | 29 | Also see ptchunk description 30 | 31 | Inspired by jp's phrack print and arena.c""" 32 | 33 | def __init__(self, ptm): 34 | log.debug("ptlist.__init__()") 35 | super(ptlist, self).__init__(ptm, "ptlist") 36 | 37 | self.parser = argparse.ArgumentParser( 38 | description="""Print a flat listing of all the chunks in an arena""", 39 | add_help=False, 40 | formatter_class=argparse.RawTextHelpFormatter, 41 | epilog="""E.g. 42 | ptlist -M "tag, backtrace:5" """) 43 | self.parser.add_argument( 44 | "address", default=None, nargs="?", type=h.string_to_int, 45 | help="A malloc_mstate struct address. Optional with cached mstate" 46 | ) 47 | self.parser.add_argument( 48 | "-C", "--compact", dest="compact", action="store_true", default=False, 49 | help="Compact flat heap listing" 50 | ) 51 | # "ptchunk" also has this argument but default for 52 | # "ptlist" is to show unlimited number of chunks 53 | self.parser.add_argument( 54 | "-c", "--count", dest="count", type=h.check_positive, default=None, 55 | help="Number of chunks to print linearly" 56 | ) 57 | # other arguments are implemented in the "ptchunk" command 58 | # and will be shown after the above 59 | ptchunk.ptchunk.add_arguments(self) 60 | 61 | @h.catch_exceptions 62 | @ptcmd.ptcmd.init_and_cleanup 63 | def invoke(self, arg, from_tty): 64 | """Inherited from gdb.Command 65 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 66 | """ 67 | 68 | log.debug("ptlist.invoke()") 69 | 70 | self.cache.update_all(show_status=self.args.debug, use_cache=self.args.use_cache, arena_address=self.args.address) 71 | 72 | log.debug("ptlist.invoke() (2)") 73 | 74 | mstate = self.cache.mstate 75 | par = self.cache.par 76 | 77 | if mstate.address == self.cache.main_arena_address: 78 | start, _ = self.dbg.get_heap_address(par) 79 | else: 80 | print("Using manual arena calculation for heap start") 81 | start = (mstate.address + mstate.size + self.ptm.MALLOC_ALIGN_MASK) & ~self.ptm.MALLOC_ALIGN_MASK 82 | self.sbrk_base = start 83 | 84 | if self.args.compact: 85 | self.compact_listing() 86 | else: 87 | self.listing() 88 | 89 | def listing(self): 90 | """Print all the chunks in all the given arenas using a flat listing 91 | """ 92 | 93 | pu.print_title("{:>15} for arena @ {:#x}".format("flat heap listing", self.cache.mstate.address), end="\n") 94 | 95 | # Prepare arguments for "ptchunk" format 96 | # i.e. there is only one start address == sbrk_base 97 | if self.ptm.SIZE_SZ == 4: 98 | # Workaround on 32-bit. Empirically it seems the first chunk starts at offset +0x8? 99 | self.args.addresses = [ f"{self.sbrk_base+0x8:#x}"] 100 | else: 101 | self.args.addresses = [ f"{self.sbrk_base:#x}"] 102 | self.args.no_newline = False 103 | 104 | chunks = ptchunk.ptchunk.parse_many2(self) 105 | 106 | if len(chunks) > 0: 107 | if self.args.count == None: 108 | print(f"Total of {len(chunks)} chunks") 109 | else: 110 | print(f"Total of {len(chunks)}+ chunks") 111 | 112 | if self.args.json_filename != None: 113 | ptchunk.ptchunk.dump_json(self, chunks) 114 | 115 | def compact_listing(self): 116 | """Print all the chunks in a given arena using a compact flat listing 117 | """ 118 | 119 | max_count = self.args.count 120 | 121 | pu.print_title("{:>15} for arena @ {:#x}".format("compact flat heap listing", self.cache.mstate.address), end="\n") 122 | 123 | if self.ptm.SIZE_SZ == 4: 124 | # Workaround on 32-bit. Empirically it seems the first chunk starts at offset +0x8? 125 | addr = self.sbrk_base+8 126 | else: 127 | addr = self.sbrk_base 128 | 129 | count = 0 130 | while True: 131 | p = mc.malloc_chunk( 132 | self.ptm, 133 | addr, 134 | read_data=False, 135 | debugger=self.dbg, 136 | use_cache=True 137 | ) 138 | 139 | if p.address == self.ptm.top(self.cache.mstate): 140 | print("|T", end="") 141 | count += 1 142 | break 143 | 144 | if p.type == pt.chunk_type.FREE_FAST: 145 | print("|f%d" % self.ptm.fast_bin_index(self.ptm.chunksize(p)), end="") 146 | elif p.type == pt.chunk_type.FREE_TCACHE: 147 | print("|t%d" % self.ptm.tcache_bin_index(self.ptm.chunksize(p)), end="") 148 | elif p.type == pt.chunk_type.INUSE: 149 | print("|M", end="") 150 | else: 151 | if ( 152 | (p.fd == self.cache.mstate.last_remainder) 153 | and (p.bk == self.cache.mstate.last_remainder) 154 | and (self.cache.mstate.last_remainder != 0) 155 | ): 156 | print("|L", end="") 157 | else: 158 | print("|F%d" % self.ptm.bin_index(self.ptm.chunksize(p)), end="") 159 | count += 1 160 | sys.stdout.flush() 161 | 162 | if max_count != None and count == max_count: 163 | break 164 | 165 | addr = self.ptm.next_chunk(p) 166 | 167 | print("|") 168 | if max_count == None: 169 | print(f"Total of {count} chunks") 170 | else: 171 | print(f"Total of {count}+ chunks") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptmeta.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import binascii 6 | import struct 7 | import sys 8 | import logging 9 | import pprint 10 | import re 11 | import pickle 12 | 13 | from libptmalloc.frontend import printutils as pu 14 | from libptmalloc.frontend import helpers as h 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("ptmeta.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | meta_cache = {} 27 | backtrace_ignore = set([]) 28 | 29 | colorize_table = { 30 | "red": pu.red, 31 | "green": pu.green, 32 | "yellow": pu.yellow, 33 | "blue": pu.blue, 34 | "purple": pu.purple, 35 | "cyan": pu.cyan, 36 | "gray": pu.gray, 37 | "lightred": pu.light_red, 38 | "lightgreen": pu.light_green, 39 | "lightyellow": pu.light_yellow, 40 | "lightblue": pu.light_blue, 41 | "lightpurple": pu.light_purple, 42 | "lightcyan": pu.light_cyan, 43 | "lightgray": pu.light_gray, 44 | "white": pu.white, 45 | "black": pu.black, 46 | } 47 | 48 | METADATA_DB = "metadata.pickle" 49 | def save_metadata_to_file(filename): 50 | """During development, we reload libptmalloc and lose the metadata database 51 | so this allows saving it easily into a file before doing so 52 | """ 53 | d = {} 54 | d["meta_cache"] = meta_cache 55 | d["backtrace_ignore"] = backtrace_ignore 56 | pickle.dump(d, open(filename, "wb")) 57 | 58 | def load_metadata_from_file(filename): 59 | """During development, we reload libptmalloc and lose the metadata database 60 | so this allows reloading it easily from a file 61 | """ 62 | global meta_cache, backtrace_ignore 63 | d = pickle.load(open(filename, "rb")) 64 | meta_cache = d["meta_cache"] 65 | backtrace_ignore = d["backtrace_ignore"] 66 | 67 | def get_metadata(address, list_metadata=[]): 68 | """ 69 | :param address: the address to retrieve metatada from 70 | :param list_metadata: If a list, the list of metadata to retrieve (even empty list). 71 | If the "all" string, means to retrieve all metadata 72 | :return: the following L, suffix, epilog, colorize_func 73 | """ 74 | 75 | L = [] # used for json output 76 | suffix = "" # used for one-line output 77 | epilog = "" # used for verbose output 78 | colorize_func = str # do not colorize by default 79 | 80 | if address not in meta_cache: 81 | epilog += "chunk address not found in metadata database\n" 82 | return None, suffix, epilog, colorize_func 83 | 84 | # This allows calling get_metadata() by not specifying any metadata 85 | # but meaning we want to retrieve them all 86 | if list_metadata == "all": 87 | list_metadata = list(meta_cache[address].keys()) 88 | if "backtrace" in list_metadata: 89 | # enforce retrieving all the functions from the backtrace 90 | list_metadata.remove("backtrace") 91 | list_metadata.append("backtrace:-1") 92 | 93 | opened = False 94 | for key in list_metadata: 95 | param = None 96 | if ":" in key: 97 | key, param = key.split(":") 98 | if key not in meta_cache[address]: 99 | if key != "color": 100 | suffix += " | N/A" 101 | epilog += "'%s' key not found in metadata database\n" % key 102 | opened = True 103 | L.append(None) 104 | continue 105 | if key == "backtrace": 106 | if param == None: 107 | funcs_list = get_first_function(address) 108 | else: 109 | funcs_list = get_functions(address, max_len=int(param)) 110 | if funcs_list == None: 111 | suffix += " | N/A" 112 | elif len(funcs_list) == 0: 113 | # XXX - atm if we failed to parse the functions from the debugger 114 | # we will also show "filtered" even if it is not the case 115 | suffix += " | filtered" 116 | else: 117 | suffix += " | %s" % ",".join(funcs_list) 118 | epilog += "%s" % meta_cache[address]["backtrace"]["raw"] 119 | L.append(funcs_list) 120 | opened = True 121 | elif key == "color": 122 | color = meta_cache[address][key] 123 | colorize_func = colorize_table[color] 124 | else: 125 | suffix += " | %s" % meta_cache[address][key] 126 | epilog += "%s\n" % meta_cache[address][key] 127 | L.append(meta_cache[address][key]) 128 | opened = True 129 | if opened: 130 | suffix += " |" 131 | 132 | return L, suffix, epilog, colorize_func 133 | 134 | def get_first_function(address): 135 | return get_functions(address, max_len=1) 136 | 137 | def get_functions(address, max_len=None): 138 | L = [] 139 | if address not in meta_cache: 140 | return None 141 | if "backtrace" not in meta_cache[address]: 142 | return None 143 | funcs = meta_cache[address]["backtrace"]["funcs"] 144 | for f in funcs: 145 | if f in backtrace_ignore: 146 | continue 147 | L.append(f) 148 | if max_len != None and len(L) == max_len: 149 | break 150 | return L 151 | 152 | class ptmeta(ptcmd.ptcmd): 153 | """Command to manage metadata for a given address""" 154 | 155 | def __init__(self, ptm): 156 | log.debug("ptmeta.__init__()") 157 | super(ptmeta, self).__init__(ptm, "ptmeta") 158 | 159 | self.parser = argparse.ArgumentParser( 160 | description="""Handle metadata associated with chunk addresses""", 161 | formatter_class=argparse.RawTextHelpFormatter, 162 | add_help=False, 163 | epilog="""NOTE: use 'ptmeta -h' to get more usage info""") 164 | self.parser.add_argument( 165 | "-v", "--verbose", dest="verbose", action="count", default=0, 166 | help="Use verbose output (multiple for more verbosity)" 167 | ) 168 | self.parser.add_argument( 169 | "-h", "--help", dest="help", action="store_true", default=False, 170 | help="Show this help" 171 | ) 172 | 173 | actions = self.parser.add_subparsers( 174 | help="Action to perform", 175 | dest="action" 176 | ) 177 | 178 | add_parser = actions.add_parser( 179 | "add", 180 | help="""Save metadata for a given chunk address""", 181 | formatter_class=argparse.RawTextHelpFormatter, 182 | epilog="""The saved metadata can then be shown in any other commands like 183 | 'ptlist', 'ptchunk', 'pfree', etc. 184 | 185 | E.g. 186 | ptmeta add mem-0x10 tag "service_user struct" 187 | ptmeta add 0xdead0030 color green 188 | ptmeta add 0xdead0030 backtrace""" 189 | ) 190 | add_parser.add_argument( 191 | 'address', 192 | help='Address to link the metadata to' 193 | ) 194 | add_parser.add_argument( 195 | 'key', 196 | help='Key name of the metadata (e.g. "backtrace", "color", "tag" or any name)' 197 | ) 198 | add_parser.add_argument( 199 | 'value', nargs="?", 200 | help='Value of the metadata, associated with the key (required except when adding a "backtrace")' 201 | ) 202 | 203 | del_parser = actions.add_parser( 204 | "del", 205 | help="""Delete metadata associated with a given chunk address""", 206 | formatter_class=argparse.RawTextHelpFormatter, 207 | epilog="""E.g. 208 | ptmeta del mem-0x10 209 | ptmeta del 0xdead0030""" 210 | ) 211 | del_parser.add_argument('address', help='Address to remove the metadata for') 212 | 213 | list_parser = actions.add_parser( 214 | "list", 215 | help="""List metadata for a chunk address or all chunk addresses (debugging)""", 216 | formatter_class=argparse.RawTextHelpFormatter, 217 | epilog="""E.g. 218 | ptmeta list mem-0x10 219 | ptmeta list 0xdead0030 -M backtrace 220 | ptmeta list 221 | ptmeta list -vvvv 222 | ptmeta list -M "tag, backtrace:3""" 223 | ) 224 | list_parser.add_argument( 225 | 'address', nargs="?", 226 | help='Address to remove the metadata for' 227 | ) 228 | list_parser.add_argument( 229 | "-M", "--metadata", dest="metadata", type=str, default=None, 230 | help="Comma separated list of metadata to print" 231 | ) 232 | 233 | config_parser = actions.add_parser( 234 | "config", 235 | help="Configure general metadata behaviour", 236 | formatter_class=argparse.RawTextHelpFormatter, 237 | epilog="""E.g. 238 | ptmeta config ignore backtrace _nl_make_l10nflist __GI___libc_free""" 239 | ) 240 | config_parser.add_argument( 241 | 'feature', 242 | help='Feature to configure (e.g. "ignore")' 243 | ) 244 | config_parser.add_argument( 245 | 'key', 246 | help='Key name of the metadata (e.g. "backtrace")' 247 | ) 248 | config_parser.add_argument( 249 | 'values', nargs="+", 250 | help='Values of the metadata, associated with the key (e.g. list of function to ignore in a backtrace)' 251 | ) 252 | 253 | # allows to enable a different log level during development/debugging 254 | self.parser.add_argument( 255 | "--loglevel", dest="loglevel", default=None, 256 | help=argparse.SUPPRESS 257 | ) 258 | # allows to save metadata to file during development/debugging 259 | self.parser.add_argument( 260 | "-S", "--save-db", dest="save", action="store_true", default=False, 261 | help=argparse.SUPPRESS 262 | ) 263 | # allows to load metadata from file during development/debugging 264 | self.parser.add_argument( 265 | "-L", "--load-db", dest="load", action="store_true", default=False, 266 | help=argparse.SUPPRESS 267 | ) 268 | 269 | @h.catch_exceptions 270 | @ptcmd.ptcmd.init_and_cleanup 271 | def invoke(self, arg, from_tty): 272 | """Inherited from gdb.Command 273 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 274 | """ 275 | 276 | log.debug("ptmeta.invoke()") 277 | 278 | if self.args.action is None and not self.args.save and not self.args.load: 279 | pu.print_error("WARNING: requires an action") 280 | self.parser.print_help() 281 | return 282 | 283 | if self.args.action == "list" \ 284 | or self.args.action == "add" \ 285 | or self.args.action == "del": 286 | address = None 287 | if self.args.address != None: 288 | addresses = self.dbg.parse_address(self.args.address) 289 | if len(addresses) == 0: 290 | pu.print_error("WARNING: No valid address supplied") 291 | self.parser.print_help() 292 | return 293 | address = addresses[0] 294 | 295 | if self.args.action == "list": 296 | self.list_metadata(address) 297 | return 298 | 299 | if self.args.action == "del": 300 | self.delete_metadata(address) 301 | return 302 | 303 | if self.args.action == "config": 304 | self.configure_metadata(self.args.feature, self.args.key, self.args.values) 305 | return 306 | 307 | if self.args.action == "add": 308 | self.add_metadata(address, self.args.key, self.args.value) 309 | return 310 | 311 | if self.args.save: 312 | if self.args.verbose >= 0: # always print since debugging feature 313 | print("Saving metadata database to file...") 314 | save_metadata_to_file(METADATA_DB) 315 | return 316 | 317 | if self.args.load: 318 | if self.args.verbose >= 0: # always print since debugging feature 319 | print("Loading metadata database from file...") 320 | load_metadata_from_file(METADATA_DB) 321 | return 322 | 323 | def list_metadata(self, address): 324 | """Show the metadata database for all addresses or a given address 325 | 326 | if verbose == 0, shows single-line entries (no "backtrace" if not requested) 327 | if verbose == 1, shows single-line entries (all keys) 328 | if verbose == 2, shows multi-line entries (no "backtrace" if not requested) 329 | if verbose == 3, shows multi-line entries (all keys) 330 | """ 331 | 332 | if len(meta_cache) != 0: 333 | pu.print_header("Metadata database", end=None) 334 | 335 | if self.args.metadata == None: 336 | # if no metadata provided by user, we get them all 337 | list_metadata = [] 338 | for k, d in meta_cache.items(): 339 | for k2, d2 in d.items(): 340 | if k2 not in list_metadata: 341 | list_metadata.append(k2) 342 | if self.args.verbose == 0 and "backtrace" in list_metadata: 343 | list_metadata.remove("backtrace") 344 | else: 345 | list_metadata = [e.strip() for e in self.args.metadata.split(",")] 346 | 347 | if self.args.verbose <= 1: 348 | print("| address | ", end="") 349 | print(" | ".join(list_metadata), end="") 350 | print(" |") 351 | for k, d in meta_cache.items(): 352 | if address == None or k == address: 353 | L, s, e, colorize_func = get_metadata(k, list_metadata=list_metadata) 354 | addr = colorize_func(f"0x{k:x}") 355 | print(f"| {addr}", end="") 356 | print(s) 357 | else: 358 | for k, d in meta_cache.items(): 359 | if address == None or k == address: 360 | L, s, e, colorize_func = get_metadata(k, list_metadata=list_metadata) 361 | addr = colorize_func(f"0x{k:x}") 362 | print(f"{addr}:") 363 | print(e) 364 | else: 365 | pu.print_header("Metadata database", end=None) 366 | print("N/A") 367 | 368 | print("") 369 | 370 | if len(backtrace_ignore) != 0: 371 | pu.print_header("Function ignore list for backtraces", end=None) 372 | pprint.pprint(backtrace_ignore) 373 | else: 374 | pu.print_header("Function ignore list for backtraces", end=None) 375 | print("N/A") 376 | 377 | def configure_metadata(self, feature, key, values): 378 | """Save given metadata (key, values) for a given feature (e.g. "backtrace") 379 | 380 | :param feature: name of the feature (e.g. "ignore") 381 | :param key: name of the metadata (e.g. "backtrace") 382 | :param values: list of values to associate to the key 383 | """ 384 | 385 | if self.args.verbose >= 1: 386 | print("Configuring metadata database...") 387 | if key == "backtrace": 388 | if feature == "ignore": 389 | backtrace_ignore.update(values) 390 | else: 391 | pu.print_error("WARNING: Unsupported feature") 392 | return 393 | else: 394 | pu.print_error("WARNING: Unsupported key") 395 | return 396 | 397 | def delete_metadata(self, address): 398 | """Delete metadata for a given chunk's address 399 | """ 400 | 401 | if address not in meta_cache: 402 | return 403 | 404 | if self.args.verbose >= 1: 405 | print(f"Deleting metadata for {address} from database...") 406 | del meta_cache[address] 407 | 408 | def add_metadata(self, address, key, value): 409 | """Save given metadata (key, value) for a given chunk's address 410 | E.g. key = "tag" and value is an associated user-defined tag 411 | """ 412 | 413 | if self.args.verbose >= 1: 414 | print("Adding to metadata database...") 415 | if key == "backtrace": 416 | result = self.dbg.get_backtrace() 417 | elif key == "color": 418 | if value not in colorize_table: 419 | pu.print_error(f"ERROR: Unsupported color. Need one of: {', '.join(colorize_table.keys())}") 420 | return 421 | result = value 422 | else: 423 | result = value 424 | 425 | if address not in meta_cache: 426 | meta_cache[address] = {} 427 | meta_cache[address][key] = result 428 | 429 | -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptparam.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import binascii 6 | import struct 7 | import sys 8 | import logging 9 | 10 | from libptmalloc.frontend import printutils as pu 11 | from libptmalloc.ptmalloc import ptmalloc as pt 12 | from libptmalloc.frontend import helpers as h 13 | from libptmalloc.frontend.commands.gdb import ptcmd 14 | 15 | log = logging.getLogger("libptmalloc") 16 | log.trace("ptparam.py") 17 | 18 | try: 19 | import gdb 20 | except ImportError: 21 | print("Not running inside of GDB, exiting...") 22 | raise Exception("sys.exit()") 23 | 24 | class ptparam(ptcmd.ptcmd): 25 | """Command to print information about malloc parameters represented by the malloc_par structure 26 | """ 27 | 28 | def __init__(self, ptm): 29 | log.debug("ptparam.__init__()") 30 | super(ptparam, self).__init__(ptm, "ptparam") 31 | 32 | self.parser = argparse.ArgumentParser( 33 | description="""Print malloc parameter(s) information 34 | 35 | Analyze the malloc_par structure's fields.""", 36 | add_help=False, 37 | formatter_class=argparse.RawTextHelpFormatter, 38 | epilog='NOTE: Last defined mp_ will be cached for future use') 39 | # self.parser.add_argument( 40 | # "-v", "--verbose", dest="verbose", action="count", default=0, 41 | # help="Use verbose output (multiple for more verbosity)" 42 | # ) 43 | self.parser.add_argument( 44 | "-h", "--help", dest="help", action="store_true", default=False, 45 | help="Show this help" 46 | ) 47 | self.parser.add_argument( 48 | "-l", dest="list", action="store_true", default=False, 49 | help="List malloc parameter(s)' address only" 50 | ) 51 | self.parser.add_argument( 52 | "--use-cache", dest="use_cache", action="store_true", default=False, 53 | help="Do not fetch parameters data if you know they haven't changed since last time they were cached" 54 | ) 55 | self.parser.add_argument( 56 | "address", default=None, nargs="?", type=h.string_to_int, 57 | help="A malloc_par struct address. Optional with cached malloc parameters" 58 | ) 59 | # allows to enable a different log level during development/debugging 60 | self.parser.add_argument( 61 | "--loglevel", dest="loglevel", default=None, 62 | help=argparse.SUPPRESS 63 | ) 64 | 65 | @h.catch_exceptions 66 | @ptcmd.ptcmd.init_and_cleanup 67 | def invoke(self, arg, from_tty): 68 | """Inherited from gdb.Command 69 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 70 | """ 71 | 72 | log.debug("ptparam.invoke()") 73 | 74 | self.cache.update_param(self.args.address, show_status=True, use_cache=self.args.use_cache) 75 | 76 | if self.args.list: 77 | self.list_parameters() 78 | return 79 | 80 | print(self.cache.par) 81 | 82 | def list_parameters(self): 83 | """List malloc parameter(s)' address only""" 84 | 85 | par = self.cache.par 86 | 87 | print("Parameter(s) found:", end="\n") 88 | print(" parameter @ ", end="") 89 | pu.print_header("{:#x}".format(int(par.address)), end="\n") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/ptstats.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import sys 5 | import logging 6 | import argparse 7 | 8 | from libptmalloc.frontend import printutils as pu 9 | from libptmalloc.ptmalloc import malloc_chunk as mc 10 | from libptmalloc.ptmalloc import malloc_par as mp 11 | from libptmalloc.ptmalloc import malloc_state as ms 12 | from libptmalloc.ptmalloc import ptmalloc as pt 13 | from libptmalloc.frontend import helpers as h 14 | from libptmalloc.frontend.commands.gdb import ptcmd 15 | 16 | log = logging.getLogger("libptmalloc") 17 | log.trace("ptstats.py") 18 | 19 | try: 20 | import gdb 21 | except ImportError: 22 | print("Not running inside of GDB, exiting...") 23 | raise Exception("sys.exit()") 24 | 25 | class ptstats(ptcmd.ptcmd): 26 | """Command to print general malloc stats, adapted from malloc.c mSTATs()""" 27 | 28 | def __init__(self, ptm): 29 | log.debug("ptstats.__init__()") 30 | super(ptstats, self).__init__(ptm, "ptstats") 31 | 32 | self.parser = argparse.ArgumentParser( 33 | description="""Print memory alloc statistics similar to malloc_stats(3)""", 34 | add_help=False, 35 | formatter_class=argparse.RawTextHelpFormatter, 36 | ) 37 | self.parser.add_argument( 38 | "-v", "--verbose", dest="verbose", action="count", default=0, 39 | help="Use verbose output (multiple for more verbosity)" 40 | ) 41 | self.parser.add_argument( 42 | "-h", "--help", dest="help", action="store_true", default=False, 43 | help="Show this help" 44 | ) 45 | # allows to enable a different log level during development/debugging 46 | self.parser.add_argument( 47 | "--loglevel", dest="loglevel", default=None, 48 | help=argparse.SUPPRESS 49 | ) 50 | 51 | @h.catch_exceptions 52 | @ptcmd.ptcmd.init_and_cleanup 53 | def invoke(self, arg, from_tty): 54 | """Inherited from gdb.Command 55 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 56 | """ 57 | 58 | # We don't yet update the tcache, arena and param structs as well as the 59 | # tcache bins, tcache bins and unsorted/small/large bins 60 | # yet as we need to do it for every single arena so will do when browsing them 61 | self.show_stats() 62 | 63 | def show_stats(self): 64 | """Show a summary of the memory statistics 65 | """ 66 | 67 | self.cache.update_arena(show_status=False) 68 | self.cache.update_param(show_status=False) 69 | self.cache.update_tcache(show_status=False) 70 | self.cache.update_tcache_bins(show_status=False) 71 | 72 | main_arena_address = self.cache.main_arena_address 73 | par = self.cache.par 74 | 75 | in_use_b = par.mmapped_mem 76 | avail_b = 0 77 | system_b = in_use_b 78 | 79 | pu.print_title("Malloc Stats", end="\n\n") 80 | 81 | arena = 0 82 | mstate = ms.malloc_state( 83 | self.ptm, main_arena_address, debugger=self.dbg, version=self.version 84 | ) 85 | while 1: 86 | 87 | self.cache.update_arena(address=mstate.address, show_status=False) 88 | self.cache.update_fast_bins(show_status=False) 89 | self.cache.update_bins(show_status=False) 90 | 91 | if mstate.address == self.cache.main_arena_address: 92 | sbrk_base, _ = self.dbg.get_heap_address(par) 93 | else: 94 | sbrk_base = (mstate.address + mstate.size + self.ptm.MALLOC_ALIGN_MASK) & ~self.ptm.MALLOC_ALIGN_MASK 95 | 96 | avail = 0 97 | inuse = 0 98 | nblocks = 1 99 | addr = sbrk_base 100 | while True: 101 | p = mc.malloc_chunk( 102 | self.ptm, 103 | addr, 104 | read_data=False, 105 | debugger=self.dbg, 106 | use_cache=True 107 | ) 108 | 109 | if p.address == self.ptm.top(self.cache.mstate): 110 | avail += self.ptm.chunksize(p) 111 | break 112 | 113 | if p.type == pt.chunk_type.FREE_FAST: 114 | avail += self.ptm.chunksize(p) 115 | elif p.type == pt.chunk_type.FREE_TCACHE: 116 | avail += self.ptm.chunksize(p) 117 | elif p.type == pt.chunk_type.INUSE: 118 | inuse += self.ptm.chunksize(p) 119 | else: 120 | avail += self.ptm.chunksize(p) 121 | nblocks += 1 122 | 123 | addr = self.ptm.next_chunk(p) 124 | 125 | pu.print_header("Arena {} @ {:#x}:".format(arena, mstate.address), end="\n") 126 | print("{:16} = ".format("system bytes"), end="") 127 | pu.print_value("{} ({:#x})".format(mstate.max_system_mem, mstate.max_system_mem), end="\n") 128 | print("{:16} = ".format("free bytes"), end="") 129 | pu.print_value("{} ({:#x})".format(avail, avail), end="\n") 130 | print("{:16} = ".format("in use bytes"), end="") 131 | pu.print_value("{} ({:#x})".format(inuse, inuse), end="\n") 132 | 133 | system_b += mstate.max_system_mem 134 | avail_b += avail 135 | in_use_b += inuse 136 | 137 | if mstate.next == main_arena_address: 138 | break 139 | else: 140 | next_addr = self.dbg.format_address(mstate.next) 141 | mstate = ms.malloc_state( 142 | self.ptm, next_addr, debugger=self.dbg, version=self.version 143 | ) 144 | arena += 1 145 | 146 | pu.print_header("\nTotal (including mmap):", end="\n") 147 | print("{:16} = ".format("system bytes"), end="") 148 | pu.print_value("{} ({:#x})".format(system_b, system_b), end="\n") 149 | print("{:16} = ".format("free bytes"), end="") 150 | pu.print_value("{} ({:#x})".format(avail_b, avail_b), end="\n") 151 | print("{:16} = ".format("in use bytes"), end="") 152 | pu.print_value("{} ({:#x})".format(in_use_b, in_use_b), end="\n") 153 | 154 | if self.version <= 2.23: 155 | # catch the error before we print anything 156 | val = par.max_total_mem 157 | 158 | print("{:16} = ".format("max system bytes"), end="") 159 | pu.print_value("{}".format(val), end="\n") 160 | 161 | print("{:16} = ".format("max mmap regions"), end="") 162 | pu.print_value("{}".format(par.max_n_mmaps), end="\n") 163 | print("{:16} = ".format("max mmap bytes"), end="") 164 | pu.print_value("{}".format(par.max_mmapped_mem), end="\n") -------------------------------------------------------------------------------- /libptmalloc/frontend/commands/gdb/pttcache.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from __future__ import print_function 3 | 4 | import argparse 5 | import struct 6 | import sys 7 | import logging 8 | 9 | from libptmalloc.frontend import printutils as pu 10 | from libptmalloc.ptmalloc import ptmalloc as pt 11 | from libptmalloc.frontend import helpers as h 12 | from libptmalloc.ptmalloc import malloc_chunk as mc 13 | from libptmalloc.frontend.commands.gdb import ptchunk 14 | from libptmalloc.frontend.commands.gdb import ptfree 15 | from libptmalloc.frontend.commands.gdb import ptcmd 16 | 17 | log = logging.getLogger("libptmalloc") 18 | log.trace("pttcache.py") 19 | 20 | try: 21 | import gdb 22 | except ImportError: 23 | print("Not running inside of GDB, exiting...") 24 | raise Exception("sys.exit()") 25 | 26 | class pttcache(ptcmd.ptcmd): 27 | """Command to walk and print the tcache bins 28 | 29 | Also see ptchunk description""" 30 | 31 | def __init__(self, ptm): 32 | log.debug("pttcache.__init__()") 33 | super(pttcache, self).__init__(ptm, "pttcache") 34 | 35 | self.parser = argparse.ArgumentParser( 36 | description="""Print tcache bins information 37 | 38 | All these bins are part of the tcache_perthread_struct structure. 39 | tcache is only available from glibc 2.26""", 40 | add_help=False, 41 | formatter_class=argparse.RawTextHelpFormatter, 42 | ) 43 | self.parser.add_argument( 44 | "address", default=None, nargs="?", type=h.string_to_int, 45 | help="An optional tcache address" 46 | ) 47 | self.parser.add_argument( 48 | "-l", dest="list", action="store_true", default=False, 49 | help="List tcache(s)' addresses only" 50 | ) 51 | self.parser.add_argument( 52 | "-i", "--index", dest="index", default=None, type=int, 53 | help="Index to the tcache bin to show (0 to 63)" 54 | ) 55 | self.parser.add_argument( 56 | "-b", "--bin-size", dest="size", default=None, type=h.string_to_int, 57 | help="Tcache bin size to show" 58 | ) 59 | # "ptchunk" also has this argument but default and help is different 60 | self.parser.add_argument( 61 | "-c", "--count", dest="count", type=h.check_positive, default=None, 62 | help="Maximum number of chunks to print in each bin" 63 | ) 64 | # other arguments are implemented in the "ptchunk" command 65 | # and will be shown after the above 66 | ptchunk.ptchunk.add_arguments(self) 67 | 68 | @h.catch_exceptions 69 | @ptcmd.ptcmd.init_and_cleanup 70 | def invoke(self, arg, from_tty): 71 | """Inherited from gdb.Command 72 | See https://sourceware.org/gdb/current/onlinedocs/gdb/Commands-In-Python.html 73 | """ 74 | 75 | log.debug("pttcache.invoke()") 76 | 77 | if not self.ptm.is_tcache_enabled(): 78 | print("tcache is currently disabled. Check glibc version or manually overide the tcache settings") 79 | return 80 | if not self.ptm.tcache_available: 81 | print("tcache is not currently available. Your target binary does not use threads to leverage tcache?") 82 | return 83 | 84 | self.cache.update_tcache(self.args.address, show_status=self.args.debug, use_cache=self.args.use_cache) 85 | # This is required by ptchunk.parse_many() 86 | self.cache.update_arena(show_status=self.args.debug, use_cache=self.args.use_cache) 87 | self.cache.update_param(show_status=self.args.debug, use_cache=self.args.use_cache) 88 | 89 | # This is required by show_one_bin(), see description 90 | self.args.real_count = self.args.count 91 | 92 | if self.args.list: 93 | self.list_tcaches() 94 | return 95 | 96 | if self.args.index != None and self.args.size != None: 97 | pu.print_error("Only one of -i and -s can be provided") 98 | return 99 | 100 | log.debug("tcache_address = 0x%x" % self.cache.tcache.address) 101 | if self.args.index == None and self.args.size == None: 102 | if self.args.verbose == 0: 103 | print(self.cache.tcache.to_summary_string()) 104 | elif self.args.verbose == 1: 105 | print(self.cache.tcache) 106 | elif self.args.verbose == 2: 107 | print(self.cache.tcache.to_string(verbose=True)) 108 | else: 109 | ptfree.ptfree.show_one_bin(self, "tcache", index=self.args.index, size=self.args.size, use_cache=self.args.use_cache) 110 | 111 | def list_tcaches(self): 112 | """List tcache addresses""" 113 | 114 | tcache = self.cache.tcache 115 | 116 | print("Tcache(s) found:", end="\n") 117 | print(" tcache @ ", end="") 118 | pu.print_header("{:#x}".format(int(tcache.address)), end="\n") 119 | 120 | # XXX - support the "size" argument if needed 121 | @staticmethod 122 | def is_in_tcache(address, ptm, dbg=None, size=None, index=None, use_cache=False): 123 | """Similar to ptfast.is_in_fastbin() but for tcache""" 124 | 125 | if not ptm.is_tcache_enabled() or not ptm.tcache_available: 126 | return False # address can't be in tcache bins if tcache is disabled globally :) 127 | 128 | if index != None: 129 | ptm.cache.update_tcache_bins(use_cache=use_cache, bins_list=[index]) 130 | if address in ptm.cache.tcache_bins[index]: 131 | return True 132 | else: 133 | return False 134 | else: 135 | ptm.cache.update_tcache_bins(use_cache=use_cache) 136 | for i in range(0, ptm.TCACHE_MAX_BINS): 137 | if address in ptm.cache.tcache_bins[i]: 138 | return True 139 | return False -------------------------------------------------------------------------------- /libptmalloc/frontend/frontend_gdb.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | 4 | log = logging.getLogger("libptmalloc") 5 | log.trace(f"frontend_gdb.py") 6 | 7 | from libptmalloc.frontend.commands.gdb import ptfast 8 | from libptmalloc.frontend.commands.gdb import pthelp 9 | from libptmalloc.frontend.commands.gdb import ptlist 10 | from libptmalloc.frontend.commands.gdb import ptstats 11 | from libptmalloc.frontend.commands.gdb import ptchunk 12 | from libptmalloc.frontend.commands.gdb import ptfree 13 | from libptmalloc.frontend.commands.gdb import ptbin 14 | from libptmalloc.frontend.commands.gdb import ptarena 15 | from libptmalloc.frontend.commands.gdb import ptparam 16 | from libptmalloc.frontend.commands.gdb import ptmeta 17 | from libptmalloc.frontend.commands.gdb import ptconfig 18 | from libptmalloc.frontend.commands.gdb import pttcache 19 | 20 | class frontend_gdb: 21 | """Register commands with GDB""" 22 | 23 | def __init__(self, ptm): 24 | 25 | # We share ptm (globals as well as cached info (such as the mstate)) 26 | # among all commands below 27 | 28 | # The below dictates in what order they will be shown in gdb 29 | cmds = [] 30 | cmds.append(ptconfig.ptconfig(ptm)) 31 | cmds.append(ptmeta.ptmeta(ptm)) 32 | cmds.append(ptarena.ptarena(ptm)) 33 | cmds.append(ptparam.ptparam(ptm)) 34 | cmds.append(ptlist.ptlist(ptm)) 35 | cmds.append(ptchunk.ptchunk(ptm)) 36 | cmds.append(ptbin.ptbin(ptm)) 37 | cmds.append(ptfast.ptfast(ptm)) 38 | cmds.append(pttcache.pttcache(ptm)) 39 | cmds.append(ptfree.ptfree(ptm)) 40 | cmds.append(ptstats.ptstats(ptm)) 41 | 42 | pthelp.pthelp(ptm, cmds) 43 | 44 | output = ptm.dbg.execute("ptconfig") 45 | print(output) 46 | -------------------------------------------------------------------------------- /libptmalloc/frontend/helpers.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import sys 3 | import traceback 4 | import argparse 5 | from functools import wraps 6 | 7 | def show_last_exception(): 8 | """Taken from gef. Let us see proper backtraces from python exceptions""" 9 | 10 | PYTHON_MAJOR = sys.version_info[0] 11 | horizontal_line = "-" 12 | right_arrow = "->" 13 | down_arrow = "\\->" 14 | 15 | print("") 16 | exc_type, exc_value, exc_traceback = sys.exc_info() 17 | print(" Exception raised ".center(80, horizontal_line)) 18 | print("{}: {}".format(exc_type.__name__, exc_value)) 19 | print(" Detailed stacktrace ".center(80, horizontal_line)) 20 | for fs in traceback.extract_tb(exc_traceback)[::-1]: 21 | if PYTHON_MAJOR == 2: 22 | filename, lineno, method, code = fs 23 | else: 24 | try: 25 | filename, lineno, method, code = ( 26 | fs.filename, 27 | fs.lineno, 28 | fs.name, 29 | fs.line, 30 | ) 31 | except: 32 | filename, lineno, method, code = fs 33 | 34 | print( 35 | """{} File "{}", line {:d}, in {}()""".format( 36 | down_arrow, filename, lineno, method 37 | ) 38 | ) 39 | print(" {} {}".format(right_arrow, code)) 40 | 41 | def is_ascii(s): 42 | return all(c < 128 and c > 1 for c in s) 43 | 44 | # https://stackoverflow.com/questions/2556108/rreplace-how-to-replace-the-last-occurrence-of-an-expression-in-a-string 45 | def rreplace(s, old, new, occurrence): 46 | li = s.rsplit(old, occurrence) 47 | return new.join(li) 48 | 49 | def prepare_list(L): 50 | return rreplace(', '.join([str(x) for x in L]), ',', ' or', 1) 51 | 52 | def string_to_int(num): 53 | """Convert an integer or hex integer string to an int 54 | :returns: converted integer 55 | 56 | especially helpful for using ArgumentParser() 57 | """ 58 | if num.find("0x") != -1: 59 | return int(num, 16) 60 | else: 61 | return int(num) 62 | 63 | def catch_exceptions(f): 64 | "Decorator to catch exceptions" 65 | 66 | @wraps(f) 67 | def _catch_exceptions(*args, **kwargs): 68 | try: 69 | f(*args, **kwargs) 70 | except Exception: 71 | show_last_exception() 72 | return _catch_exceptions 73 | 74 | def check_positive(value): 75 | try: 76 | ivalue = int(value) 77 | except: 78 | raise argparse.ArgumentTypeError("%s is an invalid positive int value" % value) 79 | if ivalue <= 0: 80 | raise argparse.ArgumentTypeError("%s is an invalid positive int value" % value) 81 | return ivalue 82 | 83 | def check_count_value(value): 84 | if value == "unlimited": 85 | return None # unlimited 86 | try: 87 | ivalue = int(value) 88 | except: 89 | raise argparse.ArgumentTypeError("%s is an invalid int value" % value) 90 | if ivalue == 0: 91 | return None # unlimited 92 | 93 | return ivalue 94 | 95 | 96 | hexdump_units = [1, 2, 4, 8, "dps"] 97 | def check_hexdump_unit(value): 98 | """Especially helpful for using ArgumentParser() 99 | """ 100 | if value == "dps": 101 | return value 102 | 103 | try: 104 | ivalue = int(value) 105 | except: 106 | raise argparse.ArgumentTypeError("%s is not a valid hexdump unit" % value) 107 | if ivalue not in hexdump_units: 108 | raise argparse.ArgumentTypeError("%s is not a valid hexdump unit" % value) 109 | return ivalue 110 | -------------------------------------------------------------------------------- /libptmalloc/frontend/printutils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | # taken from https://github.com/cloudburst/libheap/blob/master/libheap/frontend/printutils.py 3 | 4 | from __future__ import print_function 5 | 6 | import re 7 | 8 | colors_enabled = True 9 | 10 | # bash color support, taken from pwndbg 11 | NORMAL = "\x1b[0m" 12 | BLACK = "\x1b[30m" 13 | RED = "\x1b[31m" 14 | GREEN = "\x1b[32m" 15 | YELLOW = "\x1b[33m" 16 | BLUE = "\x1b[34m" 17 | PURPLE = "\x1b[35m" 18 | CYAN = "\x1b[36m" 19 | LIGHT_GREY = LIGHT_GRAY = "\x1b[37m" 20 | FOREGROUND = "\x1b[39m" 21 | GREY = GRAY = "\x1b[90m" 22 | LIGHT_RED = "\x1b[91m" 23 | LIGHT_GREEN = "\x1b[92m" 24 | LIGHT_YELLOW = "\x1b[93m" 25 | LIGHT_BLUE = "\x1b[94m" 26 | LIGHT_PURPLE = "\x1b[95m" 27 | LIGHT_CYAN = "\x1b[96m" 28 | WHITE = "\x1b[97m" 29 | BOLD = "\x1b[1m" 30 | UNDERLINE = "\x1b[4m" 31 | 32 | 33 | def none(x): 34 | return str(x) 35 | 36 | 37 | def normal(x): 38 | return colorize(x, NORMAL) 39 | 40 | 41 | def black(x): 42 | return colorize(x, BLACK) 43 | 44 | 45 | def red(x): 46 | return colorize(x, RED) 47 | 48 | 49 | def green(x): 50 | return colorize(x, GREEN) 51 | 52 | 53 | def yellow(x): 54 | return colorize(x, YELLOW) 55 | 56 | 57 | def blue(x): 58 | return colorize(x, BLUE) 59 | 60 | 61 | def purple(x): 62 | return colorize(x, PURPLE) 63 | 64 | 65 | def cyan(x): 66 | return colorize(x, CYAN) 67 | 68 | 69 | def foreground(x): 70 | return colorize(x, FOREGROUND) 71 | 72 | 73 | def gray(x): 74 | return colorize(x, GRAY) 75 | 76 | 77 | def light_red(x): 78 | return colorize(x, LIGHT_RED) 79 | 80 | 81 | def light_green(x): 82 | return colorize(x, LIGHT_GREEN) 83 | 84 | 85 | def light_yellow(x): 86 | return colorize(x, LIGHT_YELLOW) 87 | 88 | 89 | def light_blue(x): 90 | return colorize(x, LIGHT_BLUE) 91 | 92 | 93 | def light_purple(x): 94 | return colorize(x, LIGHT_PURPLE) 95 | 96 | 97 | def light_cyan(x): 98 | return colorize(x, LIGHT_CYAN) 99 | 100 | 101 | def light_gray(x): 102 | return colorize(x, LIGHT_GRAY) 103 | 104 | 105 | def white(x): 106 | return colorize(x, WHITE) 107 | 108 | 109 | def bold(x): 110 | return colorize(x, BOLD) 111 | 112 | 113 | def underline(x): 114 | return colorize(x, UNDERLINE) 115 | 116 | 117 | def colorize(x, color): 118 | if colors_enabled: 119 | return color + terminateWith(str(x), color) + NORMAL 120 | else: 121 | return x 122 | 123 | 124 | def terminateWith(x, color): 125 | return re.sub('\x1b\\[0m', NORMAL + color, x) 126 | 127 | 128 | def print_debug(s, end='\n'): 129 | debug = "[#] {0}".format(s) 130 | color = LIGHT_PURPLE 131 | debug = colorize(debug, color) 132 | print(debug, end=end) 133 | 134 | def print_error(s, end="\n"): 135 | error = "[!] {0}".format(s) 136 | color = RED 137 | error = colorize(error, color) 138 | print(error, end=end) 139 | 140 | 141 | def print_title(s, end="\n"): 142 | print(color_title(s), end=end) 143 | 144 | 145 | def print_title_wide(s, end="\n"): 146 | width = 80 147 | lwidth = (width-len(s))/2 148 | rwidth = (width-len(s))/2 149 | title = '{:=<{lwidth}}{}{:=<{rwidth}}'.format( 150 | '', s, '', lwidth=lwidth, rwidth=rwidth) 151 | print(color_title(title), end=end) 152 | 153 | 154 | def print_header(s, end=""): 155 | color = YELLOW 156 | s = colorize(s, color) 157 | print(s, end=end) 158 | 159 | 160 | def print_footer(s, end=""): 161 | color = PURPLE 162 | s = colorize(s, color) 163 | print(s, end=end) 164 | 165 | 166 | def print_value(s, end=""): 167 | print(color_value(s), end=end) 168 | 169 | 170 | def color_header(s): 171 | color = YELLOW 172 | return colorize(s, color) 173 | 174 | def color_title(s): 175 | color = GREEN + UNDERLINE 176 | return colorize(s, color) 177 | 178 | 179 | def color_value(s): 180 | color = BLUE 181 | return colorize(s, color) 182 | 183 | 184 | def color_footer(s): 185 | color = PURPLE 186 | return colorize(s, color) 187 | -------------------------------------------------------------------------------- /libptmalloc/libptmalloc.cfg: -------------------------------------------------------------------------------- 1 | [Glibc] 2 | version = 2.27 3 | tcache = true 4 | #[OperatingSystem] 5 | #distribution = photon 6 | #release = 3.0 7 | -------------------------------------------------------------------------------- /libptmalloc/logger.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | 4 | # https://stackoverflow.com/questions/2183233/how-to-add-a-custom-loglevel-to-pythons-logging-facility 5 | def trace(self, message, *args, **kws): 6 | if self.isEnabledFor(logging.TRACE): 7 | # Yes, logger takes its '*args' as 'args'. 8 | self._log(logging.TRACE, message, args, **kws) 9 | 10 | class MyFormatter(logging.Formatter): 11 | """Defines how we format logs in stdout and files 12 | """ 13 | 14 | # We use the TRACE level to check loaded files in gdb after reloading the script 15 | # so is mainly useful during development 16 | logging.TRACE = 5 17 | logging.addLevelName(logging.TRACE, 'TRACE') 18 | logging.Logger.trace = trace 19 | 20 | FORMATS = { 21 | logging.ERROR: "(%(asctime)s) [!] %(msg)s", 22 | logging.WARNING: "(%(asctime)s) WARNING: %(msg)s", 23 | logging.INFO: "(%(asctime)s) [*] %(msg)s", 24 | logging.DEBUG: "(%(asctime)s) DBG: %(msg)s", 25 | logging.TRACE: "(%(asctime)s) TRACE: %(msg)s", 26 | "DEFAULT": "%(asctime)s - %(msg)s" 27 | } 28 | 29 | def format(self, record): 30 | """Hooked Formatter.format() method to modify its behaviour 31 | """ 32 | 33 | format_orig = self._style._fmt 34 | 35 | self._style._fmt = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT']) 36 | result = logging.Formatter.format(self, record) 37 | 38 | self._style._fmt = format_orig 39 | 40 | return result -------------------------------------------------------------------------------- /libptmalloc/ptmalloc/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | 4 | log = logging.getLogger("libptmalloc") 5 | log.trace("ptmalloc/__init__.py") -------------------------------------------------------------------------------- /libptmalloc/ptmalloc/cache.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import struct 3 | import sys 4 | import hexdump 5 | import logging 6 | 7 | from libptmalloc.ptmalloc import tcache_perthread as tp 8 | from libptmalloc.ptmalloc import malloc_chunk as mc 9 | from libptmalloc.ptmalloc import malloc_state as ms 10 | from libptmalloc.ptmalloc import malloc_par as mp 11 | 12 | log = logging.getLogger("libptmalloc") 13 | log.trace("cache.py") 14 | 15 | class cache: 16 | """Hold cached information such as objects representing ptmalloc structures, 17 | as well as chunk's addresses in the respective bins. 18 | 19 | Since browsing all these big structures and arrays can be slow in gdb, we cache 20 | them in this object.""" 21 | 22 | def __init__(self, ptm): 23 | self.ptm = ptm 24 | 25 | # Assume we can re-use known mstate when not specified 26 | self.main_arena_address = None 27 | self.mstate = None # latest cached arena i.e. malloc_mstate object used in all future commands 28 | self.par = None # latest cached parameters i.e. malloc_par object 29 | self.tcache = None # latest cached tcache i.e. tcache_perthread object 30 | 31 | # Arrays of arrays holding the addresses of all the chunks in the respective bins 32 | self.bins = None # even though we don't technically need to keep track of freed chunks 33 | # in unsorted/small/large bins to print chunks types with "ptchunk", it 34 | # is still handy to cache them when printing chunks in bins, hence why 35 | # we track them 36 | self.fast_bins = None 37 | self.tcache_bins = None 38 | 39 | def update_tcache(self, address=None, show_status=False, use_cache=False): 40 | """Update the tcache_perthread object 41 | 42 | :param address: tcache's name or address if don't want to use the cached or default one 43 | """ 44 | 45 | log.debug("cache.update_tcache()") 46 | 47 | if not self.ptm.is_tcache_enabled() or not self.ptm.tcache_available: 48 | return # nothing to be done if tcache disabled 49 | 50 | if address != None and address < 1024: 51 | raise Exception(f"Wrong tcache address: {address}?") 52 | 53 | if address != None: 54 | if show_status: 55 | print(f"Caching tcache @ {address:#x}") 56 | self.tcache = tp.tcache_perthread(self.ptm, address, debugger=self.ptm.dbg, version=self.ptm.version) 57 | if not self.tcache.initOK: 58 | raise Exception("Wrong tcache address?") 59 | elif self.tcache != None: 60 | if use_cache: 61 | if show_status: 62 | print("Using cached tcache") 63 | else: 64 | if show_status: 65 | print("Retrieving tcache again") 66 | tcache_address = self.tcache.address 67 | self.tcache = None # enforce retrieving it again below 68 | else: 69 | tcache_address = self.ptm.dbg.read_variable("tcache") 70 | if show_status: 71 | print(f"Caching global 'tcache' @ {int(tcache_address):#x}") 72 | 73 | if self.tcache == None: 74 | self.tcache = tp.tcache_perthread(self.ptm, tcache_address, debugger=self.ptm.dbg, version=self.ptm.version) 75 | 76 | def update_param(self, address=None, show_status=False, use_cache=False, invalid_ok=False): 77 | """Update the malloc_param object 78 | 79 | :param address: param's name or address if don't want to use the cached or default one 80 | """ 81 | 82 | log.debug("cache.update_param()") 83 | 84 | if address != None and address < 1024: 85 | raise Exception(f"Wrong mp address: {address}?") 86 | 87 | if address != None: 88 | if show_status: 89 | print(f"Caching malloc parameters @ {address:#x}") 90 | self.par = mp.malloc_par(self.ptm, address, debugger=self.ptm.dbg, version=self.ptm.version, invalid_ok=invalid_ok) 91 | if not self.par.initOK: 92 | raise Exception("Wrong mp address?") 93 | elif self.par != None: 94 | if use_cache: 95 | if show_status: 96 | print("Using cached malloc parameters") 97 | else: 98 | if show_status: 99 | print("Retrieving malloc parameters again") 100 | mp_address = self.par.address 101 | self.par = None # enforce retrieving it again below 102 | else: 103 | mp_ = self.ptm.dbg.read_variable_address("mp_") 104 | log.debug(f"mp = {mp_}") 105 | mp_address = self.ptm.dbg.format_address(mp_) 106 | log.debug(f"mp_address = {mp_address:#x}") 107 | if show_status: 108 | print(f"Caching global 'mp_' @ {mp_address:#x}") 109 | 110 | if self.par == None: 111 | self.par = mp.malloc_par(self.ptm, mp_address, debugger=self.ptm.dbg, version=self.ptm.version, invalid_ok=invalid_ok) 112 | 113 | def update_arena(self, address=None, show_status=False, use_cache=False): 114 | """Update the malloc_state object 115 | 116 | :param address: arena's name or address if don't want to use the cached or default one 117 | """ 118 | 119 | log.debug("cache.update_arena()") 120 | 121 | # XXX - also support thread_arena somehow, see libheap? 122 | # XXX - &main_arena == thread_arena? 123 | 124 | if address != None and address < 1024: 125 | raise Exception(f"Wrong arena address: {address}?") 126 | 127 | # The main arena address should never change so don't need 128 | # to retrieve it more than once (it stays in cache) 129 | if self.main_arena_address == None: 130 | if show_status: 131 | print("Retrieving 'main_arena'") 132 | main_arena = self.ptm.dbg.read_variable_address("main_arena") 133 | log.debug(f"main_arena = {main_arena}") 134 | main_arena_address = self.ptm.dbg.format_address(main_arena) 135 | log.debug(f"main_arena_address = {main_arena_address}") 136 | self.main_arena_address = main_arena_address 137 | 138 | if address != None: 139 | if show_status: 140 | print(f"Caching arena @ {address:#x}") 141 | self.mstate = ms.malloc_state(self.ptm, address, debugger=self.ptm.dbg, version=self.ptm.version) 142 | if not self.mstate.initOK: 143 | raise Exception("Wrong arena address?") 144 | elif self.mstate != None: 145 | if use_cache: 146 | if show_status: 147 | print("Using cached arena") 148 | else: 149 | if show_status: 150 | print("Retrieving arena again") 151 | mstate_address = self.mstate.address 152 | self.mstate = None # enforce retrieving it again below 153 | else: 154 | mstate_address = self.main_arena_address 155 | if show_status: 156 | print(f"Caching global 'main_arena' @ {mstate_address:#x}") 157 | 158 | if self.mstate == None: 159 | self.mstate = ms.malloc_state(self.ptm, mstate_address, debugger=self.ptm.dbg, version=self.ptm.version) 160 | 161 | # arena_address = self.ptm.dbg.read_variable_address("main_arena") 162 | # thread_arena = self.ptm.dbg.read_variable("thread_arena") 163 | # if thread_arena is not None: 164 | # thread_arena_address = self.ptm.dbg.format_address(thread_arena) 165 | # else: 166 | # thread_arena_address = arena_address 167 | 168 | # if address != None: 169 | # arena_address = address 170 | # else: 171 | # arena_address = thread_arena_address 172 | 173 | def update_bins(self, show_status=False, use_cache=False, bins_list=[]): 174 | """Fetches the chunks' addresses in the malloc_state.bins[] array 175 | and cache the information for future use 176 | 177 | :param bins_list: If non-empty, contains a list of indexes into the bin array 178 | that we update. It means the others won't be modified. It 179 | serves as an optimization so we can update only certain bins 180 | """ 181 | 182 | log.debug("cache.update_bins()") 183 | 184 | if self.bins != None: 185 | if use_cache: 186 | if show_status: 187 | print("Using cached unsorted/small/large bins") 188 | return 189 | else: 190 | if show_status: 191 | print("Retrieving unsorted/small/large bins again") 192 | self.bins = None 193 | else: 194 | if show_status: 195 | print("Retrieving unsorted/small/large bins") 196 | 197 | ptm = self.ptm 198 | dbg = self.ptm.dbg 199 | 200 | bins = [] 201 | for index in range(0, ptm.NBINS-1): 202 | if self.bins != None and bins_list and index not in bins_list: 203 | bins.append(self.bins[index]) 204 | else: 205 | bins.append(self.get_bin_chunks(index)) 206 | 207 | # Only update if no error to avoid caching incomplete info 208 | self.bins = bins 209 | 210 | def get_bin_chunks(self, index): 211 | """Fetches the chunks' addresses in the malloc_state.bins[] array 212 | for the specified index 213 | 214 | :return: the list of addresses in this specified bin 215 | """ 216 | 217 | log.debug("get_bin_chunks(%d)" % index) 218 | ptm = self.ptm 219 | mstate = ptm.cache.mstate 220 | dbg = self.ptm.dbg 221 | 222 | #ptm.mutex_lock(mstate) 223 | 224 | b = ptm.bin_at(mstate, index+1) 225 | if b == 0: # Not initialized yet 226 | return [] 227 | 228 | p = mc.malloc_chunk( 229 | ptm, 230 | b, 231 | inuse=False, 232 | debugger=dbg, 233 | tcache=False, 234 | fast=False, 235 | allow_invalid=True) 236 | 237 | addresses = [] 238 | while p.fd != int(b): 239 | addresses.append(p.address) 240 | p = mc.malloc_chunk( 241 | ptm, 242 | ptm.first(p), 243 | inuse=False, 244 | debugger=dbg, 245 | tcache=False, 246 | fast=False, 247 | allow_invalid=True) 248 | 249 | #ptm.mutex_unlock(mstate) 250 | 251 | return addresses 252 | 253 | def update_fast_bins(self, show_status=False, use_cache=False, bins_list=[]): 254 | """Fetches the chunks' addresses in the malloc_state.fastbinsY[] array 255 | and cache the information for future use 256 | 257 | :param bins_list: If non-empty, contains a list of indexes into the bin array 258 | that we update. It means the others won't be modified. It 259 | serves as an optimization so we can update only certain bins 260 | """ 261 | 262 | log.debug("cache.update_fast_bins()") 263 | 264 | if self.fast_bins != None: 265 | if use_cache: 266 | if show_status: 267 | print("Using cached fast bins") 268 | return 269 | else: 270 | if show_status: 271 | print("Retrieving fast bins again") 272 | self.fast_bins = None 273 | else: 274 | if show_status: 275 | print("Retrieving fast bins") 276 | 277 | ptm = self.ptm 278 | dbg = self.ptm.dbg 279 | 280 | fast_bins = [] 281 | for index in range(0, ptm.NFASTBINS): 282 | if self.fast_bins != None and bins_list and index not in bins_list: 283 | fast_bins.append(self.fast_bins[index]) 284 | else: 285 | fast_bins.append(self.get_fast_bin_chunks(index)) 286 | 287 | # Only update if no error to avoid caching incomplete info 288 | self.fast_bins = fast_bins 289 | 290 | def get_fast_bin_chunks(self, index): 291 | """Fetches the chunks' addresses in the malloc_state.fastbinsY[] array 292 | for the specified index 293 | 294 | :return: the list of addresses in this specified bin 295 | """ 296 | 297 | ptm = self.ptm 298 | mstate = ptm.cache.mstate 299 | dbg = self.ptm.dbg 300 | 301 | fb_base = int(mstate.address) + mstate.fastbins_offset 302 | 303 | p = mc.malloc_chunk( 304 | ptm, 305 | addr=fb_base - (2 * ptm.SIZE_SZ) + index * ptm.SIZE_SZ, 306 | fast=True, 307 | debugger=dbg, 308 | allow_invalid=True, 309 | ) 310 | 311 | addresses = [] 312 | while p.fd != 0: 313 | if p.fd is None: 314 | break 315 | addresses.append(p.fd) 316 | p = mc.malloc_chunk( 317 | ptm, 318 | p.fd, 319 | fast=True, 320 | debugger=dbg, 321 | allow_invalid=True, 322 | ) 323 | 324 | return addresses 325 | 326 | def update_tcache_bins(self, show_status=False, use_cache=False, bins_list=[]): 327 | """Fetches the chunks' addresses in the tcache_perthread_struct.entries[] array 328 | and cache the information for future use 329 | 330 | :param bins_list: If non-empty, contains a list of indexes into the bin array 331 | that we update. It means the others won't be modified. It 332 | serves as an optimization so we can update only certain bins 333 | """ 334 | 335 | log.debug("cache.update_tcache_bins()") 336 | 337 | if not self.ptm.is_tcache_enabled() or not self.ptm.tcache_available: 338 | return # nothing to be done if tcache disabled 339 | 340 | if self.tcache_bins != None: 341 | if use_cache: 342 | if show_status: 343 | print("Using cached tcache bins") 344 | return 345 | else: 346 | if show_status: 347 | print("Retrieving tcache bins again") 348 | self.tcache_bins = None 349 | else: 350 | if show_status: 351 | print("Retrieving tcache bins") 352 | 353 | ptm = self.ptm 354 | dbg = self.ptm.dbg 355 | 356 | tcache_bins = [] 357 | for index in range(0, ptm.TCACHE_MAX_BINS): 358 | if self.tcache_bins != None and bins_list and index not in bins_list: 359 | tcache_bins.append(self.tcache_bins[index]) 360 | else: 361 | tcache_bins.append(self.get_tcache_bin_chunks(index)) 362 | 363 | # Only update if no error to avoid caching incomplete info 364 | self.tcache_bins = tcache_bins 365 | 366 | def get_tcache_bin_chunks(self, index): 367 | """Fetches the chunks' addresses in the tcache_perthread_struct.entries[] array 368 | for the specified index 369 | 370 | :return: the list of addresses in this specified bin 371 | """ 372 | 373 | ptm = self.ptm 374 | tcache = ptm.cache.tcache 375 | dbg = self.ptm.dbg 376 | 377 | if tcache.entries[index] == 0: 378 | return [] 379 | # I've seen uninitialized entries[] still holding old data i.e. non-null 380 | # even though the counts is 0 381 | if tcache.counts[index] == 0: 382 | return [] 383 | 384 | addr = tcache.entries[index] - 2 * ptm.SIZE_SZ 385 | p = mc.malloc_chunk(ptm, addr, inuse=False, debugger=dbg, allow_invalid=True, tcache=True) 386 | if not p.initOK: # afaict should not happen in a normal scenario but better be safe 387 | return [] 388 | 389 | addresses = [] 390 | while True: 391 | addresses.append(p.address) 392 | if p.next == 0x0: 393 | break 394 | addr = p.next - 2 * ptm.SIZE_SZ 395 | p = mc.malloc_chunk(ptm, addr, inuse=False, debugger=dbg, allow_invalid=True, tcache=True) 396 | if not p.initOK: # same 397 | return addresses 398 | 399 | return addresses 400 | 401 | def update_all(self, show_status=False, use_cache=False, arena_address=None): 402 | self.update_arena(address=arena_address, show_status=show_status, use_cache=use_cache) 403 | self.update_param(show_status=show_status, use_cache=use_cache) 404 | self.update_tcache(show_status=show_status, use_cache=use_cache) 405 | 406 | self.update_fast_bins(show_status=show_status, use_cache=use_cache) 407 | self.update_tcache_bins(show_status=show_status, use_cache=use_cache) 408 | -------------------------------------------------------------------------------- /libptmalloc/ptmalloc/heap_structure.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import logging 3 | import struct 4 | 5 | log = logging.getLogger("libptmalloc") 6 | log.trace("heap_structure.py") 7 | 8 | class heap_structure(object): 9 | """Represent a general structure. Can be inherited by any structure like malloc_chunk. 10 | Allow factoring of functions used by many structures, so we don't duplicate code. 11 | """ 12 | 13 | def __init__(self, ptm, debugger=None): 14 | """XXX 15 | """ 16 | 17 | log.trace("heap_structure.__init__()") 18 | self.ptm = ptm 19 | self.is_x86 = self.ptm.SIZE_SZ == 4 # XXX - actually use that or delete? 20 | self.initOK = True 21 | self.address = None 22 | self.mem = None 23 | self.dbg = debugger 24 | 25 | def validate_address(self, address): 26 | """Valid that a given address can actually be used as chunk address 27 | """ 28 | log.trace("heap_structure.validate_address()") 29 | 30 | if address is None or address == 0 or type(address) != int: 31 | print("Invalid address") 32 | #raise Exception("Invalid address") 33 | self.initOK = False 34 | self.address = None 35 | return False 36 | else: 37 | self.address = address 38 | return True 39 | 40 | def unpack_variable(self, fmt, offset): 41 | return struct.unpack_from(fmt, self.mem, offset)[0] -------------------------------------------------------------------------------- /libptmalloc/ptmalloc/malloc_par.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import struct 3 | import sys 4 | import logging 5 | 6 | from libptmalloc.frontend import printutils as pu 7 | from libptmalloc.ptmalloc import heap_structure as hs 8 | 9 | log = logging.getLogger("libptmalloc") 10 | log.trace("malloc_par.py") 11 | 12 | class malloc_par(hs.heap_structure): 13 | "python representation of a struct malloc_par" 14 | 15 | CHUNK_ALIGNMENT = 16 16 | 17 | # XXX - we can probably get the version directly from the ptm argument? 18 | def __init__(self, ptm, addr=None, mem=None, debugger=None, version=None, invalid_ok=False): 19 | """ 20 | Parse malloc_par's data and initialize the malloc_par object 21 | 22 | :param ptm: ptmalloc object 23 | :param addr: address for a malloc_par where to read the structure's content from the debugger 24 | :param mem: alternatively to "addr", provides the memory bytes of that malloc_par's content 25 | :param debugger: the pydbg object 26 | :param version: the glibc version 27 | """ 28 | 29 | super(malloc_par, self).__init__(ptm, debugger=debugger) 30 | 31 | self.size = 0 # sizeof(struct malloc_par) 32 | self.invalid_ok = invalid_ok 33 | 34 | # malloc_par structure's fields, in this order for easy lookup 35 | # Note: commented ones have been added/removed at some point in glibc 36 | # so are not present in all glibc versions 37 | self.trim_threshold = 0 38 | self.top_pad = 0 39 | self.mmap_threshold = 0 40 | self.arena_test = 0 41 | self.arena_max = 0 42 | self.arena_stickiness = 0 # specific to photon 3.0 43 | self.n_mmaps = 0 44 | self.n_mmaps_max = 0 45 | self.max_n_mmaps = 0 46 | self.no_dyn_threshold = 0 47 | self.mmapped_mem = 0 48 | self.max_mmapped_mem = 0 49 | #self.max_total_mem = 0 # removed in 2.24 50 | self.sbrk_base = 0 51 | # below added in 2.26 when USE_TCACHE is set 52 | # self.tcache_bins = 0 53 | # self.tcache_max_bytes = 0 54 | # self.tcache_count = 0 55 | # self.tcache_unsorted_limit = 0 56 | 57 | if addr is None: 58 | if mem is None: 59 | pu.print_error("Please specify a struct malloc_par address") 60 | self.initOK = False 61 | return 62 | 63 | self.address = None 64 | else: 65 | self.address = addr 66 | 67 | if debugger is not None: 68 | self.dbg = debugger 69 | else: 70 | pu.print_error("Please specify a debugger") 71 | raise Exception("sys.exit()") 72 | 73 | if version is None: 74 | pu.print_error("Please specify a glibc version for malloc_par") 75 | raise Exception("sys.exit()") 76 | else: 77 | self.version = version 78 | 79 | self.initialize_sizes_and_offsets() 80 | 81 | if mem is None: 82 | # a string of raw memory was not provided, let's read it from the debugger 83 | try: 84 | self.mem = self.dbg.read_memory(addr, self.size) 85 | except TypeError: 86 | pu.print_error("Invalid address specified") 87 | self.initOK = False 88 | return 89 | except RuntimeError: 90 | pu.print_error("Could not read address {0:#x}".format(addr)) 91 | self.initOK = False 92 | return 93 | else: 94 | if len(mem) < self.size: 95 | pu.print_error("Provided memory size is too small for a malloc_par") 96 | self.initOK = False 97 | return 98 | self.mem = mem[:self.size] 99 | 100 | self.unpack_memory() 101 | 102 | def initialize_sizes_and_offsets(self): 103 | """Initialize malloc_par's specific sizes based on the glibc version and architecture 104 | """ 105 | 106 | self.size_sz = self.dbg.get_size_sz() 107 | 108 | if self.version < 2.15: 109 | # XXX - seems 2.14 has same fields as 2.15 so likely we can support 110 | # older easily... 111 | pu.print_error("Unsupported version for malloc_par") 112 | raise Exception('sys.exit()') 113 | 114 | if self.version >= 2.15 and self.version <= 2.23: 115 | if self.size_sz == 4: 116 | # sizeof(malloc_par) = 20 + 16 + 16 117 | self.size = 0x34 118 | elif self.size_sz == 8: 119 | # sizeof(malloc_par) = 40 + 16 + 32 120 | self.size = 0x58 121 | 122 | elif self.version >= 2.24 and self.version <= 2.25: 123 | # max_total_mem removed in 2.24 124 | if self.size_sz == 4: 125 | self.size = 0x30 126 | elif self.size_sz == 8: 127 | self.size = 0x50 128 | 129 | elif self.version >= 2.26: 130 | # tcache_* added in 2.26 131 | if self.ptm.is_tcache_enabled(): 132 | # USE_TCACHE is set 133 | if self.size_sz == 4: 134 | self.size = 0x40 135 | elif self.size_sz == 8: 136 | self.size = 0x70 137 | else: 138 | # revert to same sizes as [2.24, 2.25] if USE_TCACHE not set 139 | if self.size_sz == 4: 140 | self.size = 0x30 141 | elif self.size_sz == 8: 142 | self.size = 0x50 143 | if self.ptm.distribution == "photon" and self.ptm.release == "3.0": 144 | # arena_stickiness added for all 2.28 versions 145 | self.size += self.size_sz 146 | 147 | log.debug(f"malloc_par.size = {self.size:#x}") 148 | 149 | def unpack_memory(self): 150 | """Actually parse all the malloc_par's fields from the memory bytes (previously retrieved) 151 | """ 152 | 153 | if self.mem is None: 154 | pu.print_error("No memory found") 155 | raise Exception("sys.exit()") 156 | 157 | if self.size_sz == 4: 158 | fmt = "= 2.15 and self.version < 2.23: 106 | if self.size_sz == 4: 107 | # sizeof(malloc_state) = 4+4+40+4+4+(254*4)+16+4+4+4+4 108 | self.size = 0x450 109 | elif self.size_sz == 8: 110 | # sizeof(malloc_state) = 4+4+80+8+8+(254*8)+16+8+8+8+8 111 | self.size = 0x888 112 | 113 | self.fastbins_offset = 8 114 | self.bins_offset = self.fastbins_offset + 12 * self.size_sz 115 | 116 | elif self.version >= 2.23 and self.version <= 2.25: 117 | # attached_threads added in 2.23 118 | if self.size_sz == 4: 119 | self.size = 0x454 120 | elif self.size_sz == 8: 121 | self.size = 0x890 122 | 123 | self.fastbins_offset = 8 124 | self.bins_offset = self.fastbins_offset + 12 * self.size_sz 125 | 126 | elif self.version >= 2.27: 127 | # have_fastchunks added in 2.27 128 | if self.size_sz == 4: 129 | # hax, empiric: +4 for padding added after fastbinsY[] 130 | self.size = 0x458+4 131 | self.fastbins_offset = 0xC 132 | elif self.size_sz == 8: 133 | self.size = 0x898 134 | self.fastbins_offset = 0x10 135 | 136 | self.bins_offset = self.fastbins_offset + 12 * self.size_sz 137 | 138 | def unpack_memory(self): 139 | """Actually parse all the malloc_state's fields from the memory bytes (previously retrieved) 140 | """ 141 | 142 | if self.mem is None: 143 | pu.print_error("No memory found") 144 | raise Exception('sys.exit()') 145 | 146 | self.mutex = self.unpack_variable("= 2.27: 151 | # have_fastchunks added in 2.27 152 | if self.size_sz == 4: 153 | fmt = "= 2.27: 167 | if self.size_sz == 4: 168 | # hax, empiric: +4 for padding added after fastbinsY[] 169 | offset += 4 170 | 171 | if self.size_sz == 4: 172 | fmt = "= 2.23: 202 | # attached_threads added in 2.23 203 | self.attached_threads = self.unpack_variable(fmt, offset) 204 | offset = offset + self.size_sz 205 | 206 | self.system_mem = self.unpack_variable(fmt, offset) 207 | offset = offset + self.size_sz 208 | 209 | self.max_system_mem = self.unpack_variable(fmt, offset) 210 | 211 | # XXX - this is probably broken as we haven't used it yet 212 | def write(self, inferior=None): 213 | """Write malloc_state's data into memory using debugger 214 | """ 215 | 216 | if self.size_sz == 4: 217 | mem = struct.pack( 218 | "<275I", 219 | self.mutex, 220 | self.flags, 221 | self.fastbinsY, 222 | self.top, 223 | self.last_remainder, 224 | self.bins, 225 | self.binmap, 226 | self.next, 227 | self.system_mem, 228 | self.max_system_mem, 229 | ) 230 | elif self.size_sz == 8: 231 | mem = struct.pack( 232 | "= 2.27: 270 | txt += "\n{:16} = ".format("have_fastchunks") 271 | txt += pu.color_value("{:#x}".format(self.have_fastchunks)) 272 | txt += self.fastbins_to_string(verbose=verbose, use_cache=use_cache) 273 | txt += "\n{:16} = ".format("top") 274 | txt += pu.color_value("{:#x}".format(self.top)) 275 | txt += "\n{:16} = ".format("last_remainder") 276 | txt += pu.color_value("{:#x}".format(self.last_remainder)) 277 | txt += self.bins_to_string(verbose=verbose, use_cache=use_cache) 278 | if verbose > 0: 279 | for i in range(len(self.binmap)): 280 | txt += "\n{:16} = ".format("binmap[%d]" % i) 281 | txt += pu.color_value("{:#x}".format(self.binmap[i])) 282 | else: 283 | txt += "\n{:16} = ".format("binmap") 284 | txt += pu.color_value("{}".format("{...}")) 285 | txt += "\n{:16} = ".format("next") 286 | txt += pu.color_value("{:#x}".format(self.next)) 287 | txt += "\n{:16} = ".format("next_free") 288 | txt += pu.color_value("{:#x}".format(self.next_free)) 289 | if self.version >= 2.23: 290 | txt += "\n{:16} = ".format("attached_threads") 291 | txt += pu.color_value("{:#x}".format(self.attached_threads)) 292 | txt += "\n{:16} = ".format("system_mem") 293 | txt += pu.color_value("{:#x}".format(self.system_mem)) 294 | txt += "\n{:16} = ".format("max_system_mem") 295 | txt += pu.color_value("{:#x}".format(self.max_system_mem)) 296 | return txt 297 | 298 | def fastbins_to_string(self, show_status=False, verbose=2, use_cache=False): 299 | """Pretty printer for the malloc_state.fastbinsY[] array supporting different level of verbosity 300 | 301 | :param verbose: 0 for non-verbose. 1 for more verbose. 2 for even more verbose. 302 | :param use_cache: True if we want to use the cached information from the cache object. 303 | False if we want to fetch the data again 304 | """ 305 | 306 | self.ptm.cache.update_fast_bins(show_status=show_status, use_cache=use_cache) 307 | 308 | txt = "" 309 | if verbose == 0: 310 | txt += "\n{:16} = ".format("fastbinsY") 311 | txt += pu.color_value("{}".format("{...}")) 312 | return txt 313 | elif verbose == 1: 314 | show_empty = False 315 | elif verbose >= 2: 316 | show_empty = True 317 | else: 318 | raise Exception("Wrong verbosity passed to fastbins_to_string()") 319 | 320 | for i in range(len(self.fastbinsY)): 321 | count = len(self.ptm.cache.fast_bins[i]) 322 | if show_empty or count > 0: 323 | txt += "\n{:16} = ".format("fastbinsY[%d]" % i) 324 | txt += pu.color_value("{:#x}".format(self.fastbinsY[i])) 325 | txt += " (sz {:#x})".format(self.ptm.fast_bin_size(i)) 326 | msg = "entry" 327 | if count > 1: 328 | msg = "entries" 329 | if count == 0: 330 | txt += " [EMPTY]" 331 | else: 332 | txt += " [{:#d} {}]".format(count, msg) 333 | return txt 334 | 335 | def bins_to_string(self, show_status=False, verbose=2, use_cache=False): 336 | """Pretty printer for the malloc_state.bins[] array supporting different level of verbosity 337 | 338 | :param verbose: 0 for non-verbose. 1 for more verbose. 2 for even more verbose. 339 | :param use_cache: True if we want to use the cached information from the cache object. 340 | False if we want to fetch the data again 341 | """ 342 | 343 | self.ptm.cache.update_bins(show_status=show_status, use_cache=use_cache) 344 | mstate = self.ptm.cache.mstate 345 | 346 | txt = "" 347 | if verbose == 0: 348 | txt += "\n{:16} = ".format("bins") 349 | txt += pu.color_value("{}".format("{...}")) 350 | return txt 351 | elif verbose == 1: 352 | show_empty = False 353 | elif verbose >= 2: 354 | show_empty = True 355 | else: 356 | raise Exception("Wrong verbosity passed to bins_to_string()") 357 | 358 | for i in range(len(self.ptm.cache.bins)): 359 | count = len(self.ptm.cache.bins[i]) 360 | if show_empty or count > 0: 361 | txt += "\n{:16} = ".format("bins[%d]" % i) 362 | txt += pu.color_value("{:#x}, {:#x}".format(mstate.bins[i*2], mstate.bins[i*2+1])) 363 | size = self.ptm.bin_size(i) 364 | if i == self.ptm.bin_index_unsorted: 365 | txt += " (unsorted)" 366 | elif i <= self.ptm.bin_index_small_max: 367 | txt += " (small sz 0x%x)" % size 368 | elif i <= self.ptm.bin_index_large_max: 369 | txt += " (large sz 0x%x)" % size 370 | elif i == self.ptm.bin_index_uncategorized: 371 | # size == None 372 | txt += " (large uncategorized)" 373 | 374 | msg = "entry" 375 | if count > 1: 376 | msg = "entries" 377 | if count == 0: 378 | txt += " [EMPTY]" 379 | else: 380 | txt += " [{:#d} {}]".format(count, msg) 381 | return txt -------------------------------------------------------------------------------- /libptmalloc/ptmalloc/tcache_perthread.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import struct 3 | import sys 4 | import logging 5 | 6 | from libptmalloc.frontend import printutils as pu 7 | from libptmalloc.ptmalloc import heap_structure as hs 8 | 9 | log = logging.getLogger("libptmalloc") 10 | log.trace("tcache_perthread.py") 11 | 12 | class tcache_perthread(hs.heap_structure): 13 | """"python representation of a struct tcache_perthread_struct. 14 | Note: tcache was added in glibc 2.26""" 15 | 16 | # XXX - we can probably get the version directly from the ptm argument? 17 | def __init__(self, ptm, addr=None, mem=None, debugger=None, version=None): 18 | """ 19 | Parse tcache_perthread_struct's data and initialize the tcache_perthread object 20 | 21 | :param ptm: ptmalloc object 22 | :param addr: address for a tcache_perthread_struct where to read the structure's content from the debugger 23 | :param mem: alternatively to "addr", provides the memory bytes of that tcache_perthread_struct's content 24 | :param debugger: the pydbg object 25 | :param version: the glibc version 26 | """ 27 | 28 | super(tcache_perthread, self).__init__(ptm, debugger=debugger) 29 | 30 | self.size = 0 # sizeof(struct tcache_perthread_struct) 31 | 32 | self.counts = [] 33 | self.entries = [] 34 | 35 | if addr is None: 36 | if mem is None: 37 | pu.print_error("Please specify a struct tcache_perthread address") 38 | self.initOK = False 39 | return 40 | 41 | self.address = None 42 | else: 43 | self.address = addr 44 | 45 | if debugger is not None: 46 | self.dbg = debugger 47 | else: 48 | pu.print_error("Please specify a debugger") 49 | raise Exception("sys.exit()") 50 | 51 | if version is None: 52 | pu.print_error("Please specify a glibc version for tcache_perthread") 53 | raise Exception("sys.exit()") 54 | else: 55 | self.version = version 56 | 57 | if version <= 2.25: 58 | pu.print_error("tcache was added in glibc 2.26. Wrong version configured?") 59 | raise Exception("sys.exit()") 60 | if not self.ptm.is_tcache_enabled(): 61 | pu.print_error("tcache is configured as disabled. Wrong configuration?") 62 | raise Exception("sys.exit()") 63 | 64 | self.initialize_sizes_and_offsets() 65 | 66 | if mem is None: 67 | # a string of raw memory was not provided, let's read it from the debugger 68 | try: 69 | self.mem = self.dbg.read_memory(addr, self.size) 70 | except TypeError: 71 | pu.print_error("Invalid address specified") 72 | self.initOK = False 73 | return 74 | except RuntimeError: 75 | pu.print_error("Could not read address {0:#x}".format(addr)) 76 | self.initOK = False 77 | return 78 | else: 79 | if len(mem) < self.size: 80 | pu.print_error("Provided memory size is too small for a tcache_perthread") 81 | self.initOK = False 82 | return 83 | self.mem = mem[:self.size] 84 | 85 | self.unpack_memory() 86 | 87 | def initialize_sizes_and_offsets(self): 88 | """Initialize tcache_perthread_struct's specific sizes based on the glibc version and architecture 89 | """ 90 | 91 | self.size_sz = self.dbg.get_size_sz() 92 | 93 | if self.size_sz == 4: 94 | # sizeof(tcache_perthread_struct) = 64+64*4 95 | self.size = 0x140 96 | elif self.size_sz == 8: 97 | # sizeof(tcache_perthread_struct) = 64+64*8 98 | self.size = 0x240 99 | 100 | log.debug(f"tcache_perthread_struct.size = {self.size:#x}") 101 | 102 | def unpack_memory(self): 103 | """Actually parse all the tcache_perthread_struct's fields from the memory bytes (previously retrieved) 104 | """ 105 | 106 | if self.mem is None: 107 | pu.print_error("No memory found") 108 | raise Exception("sys.exit()") 109 | 110 | self.counts = struct.unpack_from("64B", self.mem, 0) 111 | offset = 64 112 | 113 | if self.size_sz == 4: 114 | fmt = "<64I" 115 | elif self.size_sz == 8: 116 | fmt = "<64Q" 117 | self.entries = struct.unpack_from(fmt, self.mem, offset) 118 | offset = offset + 64 * self.size_sz 119 | 120 | # XXX - fixme 121 | def write(self, inferior=None): 122 | """Write tcache_perthread_struct's data into memory using debugger 123 | """ 124 | pu.print_error("tcache_perthread write() not yet implemented.") 125 | 126 | def __str__(self): 127 | """Pretty printer for the tcache_perthread_struct 128 | """ 129 | return self.to_string() 130 | 131 | def to_string(self, verbose=False): 132 | """Pretty printer for the tcache_perthread_struct supporting different level of verbosity 133 | 134 | :param verbose: False for non-verbose. True for more verbose 135 | """ 136 | 137 | title = "struct tcache_perthread_struct @ 0x%x {" % self.address 138 | txt = pu.color_title(title) 139 | for i in range(len(self.counts)): 140 | if verbose or self.counts[i] > 0: 141 | curr_size = self.ptm.tcache_bin_size(i) 142 | txt += "\n{:11} = ".format("counts[%d]" % i) 143 | txt += pu.color_value("{:#d}".format(self.counts[i])) 144 | txt += " (sz {:#x})".format(curr_size) 145 | for i in range(len(self.entries)): 146 | if verbose or self.entries[i] != 0: 147 | curr_size = self.ptm.tcache_bin_size(i) 148 | txt += "\n{:11} = ".format("entries[%d]" % i) 149 | txt += pu.color_value("{:#x}".format(self.entries[i])) 150 | txt += " (sz {:#x})".format(curr_size) 151 | return txt 152 | 153 | def to_summary_string(self, verbose=False): 154 | """Pretty printer for the tcache_perthread_struct supporting different level of verbosity 155 | with a simplified output. We don't show the tcache_perthread_struct.count values 156 | but instead print them in front of their associated tcache_perthread_struct.entries[] 157 | 158 | :param verbose: False for non-verbose. True for more verbose 159 | """ 160 | 161 | title = "struct tcache_perthread_struct @ 0x%x {" % self.address 162 | txt = pu.color_title(title) 163 | #txt += "\n{:11} = {}".format("counts[]", "{...}") 164 | for i in range(len(self.entries)): 165 | if verbose or self.entries[i] != 0: 166 | curr_size = self.ptm.tcache_bin_size(i) 167 | txt += "\n{:11} = ".format("entries[%d]" % i) 168 | txt += pu.color_value("{:#x}".format(self.entries[i])) 169 | txt += " (sz {:#x})".format(curr_size) 170 | msg = "entry" 171 | if self.counts[i] > 1: 172 | msg = "entries" 173 | if self.counts[i] == 0: 174 | txt += " [EMPTY]" 175 | else: 176 | txt += " [{:#d} {}]".format(self.counts[i], msg) 177 | return txt 178 | -------------------------------------------------------------------------------- /libptmalloc/pydbg/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nccgroup/libptmalloc/e9011393db1ea79b769dcf5f52bd1170a367b304/libptmalloc/pydbg/__init__.py -------------------------------------------------------------------------------- /libptmalloc/pydbg/debugger.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | from libptmalloc.frontend import helpers as h 3 | 4 | class pydbg: 5 | """Python abstraction interface that allows calling into any specific debugger APIs 6 | 7 | Any debugger implementation should implement the methods called on self.debugger 8 | """ 9 | 10 | def __init__(self, debugger): 11 | """Initialize the debugger to be used for any future API 12 | """ 13 | self.debugger = debugger 14 | 15 | def execute(self, cmd, to_string=True): 16 | """Execute a command in the debugger CLI 17 | """ 18 | return self.debugger.execute(cmd, to_string=to_string) 19 | 20 | def format_address(self, value): 21 | """XXX""" 22 | return self.debugger.format_address(value) 23 | 24 | def get_heap_address(self, par=None): 25 | """XXX""" 26 | return self.debugger.get_heap_address(par) 27 | 28 | def get_size_sz(self): 29 | """Retrieve the size_t size for the current architecture 30 | """ 31 | return self.debugger.get_size_sz() 32 | 33 | def read_memory(self, address, length): 34 | """Read bytes at the given address of the given length 35 | """ 36 | return self.debugger.read_memory(address, length) 37 | 38 | def parse_variable(self, variable): 39 | """Parse and evaluate a debugger variable expression 40 | """ 41 | return self.debugger.parse_variable(variable) 42 | 43 | def read_variable(self, variable): 44 | """Read the value stored at the variable name or address""" 45 | return self.debugger.read_variable(variable) 46 | 47 | def read_variable_address(self, variable): 48 | """Gets the variable name's address""" 49 | return self.debugger.read_variable_address(variable) 50 | 51 | def string_to_argv(self, arg): 52 | """XXX""" 53 | return self.debugger.string_to_argv(arg) 54 | 55 | def write_memory(self, address, buf, length=None): 56 | """Write bytes from buf at the given address in memory 57 | """ 58 | return self.debugger.write_memory(address, buf, length) 59 | 60 | def search_chunk(self, ptm, p, search_value, search_type, depth=0, skip=False): 61 | """Searches a chunk for a specific value of a given type 62 | Includes the chunk header in the search by default 63 | 64 | :param ptm: ptmalloc object 65 | :param p: malloc_chunk object representing the chunk 66 | :param search_value: string representing what to search for 67 | :param search_type: "byte", "word", "dword", "qword" or "string" 68 | :param depth: How far into each chunk to search, starting from chunk header address 69 | :param skip: True if don't include chunk header contents in search results 70 | :return: True if the value was found, False otherwise 71 | 72 | Note: this method is generic and does not need a debugger-specific implementation 73 | """ 74 | if depth == 0 or depth > ptm.chunksize(p): 75 | depth = ptm.chunksize(p) 76 | 77 | start_address = p.address 78 | if skip: 79 | start_address += p.hdr_size 80 | try: 81 | result = self.search( 82 | start_address, p.address + depth, search_value, search_type=search_type 83 | ) 84 | return result 85 | except Exception: 86 | print("WARNING: search failed") 87 | h.show_last_exception() 88 | return False 89 | 90 | def search( 91 | self, start_address, end_address, search_value, search_type="string" 92 | ): 93 | """Find a value within some address range 94 | 95 | :param start_address: where to start searching 96 | :param end_address: where to end searching 97 | :param search_value: string representing what to search for 98 | :param search_type: "byte", "word", "dword", "qword" or "string" 99 | :return: True if the value was found, False otherwise 100 | """ 101 | return self.debugger.search( 102 | start_address, end_address, search_value, search_type=search_type 103 | ) 104 | 105 | def parse_address(self, addresses): 106 | """Parse one or more addresses or debugger variables 107 | 108 | :param address: an address string containing hex, int, or debugger variable 109 | :return: the resolved addresses as integers 110 | 111 | It this should be able to handle: hex, decimal, program variables 112 | with or without special characters (like $, &, etc.), 113 | basic addition and subtraction of variables, etc. 114 | """ 115 | return self.debugger.parse_address(addresses) 116 | 117 | def get_backtrace(self): 118 | """Get the current backtrace returned in a dictionary such as: 119 | 120 | { 121 | "raw": "...raw backtrace retured by the debugger" 122 | "funcs": ["list", "of", "functions"] 123 | } 124 | """ 125 | return self.debugger.get_backtrace() 126 | 127 | def get_libc_version(self): 128 | """Retrieve the glibc version if possible as a float (e.g. 2.27) or None if unknown 129 | """ 130 | return self.debugger.get_libc_version() 131 | 132 | 133 | def print_hexdump_chunk(self, ptm, p, maxlen=0, off=0, debug=False, unit=8, verbose=1): 134 | """Hexdump chunk data to stdout 135 | 136 | :param ptm: ptmalloc object 137 | :param p: malloc_chunk() object representing the chunk 138 | :param maxlen: maximum amount of bytes to hexdump 139 | :param off: offset into the chunk's data to hexdump (after the malloc_chunk header) 140 | :param debug: debug enabled or not 141 | :param unit: hexdump unit (e.g. 1, 2, 4, 8, "dps") 142 | :param verbose: see ptchunk's ArgumentParser definition 143 | """ 144 | 145 | address = p.address + p.hdr_size + off 146 | size = ptm.chunksize(p) - p.hdr_size - off 147 | if size <= 0: 148 | if p.inuse: 149 | print("[!] Chunk corrupt? Bad size") 150 | return 151 | else: 152 | if debug: 153 | print("") 154 | return 155 | # ptmalloc can optimize chunks contents and sizes since the a prev_size field is not used 156 | # when the previous chunk is allocated, so ptmalloc can use the extra 8 bytes in 64-bit 157 | # or 4 bytes in 32-bit to hold user content. We allow to show it with -vv 158 | real_size = size 159 | if verbose >= 2: 160 | real_size += self.get_size_sz() 161 | if real_size > size: 162 | print("0x%x+0x%x bytes of chunk data:" % (size, real_size-size)) 163 | else: 164 | print("0x%x bytes of chunk data:" % real_size) 165 | shown_size = real_size 166 | if maxlen != 0: 167 | if shown_size > maxlen: 168 | shown_size = maxlen 169 | 170 | self.print_hexdump(address, shown_size, unit=unit) 171 | if verbose >= 2 and shown_size > size: 172 | print("INFO: the following chunk prev_size field is shown above as could contain user data (ptmalloc optimization)") 173 | 174 | def print_hexdump(self, address, size, unit=8): 175 | """Hexdump data to stdout 176 | 177 | :param address: starting address 178 | :param size: number of bytes from the address 179 | :param unit: hexdump unit (e.g. 1, 2, 4, 8, "dps") 180 | """ 181 | 182 | self.debugger.print_hexdump(address, size, unit=unit) 183 | 184 | def is_tcache_available(self): 185 | """Check if tcache is available by looking up global "tcache" symbol. 186 | 187 | Sometimes, even though glibc has tcache enabled, "tcache" symbol is not available and it seems 188 | tcache is not used because no additional thread is created (?) 189 | """ 190 | return self.debugger.is_tcache_available() 191 | -------------------------------------------------------------------------------- /libptmalloc/pyptmalloc.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import os 3 | import sys 4 | 5 | try: 6 | import gdb 7 | except ImportError: 8 | print("Not running inside of GDB, exiting...") 9 | sys.exit() 10 | 11 | try: 12 | import configparser # py3 13 | except: 14 | import ConfigParser as configparser # py2 15 | 16 | from libptmalloc.frontend import frontend_gdb as fg 17 | from libptmalloc.ptmalloc import ptmalloc as pt 18 | from libptmalloc.pydbg import debugger as d 19 | from libptmalloc.pydbg import pygdbpython as pgp 20 | from libptmalloc.frontend.commands.gdb import ptconfig as ptconfig 21 | 22 | class pyptmalloc: 23 | """Entry point of libptmalloc""" 24 | 25 | def __init__(self): 26 | 27 | # Setup GDB debugger interface 28 | debugger = pgp.pygdbpython() 29 | self.dbg = d.pydbg(debugger) 30 | 31 | config = configparser.SafeConfigParser() 32 | path = os.path.abspath(os.path.dirname(__file__)) 33 | config.read(os.path.join(path, "libptmalloc.cfg")) 34 | 35 | # Try to automatically figure out glibc version and configuration 36 | glibc_version = self.dbg.get_libc_version() 37 | if glibc_version is not None: 38 | self.ptm = pt.ptmalloc(debugger=self.dbg, version=glibc_version) 39 | if glibc_version >= 2.26: 40 | # We assume tcache is enabled, and build a malloc_par() 41 | # object to check if it seems valid 42 | self.ptm.cache.update_param(invalid_ok=True) 43 | par = self.ptm.cache.par 44 | # XXX - we could check other tcache_* fields of malloc_par() if needed 45 | if par.tcache_bins != self.ptm.TCACHE_MAX_BINS: 46 | self.ptm.tcache_enabled = False 47 | else: 48 | self.ptm.tcache_enabled = True 49 | else: 50 | self.ptm.tcache_enabled = False 51 | print("Detected glibc configuration automatically") 52 | 53 | else: 54 | # Roll back to user config file 55 | glibc_version = config.getfloat("Glibc", "version") 56 | try: 57 | tcache_enabled = config.getboolean("Glibc", "tcache") 58 | except configparser.NoOptionError: 59 | if glibc_version >= 2.26: 60 | tcache_enabled = True 61 | else: 62 | tcache_enabled = False 63 | 64 | if tcache_enabled is True and glibc_version < 2.26: 65 | print("ERROR: configuration file sets tcache enabled but glibc < 2.26 didn't support tcache!") 66 | raise Exception("sys.exit()") 67 | print("Read glibc configuration from config file") 68 | 69 | self.ptm = pt.ptmalloc(debugger=self.dbg, version=glibc_version, tcache_enabled=tcache_enabled) 70 | 71 | try: 72 | ptconfig.ptconfig.set_distribution(self.ptm, config.get("OperatingSystem", "distribution")) 73 | except (configparser.NoOptionError, configparser.NoSectionError): 74 | pass 75 | try: 76 | ptconfig.ptconfig.set_release(self.ptm, config.get("OperatingSystem", "release")) 77 | except (configparser.NoOptionError, configparser.NoSectionError): 78 | pass 79 | 80 | # Register GDB commands 81 | fg.frontend_gdb(self.ptm) 82 | -------------------------------------------------------------------------------- /pyptmalloc-dev.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import os 3 | import sys 4 | 5 | # Add the root path 6 | module_path = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) 7 | if module_path not in sys.path: 8 | #print("DEBUG: adding module path...") 9 | sys.path.insert(0, module_path) 10 | #print(sys.path) # DEBUG 11 | 12 | # We need that after the above so it finds it 13 | from libptmalloc import * 14 | -------------------------------------------------------------------------------- /reload.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | usage() 4 | { 5 | echo "Usage:" 6 | echo "./reload.sh [--disable] [--enable] [--help]" 7 | exit 8 | } 9 | 10 | ENABLE="NO" 11 | DISABLE="NO" 12 | while [ $# -gt 0 ] 13 | do 14 | key="$1" 15 | 16 | case $key in 17 | -e|--enable) 18 | ENABLE="YES" 19 | ;; 20 | -d|--disable) 21 | DISABLE="YES" 22 | ;; 23 | -h|--help) 24 | usage 25 | ;; 26 | *) 27 | echo "Unknown option" 28 | usage 29 | ;; 30 | esac 31 | shift # past argument or value 32 | done 33 | 34 | if [[ "${DISABLE}" == "YES" ]]; then 35 | echo "disabling reload (release)" 36 | find . -regex ".*\.py" -exec sed -i -e 's/^importlib.reload/#importlib.reload/g' {} \; 37 | exit 38 | fi 39 | 40 | if [[ "${ENABLE}" == "YES" ]]; then 41 | echo "enabling reload (during development)" 42 | find . -regex ".*\.py" -exec sed -i -e 's/^#importlib.reload/importlib.reload/g' {} \; 43 | exit 44 | fi 45 | 46 | echo "WARNING: requires an argument" 47 | usage 48 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | hexdump==3.3 2 | gdb==0.0.1 3 | future-fstrings==1.2.0 4 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | #!/usr/bin/env python 3 | 4 | from distutils.core import setup 5 | 6 | setup( 7 | name='libptmalloc', 8 | packages=['libptmalloc', 'libptmalloc.pydbg', 'libptmalloc.ptmalloc', 9 | 'libptmalloc.frontend', 'libptmalloc.frontend.commands', 10 | 'libptmalloc.frontend.commands.gdb'], 11 | package_data={'libptmalloc': ['libptmalloc.cfg']}, 12 | version='1.0', 13 | description='python library for examining ptmalloc (glibc userland heap)', 14 | author='Aaron Adams and Cedric Halbronn (NCC Group)', 15 | url='https://github.com/nccgroup/libptmalloc', 16 | license='MIT', 17 | keywords='ptmalloc gdb python glibc', 18 | ) 19 | -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | all: testdoc test1 test1b test2 test1-32 test2-32 sizes sizes32 2 | 3 | testdoc: test.c 4 | $(CC) -DTESTDOC -Os -W -Wall -Werror -o build/testdoc test.c -lpthread 5 | 6 | test1: test.c 7 | $(CC) -DTEST1 -Os -W -Wall -Werror -o build/test1 test.c -lpthread 8 | 9 | test1b: test.c 10 | $(CC) -DTEST1B -Os -W -Wall -Werror -o build/test1b test.c 11 | 12 | test2: test.c 13 | $(CC) -DTEST2 -Os -W -Wall -Werror -o build/test2 test.c -lpthread 14 | 15 | test1-32: test.c 16 | $(CC) -m32 -DTEST1 -Os -W -Wall -Werror -o build/test1-32 test.c -lpthread 17 | 18 | test2-32: test.c 19 | $(CC) -m32 -DTEST2 -Os -W -Wall -Werror -o build/test2-32 test.c -lpthread 20 | 21 | sizes: sizes.c 22 | $(CC) -Os -W -Wall -Werror -o build/sizes sizes.c 23 | 24 | sizes32: sizes.c 25 | $(CC) -m32 -Os -W -Wall -Werror -o build/sizes32 sizes.c 26 | 27 | clean: 28 | rm -Rf build/* 29 | -------------------------------------------------------------------------------- /test/debug.gdb: -------------------------------------------------------------------------------- 1 | 2 | set verbose on 3 | set disassembly-flavor intel 4 | set height 0 5 | set pagination off 6 | set debug-file-directory /usr/lib/debug 7 | 8 | # Dump gdb session to a file 9 | # https://stackoverflow.com/questions/1707167/how-to-dump-the-entire-gdb-session-to-a-file-including-commands-i-type-and-thei 10 | #set logging file gdb_session.log 11 | #set logging on 12 | 13 | b main 14 | commands 15 | silent 16 | source ../pyptmalloc-dev.py 17 | # DEBUG: testing with old libheap 18 | #source ../../libheap/dev.py 19 | d 1 20 | c 21 | end 22 | 23 | run 24 | -------------------------------------------------------------------------------- /test/debug.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # e.g. debug.sh build/test1 1337 3 | 4 | gdb -q -x debug.gdb --args ${1} ${2} 5 | -------------------------------------------------------------------------------- /test/doc.gdb: -------------------------------------------------------------------------------- 1 | 2 | set verbose on 3 | set disassembly-flavor intel 4 | set height 0 5 | set pagination off 6 | set debug-file-directory /usr/lib/debug 7 | 8 | # Disable ASLR so we have deterministic documentation 9 | set disable-randomization 10 | 11 | b main 12 | commands 13 | silent 14 | source ../pyptmalloc-dev.py 15 | d 1 16 | c 17 | end 18 | 19 | run 20 | 21 | # This will be executed when it hits the "int3" of our "build/test1" binary 22 | source doc2.gdb -------------------------------------------------------------------------------- /test/doc.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_fstrings -*- 2 | import gdb 3 | 4 | debug = False 5 | def debug_print(s, end=None): 6 | if debug: 7 | print(s, end=end) 8 | 9 | def print_title(title, level=1): 10 | print("#"*level + " " + title) 11 | print("") 12 | 13 | def execute_and_log(command, max_lines=-1): 14 | print("```") 15 | print(f"(gdb) {command}") 16 | output = gdb.execute(command, from_tty=True, to_string=True) 17 | if max_lines > 0: 18 | output = "\n".join(output.split("\n")[:max_lines]) + "\n" 19 | print(output, end="") 20 | if max_lines > 0: 21 | print("...") 22 | print("```") 23 | print("") 24 | 25 | print_title("Usage", 1) 26 | 27 | print_title("libptmalloc commands", 2) 28 | print("""The `pthelp` command lists all the commands provided by libptmalloc: 29 | """) 30 | execute_and_log("pthelp") 31 | 32 | print_title("Commands' usage", 2) 33 | print("""Each command has detailed usage that you can print using `-h`: 34 | """) 35 | execute_and_log("ptfree -h") 36 | 37 | print_title("Common usage and example", 1) 38 | 39 | print_title("ptconfig", 2) 40 | print("""The first thing to make sure when using libptmalloc is to have the 41 | right glibc version configured in `libptmalloc`. 42 | 43 | Note we could automatically detect the ptmalloc version (hence glibc) by pattern 44 | matching on the ptmalloc structures but it is not implemented in libptmalloc yet. 45 | 46 | The configured glibc version can be defined in the `libptmalloc.cfg` file: 47 | """) 48 | 49 | print("```") 50 | print("".join(open("../libptmalloc/libptmalloc.cfg", "r").readlines()[:3])) 51 | print("""``` 52 | """) 53 | 54 | print("""It will then reflect using the `ptconfig` command: 55 | """) 56 | 57 | execute_and_log("ptconfig") 58 | 59 | print("""You can also change it: 60 | """) 61 | 62 | execute_and_log("ptconfig -v 2.27") 63 | 64 | 65 | print_title("ptarena", 2) 66 | print("""We list all the arenas: 67 | """) 68 | execute_and_log("ptarena -l") 69 | 70 | print("""We show the arena fields: 71 | """) 72 | execute_and_log("ptarena") 73 | 74 | print("""We show more fields: 75 | """) 76 | execute_and_log("ptarena -v") 77 | 78 | print("""We show the 2nd arena by specifying its address: 79 | """) 80 | execute_and_log("ptarena 0x7ffff0000020 -v") 81 | 82 | print_title("ptlist", 2) 83 | print("""We list all the chunks linearly in an arena. 84 | By default it prints one line per chunk: 85 | """) 86 | execute_and_log("ptlist 0x7ffff0000020") 87 | 88 | print("""Note: The `ptlist` commands support a lot of features from 89 | the `ptchunk` command. 90 | """) 91 | 92 | print_title("ptchunk", 2) 93 | 94 | print_title("Allocated chunk", 3) 95 | print("""We print one allocated chunk: 96 | """) 97 | execute_and_log("ptchunk 0x5555557998e0") 98 | print("""We print the same allocated chunk with its header and data: 99 | """) 100 | execute_and_log("ptchunk 0x5555557998e0 -v -x") 101 | 102 | print_title("Free chunk in regular bin", 3) 103 | print("""We print one free chunk: 104 | """) 105 | execute_and_log("ptchunk 0x555555799ab0") 106 | print("""We print the same free chunk with its header and data: 107 | """) 108 | execute_and_log("ptchunk 0x555555799ab0 -v -x") 109 | 110 | print_title("Printing multiple chunks", 3) 111 | print("""We print multiple chunks. You can limit the number of chunks being printed: 112 | """) 113 | execute_and_log("ptchunk 0x5555557998e0 -c 5") 114 | print("""We differentiate chunks that are allocated `M`, freed in an 115 | unsorted/small/large bin `F`, freed in the fast bin `f` or freed in the tcache bin `t`. 116 | """) 117 | 118 | print_title("Combining options", 3) 119 | print("""By combininig all options: 120 | """) 121 | execute_and_log("ptchunk 0x5555557998e0 -c 3 -v -x") 122 | 123 | # Switch back to the main arena to get more output in the doc 124 | gdb.execute("ptarena 0x7ffff7baec40", from_tty=True, to_string=True) 125 | 126 | print_title("ptbin", 2) 127 | print("""We print all the unsorted/small/large bins. By default it won't print 128 | the empty bins: 129 | """) 130 | execute_and_log("ptbin") 131 | print("""We print all the bins: 132 | """) 133 | execute_and_log("ptbin -v", max_lines=11) 134 | print("""We print all the chunks in a particular bin: 135 | """) 136 | execute_and_log("ptbin -i 8") 137 | 138 | print_title("ptfast", 2) 139 | print("""We print all the fast bins. By default it won't print 140 | the empty bins: 141 | """) 142 | execute_and_log("ptfast") 143 | # print("""We print all the bins: 144 | # """) 145 | # execute_and_log("ptfast -v", max_lines=8) 146 | print("""We print all the chunks in a particular bin. Note how we limit the number of chunks shown: 147 | """) 148 | execute_and_log("ptfast -i 5 -c 3") 149 | 150 | print_title("pttcache", 2) 151 | print("""We print all the tcache bins. By default it won't print 152 | the empty bins: 153 | """) 154 | execute_and_log("pttcache", max_lines=13) 155 | print("""We print all the chunks in a particular bin: 156 | """) 157 | execute_and_log("pttcache -i 7") 158 | 159 | print_title("ptfree", 2) 160 | print("""It prints all the bins by combining the output of `ptbin`, `ptfast` and 161 | `pttcache`. It is quite verbose so we won't include an example here. 162 | """) 163 | 164 | print_title("ptstats", 2) 165 | print("""We print memory statistics for all the arenas: 166 | """) 167 | execute_and_log("ptstats") 168 | 169 | print_title("ptmeta", 2) 170 | 171 | print("""We first notice this chunk holds the libgcc path: 172 | """) 173 | 174 | execute_and_log("ptchunk 0x7ffff0001400 -v -x") 175 | 176 | print("""The 'ptmeta command is more advanced and allows to associate user-defined metadata 177 | for given chunks' addresses. E.g. you can add a tag as metadata: 178 | """) 179 | execute_and_log("ptmeta add 0x7ffff0001400 tag \"libgcc path\"") 180 | print("""Then it can be show within other commands: 181 | """) 182 | execute_and_log("ptchunk 0x7ffff0001400 -M tag") 183 | print("""Note: You can also associate a backtrace as metadata, which allows to 184 | write your own heap tracer tool 185 | """) 186 | 187 | 188 | print_title("Cache", 1) 189 | print("""In order to speed up the execution of commands, libptmalloc caches 190 | the ptmalloc structures as well as the addresses of the chunks in specific bins 191 | when you execute certain commands. 192 | """) 193 | 194 | execute_and_log("ptfast 0x7ffff7baec40") 195 | 196 | print("""That being said, by default, it won't use the cache, to avoid any misleading info: 197 | """) 198 | execute_and_log("ptfast") 199 | print("""If you want to use the cache, when you know nothing has changed since the 200 | last cached information, you can use the following: 201 | """) 202 | execute_and_log("ptfast --use-cache") 203 | 204 | print_title("Advanced usage", 1) 205 | 206 | print_title("Searching chunks", 2) 207 | print("""By default, searching will show all chunks but show a match/no-match suffix. 208 | Because we are limiting the number of chunks, and even the non-match, 209 | we see there is only one match: 210 | """) 211 | execute_and_log("ptlist -s \"GGGG\" -c 9") 212 | 213 | print("""If you only want to show matches, you use the following. Note how the 214 | no-matching chunks are not shown anymore: 215 | """) 216 | execute_and_log("ptlist -s \"GGGG\" -c 2 --match-only") 217 | 218 | print("""Analyzing the content, we see the value was found in the chunks header 219 | in the second chunk: 220 | """) 221 | execute_and_log("ptlist -s \"GGGG\" -c 2 --match-only -v -x") 222 | 223 | print("""To ignore the chunks headers, we use the following. We see a different 224 | second chunk is shown: 225 | """) 226 | execute_and_log("ptlist -s \"GGGG\" -c 2 --match-only -v -x --skip") 227 | 228 | print_title("Printing chunks of specific type(s)", 2) 229 | 230 | print("""We print chunks linearly, limiting to 10 chunks, and highlighting tcache free chunks 231 | and regular bin free chunks: 232 | """) 233 | execute_and_log("ptlist -c 10 -I \"t,F\"") 234 | 235 | print("""We filter to only show the highlighted chunks, resulting in skipping other types of chunks: 236 | """) 237 | execute_and_log("ptlist -c 10 -I \"t,F\" --highlight-only") 238 | 239 | print_title("Detailed commands' usage", 1) 240 | print("""We list all the commands' complete usage as a reference. 241 | """) 242 | 243 | print_title("ptconfig usage", 2) 244 | execute_and_log("ptconfig -h") 245 | print_title("ptmeta usage", 2) 246 | execute_and_log("ptmeta -h") 247 | execute_and_log("ptmeta add -h") 248 | execute_and_log("ptmeta del -h") 249 | execute_and_log("ptmeta list -h") 250 | execute_and_log("ptmeta config -h") 251 | print_title("ptarena usage", 2) 252 | execute_and_log("ptarena -h") 253 | print_title("ptparam usage", 2) 254 | execute_and_log("ptparam -h") 255 | print_title("ptlist usage", 2) 256 | execute_and_log("ptlist -h") 257 | print_title("ptchunk usage", 2) 258 | execute_and_log("ptchunk -h") 259 | print_title("ptbin usage", 2) 260 | execute_and_log("ptbin -h") 261 | print_title("ptfast usage", 2) 262 | execute_and_log("ptfast -h") 263 | print_title("pttcache usage", 2) 264 | execute_and_log("pttcache -h") 265 | print_title("ptfree usage", 2) 266 | execute_and_log("ptfree -h") 267 | print_title("ptstats usage", 2) 268 | execute_and_log("ptstats -h") 269 | 270 | 271 | print_title("Comparison with other tools", 1) 272 | 273 | print_title("libheap", 2) 274 | print("""libptmalloc is heavily based on other tools like 275 | [libheap](https://github.com/cloudburst/libheap) even though a lot has been 276 | changed or added. 277 | 278 | The following table shows differences: 279 | 280 | | libheap | libptmalloc | Note | 281 | |------------------|------------------|------| 282 | | print_bin_layout | ptbin -i | print_bin_layout only includes small bins. ptbin also includes unsorted and large bins | 283 | | heapls | ptlist | | 284 | | heaplsc | ptlist --compact | | 285 | | mstats | ptstats | | 286 | | smallbins | ptbin | ptbin also includes unsorted and large bins | 287 | | fastbins | ptfast | | 288 | | N/A | pttcache | | 289 | | freebin | ptfree | ptfree also includes tcache bins | 290 | """) 291 | 292 | print_title("Notes", 1) 293 | print("""This documentation is automatically generated by [doc.sh](../test/doc.sh). 294 | This also allows people to replicate the commands manually into a debugger 295 | """) -------------------------------------------------------------------------------- /test/doc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Allows to generate docs/UserGuide.md 3 | 4 | gdb -q -x doc.gdb --args build/testdoc 1 -------------------------------------------------------------------------------- /test/doc2.gdb: -------------------------------------------------------------------------------- 1 | # NOTE: we can't just do that from doc.py as it won't be taken into account into gdb? 2 | 3 | printf "Logging into a file\n" 4 | # Dump gdb session to a file 5 | # https://stackoverflow.com/questions/1707167/how-to-dump-the-entire-gdb-session-to-a-file-including-commands-i-type-and-thei 6 | set logging file ../docs/UserGuide.md 7 | set logging overwrite on 8 | set logging on 9 | 10 | source doc.py 11 | 12 | set logging off -------------------------------------------------------------------------------- /test/generate_bins.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # Helper to generate functions in libptmalloc code to convert an index in a bin 3 | # (tcache, fast, regular bin) into the corresponding chunk size (and vice versa) 4 | # It can also parses the output of logs generated by test.sh to see what chunks 5 | # get allocated in what bin, for that same purpose. 6 | 7 | import re 8 | import sys 9 | 10 | logfile = "test64.log" 11 | #logfile = "test32.log" 12 | #logfile = "test_large64.log" 13 | #logfile = "test_large32.log" 14 | 15 | debug = False 16 | def debug_print(s, end=None): 17 | if debug: 18 | print(s, end=end) 19 | 20 | # init them so they are ordered when printing 21 | tcache_bins = {} 22 | for i in range(64): 23 | tcache_bins[i] = -1 24 | fast_bins = {} 25 | for i in range(10): 26 | fast_bins[i] = -1 27 | bins = {} 28 | for i in range(127): 29 | bins[i] = -1 30 | 31 | def parse_logfile(): 32 | """Parse a logfile generated by test.sh to build the bin dictionary above""" 33 | unset = True 34 | f = open(logfile, "r") 35 | for line in f: 36 | debug_print(f"line = '{line[:-1]}'") 37 | if line.startswith("---"): 38 | unset = True 39 | m = re.match("tcache bin ([0-9]+) \(.*", line) 40 | if m: 41 | if unset is False: 42 | print("error parsing") 43 | sys.exit(1) 44 | curr_bin = tcache_bins 45 | curr_index = int(m.group(1)) 46 | unset = False 47 | continue 48 | m = re.match("fast bin ([0-9]+) \(.*", line) 49 | if m: 50 | if unset is False: 51 | print("error parsing") 52 | sys.exit(1) 53 | curr_bin = fast_bins 54 | curr_index = int(m.group(1)) 55 | unset = False 56 | continue 57 | m = re.match("(large|small|unsorted|large uncategorized) bin ([0-9]+) \(.*", line) 58 | if m: 59 | if unset is False: 60 | print("error parsing") 61 | sys.exit(1) 62 | curr_bin = bins 63 | curr_index = int(m.group(2)) 64 | unset = False 65 | continue 66 | m = re.match("[0-9a-fx]+ .* sz:([0-9a-fx]+) .*", line) 67 | if m: 68 | if unset is True: 69 | print("error parsing") 70 | sys.exit(1) 71 | size = int(m.group(1), 16) 72 | if curr_index not in curr_bin.keys(): 73 | curr_bin[curr_index] = size 74 | elif size > curr_bin[curr_index]: 75 | curr_bin[curr_index] = size 76 | f.close() 77 | 78 | def dump_all(): 79 | """Dump all bins parsed with parse_logfile()""" 80 | print("tcache bins:") 81 | for k,v in tcache_bins.items(): 82 | print(f"{k} -> {v:#x}") 83 | print("fast bins:") 84 | for k,v in fast_bins.items(): 85 | print(f"{k} -> {v:#x}") 86 | print("regular bins:") 87 | for k,v in bins.items(): 88 | print(f"{k} -> {v:#x}") 89 | 90 | def generate_tcache_64(): 91 | """Generate python code matching the maximum encountered size empirically parsed with parse_logfile""" 92 | print(" def tcache_bin_size_XX(self, idx):") 93 | for k,v in tcache_bins.items(): 94 | generate_if_case(v, k) 95 | print(" def tcache_bin_index_XX(self, size):") 96 | for k,v in tcache_bins.items(): 97 | generate_if_case(v, k) 98 | 99 | # uncomment the one you don't want to use 100 | def generate_if_case(size, idx): 101 | """Helper function to generate python code""" 102 | # for tcache/fast bins 103 | # print(f" elif size == {size:#x}:") 104 | # print(f" return {idx}") 105 | # for regular bins 106 | print(f" elif size <= {size:#x}:") 107 | print(f" return {idx}") 108 | # for tcache/fast/regular bins 109 | # print(f" elif idx == {idx}:") 110 | # print(f" return {size:#x}") 111 | 112 | # See https://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/malloc.c;h=6e766d11bc85b6480fa5c9f2a76559f8acf9deb5;hb=HEAD#l1388 113 | def generate_bin_64(): 114 | """Generate python code for small/large bin: chunk size <-> index (in both directions) for 64-bit""" 115 | print(" def small_bin_index_64(self, size):") 116 | size = 0x10 117 | for k in range(1, 63): 118 | size += 0x10 119 | generate_if_case(size, k) 120 | print(" def large_bin_index_64(self, size):") 121 | for k in range(63, 96): 122 | size += 0x40 # 64 123 | generate_if_case(size, k) 124 | for k in range(96, 97): 125 | size += 0x1c0 126 | generate_if_case(size, k) 127 | for k in range(97, 111): 128 | size += 0x200 # 512 129 | generate_if_case(size, k) 130 | for k in range(111, 112): 131 | size += 0x600 132 | generate_if_case(size, k) 133 | for k in range(112, 119): 134 | size += 0x1000 # 4096 135 | generate_if_case(size, k) 136 | for k in range(119, 120): 137 | size += 0x6000 138 | generate_if_case(size, k) 139 | for k in range(120, 123): 140 | size += 0x8000 # 32768 141 | generate_if_case(size, k) 142 | for k in range(123, 126): 143 | size += 0x40000 # 32768 144 | generate_if_case(size, k) 145 | 146 | # See https://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/malloc.c;h=6e766d11bc85b6480fa5c9f2a76559f8acf9deb5;hb=HEAD#l1388 147 | def generate_bin_32(): 148 | """Generate python code for small/large bin: chunk size <-> index (in both directions) for 32-bit""" 149 | print(" def small_bin_index_32(self, size):") 150 | size = 0x0 151 | for k in range(1, 63): 152 | size += 0x10 153 | generate_if_case(size, k) 154 | print(" def large_bin_index_32(self, size):") 155 | for k in range(63, 64): 156 | size += 0x10 157 | generate_if_case(size, k) 158 | for k in range(64, 96): 159 | size += 0x40 # 64 160 | generate_if_case(size, k) 161 | for k in range(96, 111): 162 | size += 0x200 # 512 163 | generate_if_case(size, k) 164 | for k in range(111, 112): 165 | size += 0x600 166 | generate_if_case(size, k) 167 | for k in range(112, 119): 168 | size += 0x1000 # 4096 169 | generate_if_case(size, k) 170 | for k in range(119, 120): 171 | size += 0x6000 172 | generate_if_case(size, k) 173 | for k in range(120, 123): 174 | size += 0x8000 # 32768 175 | generate_if_case(size, k) 176 | for k in range(123, 126): 177 | size += 0x40000 # 32768 178 | generate_if_case(size, k) 179 | 180 | if False: 181 | parse_logfile() 182 | dump_all() 183 | elif False: 184 | parse_logfile() 185 | generate_tcache_64() 186 | elif False: 187 | generate_bin_32() 188 | elif True: 189 | generate_bin_64() 190 | -------------------------------------------------------------------------------- /test/sizes.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | # define offsetof(type,ident) ((size_t)&(((type*)0)->ident)) 5 | 6 | //glibc-2.27/sysdeps/generic/malloc-alignment.h 7 | #define MALLOC_ALIGNMENT (2 * SIZE_SZ < __alignof__ (long double) \ 8 | ? __alignof__ (long double) : 2 * SIZE_SZ) 9 | //glibc-2.27/sysdeps/i386/malloc-alignment.h 10 | //#define MALLOC_ALIGNMENT 16 11 | 12 | //glibc-2.27/malloc/malloc-internal.h 13 | # define INTERNAL_SIZE_T size_t 14 | #define SIZE_SZ (sizeof (INTERNAL_SIZE_T)) 15 | #define MALLOC_ALIGN_MASK (MALLOC_ALIGNMENT - 1) 16 | 17 | //glibc-2.27/malloc/malloc.c 18 | struct malloc_chunk { 19 | 20 | INTERNAL_SIZE_T mchunk_prev_size; /* Size of previous chunk (if free). */ 21 | INTERNAL_SIZE_T mchunk_size; /* Size in bytes, including overhead. */ 22 | 23 | struct malloc_chunk* fd; /* double links -- used only if free. */ 24 | struct malloc_chunk* bk; 25 | 26 | /* Only used for large blocks: pointer to next larger size. */ 27 | struct malloc_chunk* fd_nextsize; /* double links -- used only if free. */ 28 | struct malloc_chunk* bk_nextsize; 29 | }; 30 | 31 | #define MIN_CHUNK_SIZE (offsetof(struct malloc_chunk, fd_nextsize)) 32 | #define MINSIZE \ 33 | (unsigned long)(((MIN_CHUNK_SIZE+MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK)) 34 | # define tidx2usize(idx) (((size_t) idx) * MALLOC_ALIGNMENT + MINSIZE - SIZE_SZ) 35 | # define TCACHE_MAX_BINS 64 36 | # define MAX_TCACHE_SIZE tidx2usize (TCACHE_MAX_BINS-1) 37 | # define csize2tidx(x) (((x) - MINSIZE + MALLOC_ALIGNMENT - 1) / MALLOC_ALIGNMENT) 38 | 39 | #define NBINS 128 40 | #define NSMALLBINS 64 41 | #define SMALLBIN_WIDTH MALLOC_ALIGNMENT 42 | #define SMALLBIN_CORRECTION (MALLOC_ALIGNMENT > 2 * SIZE_SZ) 43 | #define MIN_LARGE_SIZE ((NSMALLBINS - SMALLBIN_CORRECTION) * SMALLBIN_WIDTH) 44 | 45 | #define smallbin_index(sz) \ 46 | ((SMALLBIN_WIDTH == 16 ? (((unsigned) (sz)) >> 4) : (((unsigned) (sz)) >> 3))\ 47 | + SMALLBIN_CORRECTION) 48 | 49 | #define largebin_index_32(sz) \ 50 | (((((unsigned long) (sz)) >> 6) <= 38) ? 56 + (((unsigned long) (sz)) >> 6) :\ 51 | ((((unsigned long) (sz)) >> 9) <= 20) ? 91 + (((unsigned long) (sz)) >> 9) :\ 52 | ((((unsigned long) (sz)) >> 12) <= 10) ? 110 + (((unsigned long) (sz)) >> 12) :\ 53 | ((((unsigned long) (sz)) >> 15) <= 4) ? 119 + (((unsigned long) (sz)) >> 15) :\ 54 | ((((unsigned long) (sz)) >> 18) <= 2) ? 124 + (((unsigned long) (sz)) >> 18) :\ 55 | 126) 56 | 57 | #define largebin_index_32_big(sz) \ 58 | (((((unsigned long) (sz)) >> 6) <= 45) ? 49 + (((unsigned long) (sz)) >> 6) :\ 59 | ((((unsigned long) (sz)) >> 9) <= 20) ? 91 + (((unsigned long) (sz)) >> 9) :\ 60 | ((((unsigned long) (sz)) >> 12) <= 10) ? 110 + (((unsigned long) (sz)) >> 12) :\ 61 | ((((unsigned long) (sz)) >> 15) <= 4) ? 119 + (((unsigned long) (sz)) >> 15) :\ 62 | ((((unsigned long) (sz)) >> 18) <= 2) ? 124 + (((unsigned long) (sz)) >> 18) :\ 63 | 126) 64 | 65 | // XXX It remains to be seen whether it is good to keep the widths of 66 | // XXX the buckets the same or whether it should be scaled by a factor 67 | // XXX of two as well. 68 | #define largebin_index_64(sz) \ 69 | (((((unsigned long) (sz)) >> 6) <= 48) ? 48 + (((unsigned long) (sz)) >> 6) :\ 70 | ((((unsigned long) (sz)) >> 9) <= 20) ? 91 + (((unsigned long) (sz)) >> 9) :\ 71 | ((((unsigned long) (sz)) >> 12) <= 10) ? 110 + (((unsigned long) (sz)) >> 12) :\ 72 | ((((unsigned long) (sz)) >> 15) <= 4) ? 119 + (((unsigned long) (sz)) >> 15) :\ 73 | ((((unsigned long) (sz)) >> 18) <= 2) ? 124 + (((unsigned long) (sz)) >> 18) :\ 74 | 126) 75 | 76 | #define largebin_index(sz) \ 77 | (SIZE_SZ == 8 ? largebin_index_64 (sz) \ 78 | : MALLOC_ALIGNMENT == 16 ? largebin_index_32_big (sz) \ 79 | : largebin_index_32 (sz)) 80 | 81 | int main() 82 | { 83 | printf("SIZE_SZ = 0x%zx\n", SIZE_SZ); //0x8 on 64-bit, 0x4 on 32-bit 84 | printf("MALLOC_ALIGNMENT = 0x%zx\n", MALLOC_ALIGNMENT); //0x10 on 64-bit, 0x8 on 32-bit 85 | printf("MINSIZE = 0x%lx\n", MINSIZE); //0x20 on 64-bit, 0x10 on 32-bit 86 | #if 0 87 | // tcache stuff 88 | printf("tidx2usize(0) = 0x%lx\n", tidx2usize(0)); 89 | printf("tidx2usize(1) = 0x%lx\n", tidx2usize(1)); 90 | printf("tidx2usize(2) = 0x%lx\n", tidx2usize(2)); 91 | printf("MAX_TCACHE_SIZE = 0x%lx\n", MAX_TCACHE_SIZE); 92 | printf("csize2tidx(0x18) = %ld\n", csize2tidx(0x18)); 93 | printf("csize2tidx(0x20) = %ld\n", csize2tidx(0x20)); 94 | printf("csize2tidx(0x21) = %ld\n", csize2tidx(0x21)); 95 | printf("csize2tidx(0x28) = %ld\n", csize2tidx(0x28)); 96 | printf("csize2tidx(0x30) = %ld\n", csize2tidx(0x30)); 97 | printf("csize2tidx(0x160) = %ld\n", csize2tidx(0x160)); 98 | printf("csize2tidx(0x190) = %ld\n", csize2tidx(0x190)); 99 | #endif 100 | #if 1 101 | printf("MIN_LARGE_SIZE = 0x%zx\n", MIN_LARGE_SIZE); //0x400 on 64-bit, 0x200 on 32-bit 102 | unsigned int last_index = 64; // first index in large bin 103 | unsigned int index; 104 | for (unsigned size = MIN_LARGE_SIZE; size < 0x200000; size+= 0x1) { 105 | index = largebin_index(size); 106 | if (index != last_index) { 107 | //printf("largebin_index(0x%x) -> large bin %d\n", size, index); 108 | //printf("large bin %d: chunk sz <= 0x%zx\n", last_index, 2 * SIZE_SZ + size); 109 | // Generate some code we can use in libptmalloc 110 | printf("elif index == %d: return 0x%zx\n", last_index, 2 * SIZE_SZ + size); 111 | last_index = index; 112 | } 113 | } 114 | #endif 115 | return 0; 116 | } -------------------------------------------------------------------------------- /test/test.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | #define SEED 1337 12 | 13 | #if defined(TESTDOC) 14 | #define MAX_COUNT_ALLOCATIONS 400 15 | #define MAX_COUNT_ALLOCATIONS_FASTBINS 80 16 | #else 17 | #if defined(TEST1) || defined(TEST1B) 18 | #define MAX_COUNT_ALLOCATIONS 4096 19 | #define MAX_COUNT_ALLOCATIONS_FASTBINS 100 20 | #else 21 | #if defined(TEST2) 22 | #define MAX_COUNT_ALLOCATIONS 4096 23 | #define MAX_COUNT_ALLOCATIONS_FASTBINS 1000 24 | #else 25 | #define MAX_COUNT_ALLOCATIONS 10 26 | #define MAX_COUNT_ALLOCATIONS_FASTBINS 10 27 | #endif 28 | #endif 29 | #endif 30 | 31 | char* buffers[MAX_COUNT_ALLOCATIONS]; 32 | unsigned int sizes[MAX_COUNT_ALLOCATIONS]; 33 | static unsigned int idx = 0; // available idx to use for next alloc 34 | 35 | // We track freed sizes so we know what sizes to realloc before we free them again 36 | unsigned int sizes_freed[MAX_COUNT_ALLOCATIONS]; // upper bound size 37 | static unsigned int idx_freed = 0; // available idx to use for next freed 38 | 39 | #if defined(TEST1) || defined(TEST1B) || defined(TESTDOC) 40 | char* buffers2[MAX_COUNT_ALLOCATIONS_FASTBINS]; 41 | static unsigned int idx2 = 0; // available idx to use for next alloc 42 | #endif 43 | 44 | static unsigned int total_cnt_frees = 0; 45 | static unsigned int total_cnt_allocs = 0; 46 | 47 | char letter = 'A'; 48 | 49 | char next_letter() { 50 | if (letter == 'Z') { 51 | letter = 'A'; 52 | } else { 53 | letter += 1; 54 | } 55 | return letter; 56 | } 57 | 58 | void *f1(void *x); 59 | void *f2(void *x); 60 | 61 | /* Returns an integer in the range [0, n]. 62 | * 63 | * Uses rand(), and so is affected-by/affects the same seed. 64 | * Source: https://stackoverflow.com/questions/822323/how-to-generate-a-random-int-in-c 65 | */ 66 | int randint(int n) { 67 | if ((n - 1) == RAND_MAX) { 68 | return rand(); 69 | } else { 70 | // Supporting larger values for n would requires an even more 71 | // elaborate implementation that combines multiple calls to rand() 72 | assert (n <= RAND_MAX); 73 | 74 | // Chop off all of the values that would cause skew... 75 | int end = RAND_MAX / n; // truncate skew 76 | assert (end > 0); 77 | end *= n; 78 | 79 | // ... and ignore results from rand() that fall above that limit. 80 | // (Worst case the loop condition should succeed 50% of the time, 81 | // so we can expect to bail out of this loop pretty quickly.) 82 | int r; 83 | while ((r = rand()) >= end); 84 | return r % n; 85 | } 86 | } 87 | 88 | //https://stackoverflow.com/questions/2509679/how-to-generate-a-random-integer-number-from-within-a-range 89 | unsigned int rand_interval(unsigned int min, unsigned int max) 90 | { 91 | unsigned int r; 92 | const unsigned int range = 1 + max - min; 93 | const unsigned int buckets = RAND_MAX / range; 94 | const unsigned int limit = buckets * range; 95 | 96 | /* Create equal size buckets all in a row, then fire randomly towards 97 | * the buckets until you land in one of them. All buckets are equally 98 | * likely. If you land off the end of the line of buckets, try again. */ 99 | do 100 | { 101 | r = (unsigned int)rand(); 102 | } while (r >= limit); 103 | 104 | return min + (r / buckets); 105 | } 106 | 107 | // When trying to access "tcache" global variable in gdb, we get this error: 108 | // ``` 109 | // Cannot find thread-local storage for process 109057, shared library /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.27.so: 110 | // Cannot find thread-local variables on this target 111 | // ``` 112 | // It seems it is related to the executable not being linked to pthreads. 113 | // So the trick we use is to force using pthreads functions and it does the job lol 114 | // 115 | // This is code from https://unix.stackexchange.com/questions/33396/gcc-cant-link-to-pthread 116 | #ifndef TEST1B 117 | void create_threads() 118 | { 119 | pthread_t f2_thread, f1_thread; 120 | 121 | int i1,i2; 122 | i1 = 1; 123 | i2 = 2; 124 | pthread_create(&f1_thread, NULL, f1, &i1); 125 | pthread_create(&f2_thread, NULL, f2, &i2); 126 | pthread_join(f1_thread, NULL); 127 | pthread_join(f2_thread, NULL); 128 | } 129 | 130 | void *f1(void *x){ 131 | int i; 132 | i = *(int*)x; 133 | //sleep(1); 134 | printf("f1: %d\n", i); 135 | 136 | // for (i = 0; i < 10; i++) { 137 | // test1_alloc(); 138 | // } 139 | pthread_exit(0); 140 | } 141 | 142 | void *f2(void *x){ 143 | int i; 144 | i = *(int*)x; 145 | //sleep(1); 146 | printf("f2: %d\n", i); 147 | pthread_exit(0); 148 | } 149 | #endif 150 | 151 | // several functions so we have a backtrace to save in metadata 152 | void __attribute__((optimize("O0"))) func3() 153 | { 154 | asm("int $3"); // so we can analyse in gdb 155 | } 156 | 157 | void __attribute__((optimize("O0"))) func2() 158 | { 159 | func3(); 160 | } 161 | 162 | void __attribute__((optimize("O0"))) func1() 163 | { 164 | func2(); 165 | } 166 | 167 | 168 | #if defined(TEST1) || defined(TEST1B) || defined(TESTDOC) 169 | void test1_alloc() 170 | { 171 | unsigned int size = 0x0; 172 | switch (randint(4)) 173 | { 174 | case 0: 175 | size = 0x80; // force tcache size 176 | break; 177 | case 1: 178 | size = 0xa8; // force fastbin size on 64-bit (?) 179 | break; 180 | case 2: 181 | size = 0x400; 182 | break; 183 | case 3: 184 | size = 0x2000; 185 | break; 186 | case 4: 187 | size = 0x20000; 188 | break; 189 | } 190 | unsigned int alloc_size = randint(size); 191 | buffers[idx] = malloc(alloc_size); 192 | sizes[idx] = alloc_size; 193 | memset(buffers[idx], next_letter(), alloc_size); 194 | idx++; 195 | printf("+"); 196 | total_cnt_allocs += 1; 197 | } 198 | 199 | void test1_free() 200 | { 201 | unsigned int i; 202 | unsigned int free_idx = randint(idx-1); 203 | free(buffers[free_idx]); 204 | buffers[free_idx] = NULL; 205 | sizes_freed[idx_freed++] = sizes[free_idx]; 206 | // shift the remaining pointers so it is easier 207 | // to track which ones are not freed yet 208 | for (i = free_idx; i <= idx-1; i++) { 209 | buffers[i] = buffers[i+1]; 210 | sizes[i] = sizes[i+1]; 211 | } 212 | idx--; 213 | printf("-"); 214 | total_cnt_frees += 1; 215 | } 216 | 217 | void test1_realloc() 218 | { 219 | unsigned int i; 220 | unsigned int realloc_idx = randint(idx_freed-1); 221 | unsigned int alloc_size = sizes_freed[realloc_idx]; 222 | buffers[idx] = malloc(alloc_size); 223 | memset(buffers[idx], next_letter(), alloc_size); 224 | idx++; 225 | // shift the remaining pointers so it is easier 226 | // to track which ones are not freed yet 227 | for (i = realloc_idx; i <= idx_freed-1; i++) { 228 | sizes_freed[i] = sizes_freed[i+1]; 229 | } 230 | idx_freed--; 231 | printf("*"); 232 | total_cnt_allocs += 1; 233 | } 234 | 235 | void test1_alloc2() 236 | { 237 | // force fastbin allocs 238 | unsigned int alloc_size = randint(0xa0); 239 | buffers2[idx2] = malloc(alloc_size); 240 | memset(buffers2[idx2], next_letter(), alloc_size); 241 | idx2++; 242 | printf("^"); 243 | total_cnt_allocs += 1; 244 | } 245 | 246 | void test1_free2() 247 | { 248 | unsigned int i; 249 | unsigned int free_idx = randint(idx2-1); 250 | free(buffers2[free_idx]); 251 | buffers2[free_idx] = NULL; 252 | // shift the remaining pointers so it is easier 253 | // to track which ones are not freed yet 254 | for (i = free_idx; i <= idx2-1; i++) { 255 | buffers2[i] = buffers2[i+1]; 256 | } 257 | idx2--; 258 | printf("0"); 259 | total_cnt_frees += 1; 260 | } 261 | 262 | void test1() 263 | { 264 | unsigned int i; 265 | unsigned int count_allocs = rand_interval(1, MAX_COUNT_ALLOCATIONS); 266 | unsigned int count_frees = rand_interval(1, count_allocs); 267 | unsigned int divide_reallocs = rand_interval(1, 10); 268 | unsigned int count_reallocs = rand_interval(1, count_frees/divide_reallocs); 269 | 270 | #if defined(TESTDOC) 271 | unsigned int count_allocs_fastbins = MAX_COUNT_ALLOCATIONS_FASTBINS; 272 | unsigned int count_frees_fastbins = MAX_COUNT_ALLOCATIONS_FASTBINS; 273 | #else 274 | unsigned int count_allocs_fastbins = rand_interval(1, MAX_COUNT_ALLOCATIONS_FASTBINS); 275 | unsigned int count_frees_fastbins = rand_interval(1, count_allocs_fastbins); 276 | #endif 277 | 278 | for (i = 0; i < count_allocs; i++) { 279 | test1_alloc(); 280 | } 281 | // Free so some chunks are put in unsorted bin 282 | for (i = 0; i < count_frees; i++) { 283 | test1_free(); 284 | } 285 | // Chunks are not placed in regular bins until some of them have been given 286 | // one chance to be used in malloc 287 | for (i = 0; i < count_reallocs; i++) { 288 | test1_realloc(); 289 | } 290 | 291 | // We allocate fastbins after the above as otherwise we won't have anything 292 | // left in the fastbin as would have been moved to regular bins 293 | for (i = 0; i < count_allocs_fastbins; i++) { 294 | test1_alloc2(); 295 | } 296 | for (i = 0; i < count_frees_fastbins; i++) { 297 | test1_free2(); 298 | if (idx2 == 0) { 299 | printf("WARNING: too many frees detected!\n"); 300 | break; 301 | } 302 | } 303 | printf("\n"); 304 | printf("[+] Count allocs: %d done\n", count_allocs); 305 | printf("[+] Count frees: %d done\n", count_frees); 306 | printf("[+] Count reallocs: %d done\n", count_reallocs); 307 | printf("[+] Count allocs fastbins: %d done\n", count_allocs_fastbins); 308 | printf("[+] Count free fastbins: %d done\n", count_frees_fastbins); 309 | printf("[+] Total allocs: %d done\n", total_cnt_allocs); 310 | printf("[+] Total allocs: %d done\n", total_cnt_frees); 311 | } 312 | #else 313 | #ifdef TEST2 314 | void test2_alloc() 315 | { 316 | unsigned int multiplicator = rand_interval(1, 20); 317 | unsigned int alloc_size = randint(multiplicator*0x8000); 318 | buffers[idx] = malloc(alloc_size); 319 | sizes[idx] = alloc_size; 320 | memset(buffers[idx], next_letter(), alloc_size); 321 | idx++; 322 | printf("+"); 323 | total_cnt_allocs += 1; 324 | } 325 | 326 | void test2_free(unsigned int free_idx) 327 | { 328 | free(buffers[free_idx]); 329 | buffers[free_idx] = NULL; 330 | sizes_freed[idx_freed++] = sizes[free_idx]; 331 | // Set the size to 0x0 so it is easier 332 | // to track which ones are not freed yet 333 | sizes[free_idx] = 0x0; 334 | printf("-"); 335 | total_cnt_frees += 1; 336 | } 337 | 338 | void test2_realloc() 339 | { 340 | unsigned int i; 341 | unsigned int realloc_idx = randint(idx_freed-1); 342 | unsigned int alloc_size = sizes_freed[realloc_idx]; 343 | buffers[idx] = malloc(alloc_size); 344 | memset(buffers[idx], next_letter(), alloc_size); 345 | idx++; 346 | // shift the remaining pointers so it is easier 347 | // to track which ones are not freed yet 348 | for (i = realloc_idx; i <= idx_freed-1; i++) { 349 | sizes_freed[i] = sizes_freed[i+1]; 350 | } 351 | idx_freed--; 352 | printf("*"); 353 | total_cnt_allocs += 1; 354 | } 355 | 356 | void test2() 357 | { 358 | unsigned int i; 359 | unsigned int count_allocs = randint(MAX_COUNT_ALLOCATIONS); 360 | unsigned int count_frees = randint(count_allocs); 361 | unsigned int divide_reallocs = rand_interval(1, 10); 362 | unsigned int count_reallocs = randint(count_frees/divide_reallocs); 363 | 364 | for (i = 0; i < count_allocs; i++) { 365 | test2_alloc(); 366 | } 367 | // Free so some large chunks are put in unsorted bin 368 | // += 2 to do every other chunk to avoid coalescing 369 | for (i = 0; i < count_frees; i+=2) { 370 | test2_free(i); 371 | } 372 | // Chunks are not placed in regular bins until some of them have been given 373 | // one chance to be used in malloc 374 | for (i = 0; i < count_reallocs; i++) { 375 | test2_realloc(); 376 | } 377 | } 378 | #endif 379 | #endif 380 | 381 | int main(int argc, char* argv[]) 382 | { 383 | #ifndef TEST1B 384 | create_threads(); 385 | #endif 386 | 387 | // deterministic 388 | unsigned int seed = SEED; 389 | if (argc >= 2) { 390 | // We support passing the seed as from experience this influences a lot 391 | // the allocations happening so it eases adapting the bins we can analyse 392 | seed = atoi(argv[1]); 393 | } 394 | srand(seed); 395 | 396 | #if defined(TEST1) || defined(TEST1B) || defined(TESTDOC) 397 | test1(); 398 | #else 399 | #ifdef TEST2 400 | test2(); 401 | #endif 402 | #endif 403 | 404 | func1(); 405 | printf("[+] Exiting process\n"); 406 | exit(1); 407 | } -------------------------------------------------------------------------------- /test/test.gdb: -------------------------------------------------------------------------------- 1 | 2 | set verbose on 3 | set disassembly-flavor intel 4 | set height 0 5 | set pagination off 6 | set debug-file-directory /usr/lib/debug 7 | 8 | # Disable ASLR so we have deterministic memory layout 9 | set disable-randomization 10 | 11 | # so we can quit gdb automatically 12 | set confirm off 13 | 14 | b main 15 | commands 16 | silent 17 | source ../pyptmalloc-dev.py 18 | d 1 19 | c 20 | end 21 | 22 | run 23 | 24 | # This will be executed when it hits the "int3" of our "build/test1" binary 25 | source test2.gdb -------------------------------------------------------------------------------- /test/test.py: -------------------------------------------------------------------------------- 1 | import gdb 2 | 3 | def execute(command, log=False): 4 | output = gdb.execute(command, from_tty=True, to_string=True) 5 | if log: 6 | print(output) 7 | 8 | execute("ptfree", log=True) 9 | execute("q") -------------------------------------------------------------------------------- /test/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Allows to test different layouts 3 | # NB: delete `test.log` before running it to erase previous results 4 | # logging 100 items takes ~ 27-35 minutes 5 | 6 | for i in {1..100} 7 | do 8 | echo ------------------------------------------- >> test.log 9 | echo Handling $i >> test.log 10 | gdb -q -x test.gdb --args build/test1 $i 11 | #gdb -q -x test.gdb --args build/test1-32 $i 12 | #gdb -q -x test.gdb --args build/test2 $i 13 | #gdb -q -x test.gdb --args build/test2-32 $i 14 | done 15 | 16 | -------------------------------------------------------------------------------- /test/test2.gdb: -------------------------------------------------------------------------------- 1 | # NOTE: we can't just do that from test.py as it won't be taken into account into gdb? 2 | 3 | printf "Logging into a file\n" 4 | # Dump gdb session to a file 5 | # https://stackoverflow.com/questions/1707167/how-to-dump-the-entire-gdb-session-to-a-file-including-commands-i-type-and-thei 6 | set logging file test.log 7 | set logging overwrite off 8 | set logging on 9 | 10 | source test.py 11 | 12 | set logging off --------------------------------------------------------------------------------