├── .gitignore ├── LICENSE.md ├── Makefile ├── README.md ├── find_glibc_mapping.sh ├── find_main_arena.sh ├── ptmallocdump.c └── visualize_heap.rb /.gitignore: -------------------------------------------------------------------------------- 1 | libptmallocdump.so 2 | glibc_main_arena_address 3 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright (c) 2018-2019 Phusion Holding B.V. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | 2 | CFLAGS = -g -fPIC -DPIC -Wall -fno-strict-aliasing 3 | LDFLAGS = 4 | CC = gcc -std=c99 5 | RM = rm -f 6 | 7 | DUMPER_SRC = ptmallocdump.c 8 | DUMPER_LIB = libptmallocdump.so 9 | 10 | GLIBC_MAIN_ARENA_CACHE = glibc_main_arena_address 11 | 12 | TARGETS = $(DUMPER_LIB) $(GLIBC_MAIN_ARENA_CACHE) 13 | 14 | 15 | all: build 16 | build: $(TARGETS) 17 | 18 | $(DUMPER_LIB): $(DUMPER_SRC) 19 | $(CC) $(CFLAGS) -shared $^ $(LDFLAGS) -o $@ 20 | 21 | $(GLIBC_MAIN_ARENA_CACHE): $(DUMPER_LIB) 22 | ./find_main_arena.sh > $@ 23 | 24 | clean: 25 | @for file in $(TARGETS) ; do \ 26 | if test -f "$$file" ; then \ 27 | echo "$(RM) \"$$file\"" ; \ 28 | $(RM) "$$file" ; \ 29 | fi ; \ 30 | done 31 | 32 | .PHONY: all build clean 33 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ptmalloc2 heap dumper and visualizer 2 | 3 | This is a tool for dumping the ptmalloc2 heap into a file, and for visualizing that dump. I wrote this as part of my research into [what causes memory bloating in Ruby](https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html). 4 | 5 | ## Dumper 6 | 7 | ptmallocdump.c is a library responsible for dumping the heap to a file. 8 | 9 | ### Caveats 10 | 11 | * This dumper has only been tested on Ubuntu 18.04. The dumper relies on specific glibc/ptmalloc2 internals, so it will most likely break if you use it on any other OS/distro/glibc version. 12 | * This dumper is also *not thread-safe*. It does not acquire any ptmalloc2 mutexes. When using it, make sure that the process you're dumping is idling in all threads. 13 | * Although I haven't explicitly tested, this dumper is probably incompatible with swapping. It uses `mincore()` to check whether a page is in use or whether it has been `madvise(MADV_DONTNEED)`'ed, but I suspect that if a page has been swapped to disk then `mincore()` would return the same result. So disable swap space. 14 | 15 | ### Compilation 16 | 17 | Compile it as follows: 18 | 19 | gcc -shared -g ptmallocdump.c -fPIC -o libptmallocdump.so -Wall -fno-strict-aliasing 20 | 21 | ### Usage 22 | 23 | This library contains two functions that you must call in order to dump the heap of the current process: 24 | 25 | void dump_main_heap(const char *path, void *main_arena); 26 | void dump_non_main_heaps(const char *path, void *main_arena); 27 | 28 | If you want to call this library from Ruby then you can use FFI to load it: 29 | 30 | ~~~ruby 31 | require 'ffi' 32 | 33 | module PtmallocDumper 34 | extend FFI::Library 35 | ffi_lib '/path-to/libptmallocdump.so' 36 | 37 | attach_function :dump_non_main_heaps, [:string, :size_t], :void 38 | attach_function :dump_main_heap, [:string, :size_t], :void 39 | end 40 | ~~~ 41 | 42 | `dump_main_heap` dumps the ptmalloc2 main heap, while `dump_non_main_heaps` dumps all the other (i.e. non-main) heaps. For a full dump, you must call both functions. 43 | 44 | `path` is the file to which to dump to. That file will be opened in append mode so it's fine if you pass the same filename to both functions. 45 | 46 | `main_arena` is the address of the `main_arena` static global variable in glibc's malloc/arena.c. You can find out what that address is through the following method. It is required to have the glibc debugging symbols installed (`libc6-dbg` package). 47 | 48 | 1. Obtain the relative address of the 'main_arena' variable within glibc: 49 | 50 | objdump -t /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.27.so | grep ' main_arena' | awk '{ print $1 }' 51 | 52 | 2. Obtain the base address of the glibc library mapping in the process that you want to dump: 53 | 54 | grep '/libc-2.27.so$' /proc//maps | grep ' r-xp ' | cut -d- -f 1 55 | 56 | 3. Sum both addresses. That's the address to pass to the `main_arena` argument for those function calls. 57 | 58 | ## Visualizer 59 | 60 | ### Requirements 61 | 62 | gem install oily_png --no-document 63 | 64 | ### Usage 65 | 66 | ruby ./visualize_heap.rb 67 | -------------------------------------------------------------------------------- /find_glibc_mapping.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | die() { 4 | echo "$0: ERROR - $*" 1>&2 5 | exit 1 6 | } 7 | 8 | show_usage() { 9 | echo "Usage: $0 [options] " 10 | echo 11 | echo "OPTIONS" 12 | echo " -a, --add OFFSET Add the given hexidecimal offset" 13 | echo " to the mapped base address." 14 | } 15 | 16 | if test $# -lt 1 ; then 17 | show_usage 18 | exit 1 19 | fi 20 | 21 | mode=print 22 | offset=0 23 | 24 | if type getopt 2>&1 >/dev/null ; then 25 | # have GNU getopt (allows nicer options) 26 | SOPT="ha:" 27 | LOPT="help,add:" 28 | OPTIONS=$(getopt -o "$SOPT" --long "$LOPT" -n "$0" -- "$@") || exit 1 29 | eval set -- "$OPTIONS" 30 | fi 31 | 32 | while true ; do 33 | case "$1" in 34 | -h | --help) show_usage ; exit 0 ;; 35 | -a | --add) mode=add ; offset=$2 ; shift 2 ;; 36 | --) shift ; break ;; 37 | -*) die "bad opt: $1" ;; 38 | *) break ;; 39 | esac 40 | done 41 | 42 | pid=$1 43 | 44 | find_mapping_addr() { 45 | grep '/libc-[0-9.]*\.so$' /proc/${pid}/maps | 46 | grep ' r-xp ' | 47 | cut -d- -f 1 48 | } 49 | 50 | mapping_addr="$(find_mapping_addr)" 51 | 52 | sum_base_and_offset() { 53 | echo "ibase = 16; obase = 10; ${offset} + ${mapping_addr}" | bc 54 | } 55 | 56 | case $mode in 57 | print) echo "${mapping_addr}" ;; 58 | add) sum_base_and_offset ;; 59 | *) die "not a mode: \"${mode}\"" ;; 60 | esac 61 | -------------------------------------------------------------------------------- /find_main_arena.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | DEBUG_FILE_DIR="/usr/lib/debug" 4 | 5 | die() { 6 | echo "$0: ERROR - $*" 1>&2 7 | exit 1 8 | } 9 | 10 | DUMPER_LIB="$( grep ^DUMPER_LIB Makefile | awk '{ print $3 }' )" 11 | if ! test -f "${DUMPER_LIB}" ; then 12 | die "missing DUMPER_LIB (\"${DUMPER_LIB}\")" 13 | fi 14 | 15 | libc_soname_path() { 16 | ldd "${DUMPER_LIB}" | 17 | tr -d '\t' | 18 | grep ^libc.so.6 | 19 | awk '{ print $3 }' 20 | } 21 | 22 | real_libc_path="$( readlink -e "$(libc_soname_path)" )" 23 | libc_debug_path="${DEBUG_FILE_DIR}/${real_libc_path}.debug" 24 | 25 | if ! test -f "${libc_debug_path}" ; then 26 | die "missing libc debug symbols \"${libc_debug_path}\"" 27 | fi 28 | 29 | objdump -t "${libc_debug_path}" | 30 | grep ' main_arena' | 31 | awk '{ print $1 }' | 32 | tr 'a-f' 'A-F' 33 | -------------------------------------------------------------------------------- /ptmallocdump.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | #define SIZE_SZ (sizeof(size_t)) 11 | #define MALLOC_ALIGNMENT (2 * SIZE_SZ < __alignof__(long double) ? __alignof__(long double) : 2 * SIZE_SZ) 12 | #define MALLOC_ALIGN_MASK (MALLOC_ALIGNMENT - 1) 13 | #define PREV_INUSE 0x1 14 | #define IS_MMAPPED 0x2 15 | #define NON_MAIN_ARENA 0x4 16 | #define SIZE_BITS (PREV_INUSE | IS_MMAPPED | NON_MAIN_ARENA) 17 | #define DEFAULT_MMAP_THRESHOLD_MAX (4 * 1024 * 1024 * sizeof(long)) 18 | #define HEAP_MAX_SIZE (2 * DEFAULT_MMAP_THRESHOLD_MAX) 19 | #define NFASTBINS 10 20 | #define NBINS 128 21 | #define heap_for_ptr(ptr) ((struct heap_info *) ((unsigned long) (ptr) & ~(HEAP_MAX_SIZE - 1))) 22 | #define bin_at(m, i) \ 23 | (struct malloc_chunk *) (((char *) &((m)->bins[((i) - 1) * 2])) \ 24 | - offsetof (struct malloc_chunk, fd)) 25 | #define chunkdata(p) (((const char *) (p)) + 2 * sizeof(size_t)) 26 | #define chunksize(p) ((p)->mchunk_size & ~SIZE_BITS) 27 | 28 | #define MIN(a, b) ((a) < (b) ? (a) : (b)) 29 | 30 | struct heap_info; 31 | struct malloc_state; 32 | struct malloc_chunk; 33 | 34 | struct heap_info { 35 | struct malloc_state *ar_ptr; 36 | struct heap_info *prev; 37 | size_t size; 38 | size_t mprotect_size; 39 | char pad[0]; 40 | }; 41 | 42 | /* Also known as an arena */ 43 | struct malloc_state { 44 | int mutex; 45 | int flags; 46 | int have_fastchunks; 47 | 48 | void *fastbinsY[NFASTBINS]; 49 | struct malloc_chunk *top; 50 | struct malloc_chunk *last_remainder; 51 | struct malloc_chunk *bins[NBINS * 2 - 2]; 52 | unsigned int binmap[4]; 53 | struct malloc_state *next; 54 | struct malloc_state *next_free; 55 | 56 | size_t attached_threads; 57 | size_t system_mem; 58 | size_t max_system_mem; 59 | }; 60 | 61 | struct malloc_chunk { 62 | size_t mchunk_prev_size; 63 | size_t mchunk_size; 64 | struct malloc_chunk *fd; 65 | struct malloc_chunk *bk; 66 | struct malloc_chunk *fd_nextsize; 67 | struct malloc_chunk *bk_nextsize; 68 | }; 69 | 70 | static size_t 71 | generate_bindata_preview(char *output, size_t output_size, const char *input, size_t input_size) { 72 | size_t len = output_size; 73 | if (input_size < output_size) { 74 | len = input_size; 75 | } 76 | 77 | for (size_t i = 0; i < len; i++) { 78 | if (isprint(input[i])) { 79 | output[i] = input[i]; 80 | } else if (input[i] == '\0') { 81 | output[i] = '0'; 82 | } else { 83 | output[i] = '.'; 84 | } 85 | } 86 | 87 | return len; 88 | } 89 | 90 | static void 91 | print_page_usages(FILE *f, char *addr, size_t len, size_t pageSize) { 92 | char *baseAddr = (char *) (((uintptr_t) addr) & ~(pageSize - 1)); 93 | size_t numPages = (len + pageSize - 1) / pageSize; 94 | unsigned char pagesInUse[128 * 1024 + 1]; 95 | size_t measurableNumPages = MIN(numPages, sizeof(pagesInUse) - 1); 96 | size_t usableLen = measurableNumPages * pageSize; 97 | 98 | fprintf(f, "Pages in use for %p-%p: ", baseAddr, baseAddr + usableLen); 99 | 100 | int ret = mincore(baseAddr, usableLen, pagesInUse); 101 | if (ret == 0) { 102 | for (size_t i = 0; i < measurableNumPages; i++) { 103 | pagesInUse[i] = pagesInUse[i] ? '1' : '0'; 104 | } 105 | pagesInUse[sizeof(pagesInUse) - 1] = '\0'; 106 | fprintf(f, "%s", pagesInUse); 107 | if (measurableNumPages < numPages) { 108 | fprintf(f, " (incomplete)"); 109 | } 110 | fprintf(f, "\n"); 111 | } else { 112 | int e = errno; 113 | fprintf(f, "ERROR (%s)\n", strerror(e)); 114 | } 115 | } 116 | 117 | void 118 | dump_non_main_heap(const char *path, const struct heap_info *heap) { 119 | char *ptr; 120 | struct malloc_chunk *p, *next; 121 | FILE *f = fopen(path, "a"); 122 | long pageSize = sysconf(_SC_PAGESIZE); 123 | 124 | if (f == NULL) { 125 | fprintf(stderr, "ERROR: cannot open %s for writing.\n", path); 126 | return; 127 | } 128 | 129 | fprintf(f, "Heap %p size %10lu bytes:\n", heap, (unsigned long) heap->size); 130 | print_page_usages(f, (char *) heap, heap->size, pageSize); 131 | 132 | ptr = (heap->ar_ptr != (struct malloc_state *) (heap + 1)) ? 133 | (char *) (heap + 1) : (char *) (heap + 1) + sizeof(struct malloc_state); 134 | p = (struct malloc_chunk *) (((unsigned long) ptr + MALLOC_ALIGN_MASK) & 135 | ~MALLOC_ALIGN_MASK); 136 | 137 | while (p != NULL) { 138 | next = (struct malloc_chunk *) (((char *) p) + chunksize(p)); 139 | 140 | fprintf(f, "chunk %p size %10lu bytes", p, (unsigned long) chunksize(p)); 141 | if (p == heap->ar_ptr->top) { 142 | fprintf(f, " (top) "); 143 | next = NULL; 144 | } else if (p->mchunk_size == PREV_INUSE) { 145 | fprintf(f, " (fence)"); 146 | next = NULL; 147 | } else if (!(next->mchunk_size & PREV_INUSE)) { 148 | fprintf(f, " [free] "); 149 | } else { 150 | char preview[16]; 151 | size_t len = generate_bindata_preview(preview, sizeof(preview), 152 | chunkdata(p), chunksize(p)); 153 | fprintf(f, " "); 154 | fwrite(preview, 1, len, f); 155 | } 156 | 157 | fprintf(f, "\n"); 158 | 159 | p = next; 160 | } 161 | 162 | fclose(f); 163 | } 164 | 165 | void 166 | dump_non_main_heaps(const char *path, struct malloc_state *main_arena) { 167 | struct malloc_state *ar_ptr = main_arena->next; 168 | while (ar_ptr != main_arena) { 169 | struct heap_info *heap = heap_for_ptr(ar_ptr->top); 170 | do { 171 | dump_non_main_heap(path, heap); 172 | heap = heap->prev; 173 | } while (heap != NULL); 174 | ar_ptr = ar_ptr->next; 175 | } 176 | } 177 | 178 | void 179 | dump_main_heap(const char *path, struct malloc_state *main_arena) { 180 | struct malloc_chunk *base, *p; 181 | FILE *f = fopen(path, "a"); 182 | long pageSize = sysconf(_SC_PAGESIZE); 183 | 184 | if (f == NULL) { 185 | fprintf(stderr, "ERROR: cannot open %s for writing.\n", path); 186 | return; 187 | } 188 | 189 | base = (struct malloc_chunk *) (((const char *) main_arena->top) 190 | + chunksize(main_arena->top) - main_arena->system_mem); 191 | fprintf(f, "Heap %p size %10lu bytes:\n", base, main_arena->system_mem); 192 | print_page_usages(f, (char *) base, main_arena->system_mem, pageSize); 193 | 194 | p = base; 195 | while (p != NULL) { 196 | struct malloc_chunk *next = (struct malloc_chunk *) (((char *) p) + chunksize(p)); 197 | 198 | fprintf(f, "chunk %p size %10lu bytes", p, (unsigned long) chunksize(p)); 199 | if (p == main_arena->top) { 200 | fprintf(f, " (top) "); 201 | next = NULL; 202 | } else if (p->mchunk_size == PREV_INUSE) { 203 | fprintf(f, " (fence)"); 204 | next = NULL; 205 | } else if (!(next->mchunk_size & PREV_INUSE)) { 206 | fprintf(f, " [free] "); 207 | } else { 208 | char preview[16]; 209 | size_t len = generate_bindata_preview(preview, sizeof(preview), 210 | chunkdata(p), chunksize(p)); 211 | fprintf(f, " "); 212 | fwrite(preview, 1, len, f); 213 | } 214 | 215 | fprintf(f, "\n");fflush(f); 216 | 217 | p = next; 218 | } 219 | 220 | fclose(f); 221 | } 222 | -------------------------------------------------------------------------------- /visualize_heap.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | require 'oily_png' 4 | require 'erb' 5 | 6 | module HeapVisualizer 7 | PAGE_SIZE = 4096 8 | PAGE_SIZE_MASK = PAGE_SIZE - 1 9 | BLOCK_SIZE = 16 10 | 11 | class Heap < Struct.new(:number, :addr, :size, :chunks, :pages, :page_dirtiness) 12 | def initialize(*args) 13 | super 14 | self.chunks ||= [] 15 | self.pages ||= [] 16 | self.page_dirtiness ||= {} 17 | end 18 | 19 | def maybe_dirty_pages 20 | pages.find_all { |p| p.maybe_dirty? } 21 | end 22 | 23 | def clean_pages 24 | pages.find_all { |p| !p.maybe_dirty? } 25 | end 26 | end 27 | 28 | class Chunk < Struct.new(:heap, :addr, :number, :size, :type, :preview) 29 | def offset 30 | addr - heap.addr 31 | end 32 | end 33 | 34 | class Page < Struct.new(:heap, :addr, :dirty, :blocks) 35 | def initialize(*args) 36 | super 37 | self.blocks ||= [] 38 | end 39 | 40 | def maybe_dirty? 41 | dirty != false # considering nil too, which means 'unknown' 42 | end 43 | end 44 | 45 | class Block < Struct.new(:page, :chunk, :addr, :number, :end_of_chunk) 46 | def heap 47 | page.heap 48 | end 49 | 50 | def end_of_chunk? 51 | end_of_chunk 52 | end 53 | 54 | def used? 55 | chunk.type == :used 56 | end 57 | end 58 | 59 | # Given a heaps_chunk.log, parses the file into data structures 60 | # that represent the heaps and the containing chunks. 61 | class HeapChunksLogParser 62 | attr_reader :heaps 63 | 64 | def initialize(path) 65 | @path = path 66 | @heaps = [] 67 | @offset = 0 68 | end 69 | 70 | def parse 71 | read_each_line do |line| 72 | if line =~ /^chunk ([a-z0-9]+) size +([0-9]+) bytes (\(top\)|\(fence\) |\[free\]| ) *(.*)/ 73 | process_chunk(line, $1, $2, $3, $4) 74 | elsif line =~ /^Heap ([a-z0-9]+) size +([0-9]+) /i 75 | process_heap_start(line, $1, $2) 76 | elsif line =~ /^Pages in use for 0x([0-9a-f]+)-0x([0-9a-f]+): ([10\?]+)/ 77 | process_pages_in_use(line, $1, $2, $3) 78 | end 79 | end 80 | 81 | sort_all! 82 | 83 | self 84 | end 85 | 86 | private 87 | def read_each_line 88 | File.open(@path, 'r:utf-8') do |f| 89 | while !f.eof? 90 | line = f.readline.strip 91 | yield(line) 92 | end 93 | end 94 | end 95 | 96 | def process_heap_start(line, addr, size) 97 | heap = Heap.new(@heaps.size, addr.to_i(16), size.to_i) 98 | @heaps << heap 99 | end 100 | 101 | def process_pages_in_use(line, start_addr, end_addr, usage) 102 | heap = @heaps.last 103 | start_addr = start_addr.to_i(16) 104 | end_addr = end_addr.to_i(16) 105 | 106 | addr = start_addr 107 | while addr < end_addr 108 | heap.page_dirtiness[addr] = 109 | case usage[(addr - start_addr) / PAGE_SIZE] 110 | when '1' 111 | true 112 | when '0' 113 | false 114 | else 115 | nil 116 | end 117 | addr += PAGE_SIZE 118 | end 119 | end 120 | 121 | def process_chunk(line, addr, size, type, preview) 122 | heap = @heaps.last 123 | chunk = Chunk.new( 124 | heap, 125 | addr.to_i(16), 126 | heap.chunks.size, 127 | size.to_i, 128 | analyze_chunk_type(type), 129 | preview.empty? ? nil : preview) 130 | heap.chunks << chunk 131 | end 132 | 133 | def analyze_chunk_type(type) 134 | if type =~ /top/ 135 | :top 136 | elsif type =~ /fence/ 137 | :fence 138 | elsif type =~ /free/ 139 | :free 140 | else 141 | :used 142 | end 143 | end 144 | 145 | def sort_all! 146 | @heaps.sort! do |a, b| 147 | a.addr <=> b.addr 148 | end 149 | @heaps.each do |heap| 150 | heap.chunks.sort! do |a, b| 151 | a.addr <=> b.addr 152 | end 153 | end 154 | end 155 | end 156 | 157 | # Heap a collection of Heap objects, splits their chunks 158 | # into pages and blocks of BLOCK_SIZE, for use in visualization. 159 | class ChunkSplitter 160 | attr_reader :heaps 161 | 162 | def initialize(heaps) 163 | @heaps = heaps 164 | end 165 | 166 | def perform 167 | @heaps.each do |heap| 168 | chunk = heap.chunks.first 169 | last_chunk = heap.chunks.last 170 | last_chunk_end_addr = last_chunk.addr + last_chunk.size 171 | addr = chunk.addr 172 | 173 | while addr < last_chunk_end_addr 174 | page_addr = addr & ~PAGE_SIZE_MASK 175 | page = Page.new( 176 | heap, 177 | page_addr, 178 | heap.page_dirtiness[page_addr] 179 | ) 180 | heap.pages << page 181 | 182 | page_or_last_chunk_end_addr = [ 183 | page.addr + PAGE_SIZE, 184 | last_chunk_end_addr 185 | ].min 186 | 187 | while addr < page_or_last_chunk_end_addr 188 | block = Block.new( 189 | page, 190 | chunk, 191 | addr, 192 | page.blocks.size, 193 | addr + BLOCK_SIZE >= chunk.addr + chunk.size 194 | ) 195 | page.blocks << block 196 | 197 | addr += BLOCK_SIZE 198 | if block.end_of_chunk? 199 | chunk = heap.chunks[chunk.number + 1] 200 | end 201 | end 202 | end 203 | 204 | heap.page_dirtiness.clear 205 | heap.page_dirtiness = nil 206 | end 207 | 208 | self 209 | end 210 | end 211 | 212 | class HtmlVisualizer 213 | def initialize(heaps, dir) 214 | @heaps = heaps 215 | @dir = dir 216 | end 217 | 218 | def perform 219 | open_html_file do 220 | start_of_document 221 | save_clean_page_image 222 | @heaps.each_with_index do |heap, i| 223 | puts "Writing heap 0x#{heap.addr.to_s(16)} [#{i + 1}/#{@heaps.size}]" 224 | start_of_heap(heap) 225 | heap.pages.each do |page| 226 | start_of_page(page) 227 | write_page_image(page) 228 | end_of_page(page) 229 | end 230 | end_of_heap(heap) 231 | end 232 | end_of_document 233 | end 234 | 235 | self 236 | end 237 | 238 | private 239 | NUM_BLOCKS_1D = Math.sqrt(PAGE_SIZE / BLOCK_SIZE).to_i 240 | BLOCK_SCALE = 1 241 | PAGE_BG_COLOR = ChunkyPNG::Color.rgb(0x77, 0x77, 0x77) 242 | USED_BLOCK_COLORS = [ 243 | ChunkyPNG::Color.rgb(0xff, 0, 0), 244 | ChunkyPNG::Color.rgb(0xf0, 0, 0), 245 | ChunkyPNG::Color.rgb(0xe1, 0, 0), 246 | ChunkyPNG::Color.rgb(0xd2, 0, 0), 247 | ] 248 | FREE_BLOCK_COLORS = [ 249 | ChunkyPNG::Color.rgb(0xce, 0xce, 0xce), 250 | ChunkyPNG::Color.rgb(0xbf, 0xbf, 0xbf), 251 | ChunkyPNG::Color.rgb(0xb0, 0xb0, 0xb0), 252 | ChunkyPNG::Color.rgb(0xa1, 0xa1, 0xa1), 253 | ] 254 | CLEAN_PAGE_COLOR = ChunkyPNG::Color.rgb(0xff, 0xff, 0xff) 255 | CLEAN_PAGE_IMAGE = ChunkyPNG::Image.new( 256 | NUM_BLOCKS_1D * BLOCK_SCALE, 257 | NUM_BLOCKS_1D * BLOCK_SCALE, 258 | CLEAN_PAGE_COLOR) 259 | CLEAN_PAGE_IMAGE_BASE_NAME = 'page-clean.png' 260 | 261 | STYLESHEET = %Q{ 262 | body { 263 | font-family: sans-serif; 264 | } 265 | 266 | heap { 267 | display: block; 268 | border: solid 1px black; 269 | margin-bottom: 2rem; 270 | } 271 | 272 | page-title { 273 | display: none; 274 | } 275 | 276 | heap-title, 277 | heap-content { 278 | display: block; 279 | } 280 | 281 | heap-title { 282 | padding: 1rem; 283 | } 284 | 285 | heap-title h2 { 286 | margin: 0; 287 | } 288 | 289 | heap-title .stats td, 290 | heap-title .stats th { 291 | text-align: right; 292 | padding-right: 1em; 293 | } 294 | 295 | page { 296 | display: inline-block; 297 | vertical-align: top; 298 | border: solid 1px #777; 299 | } 300 | } 301 | 302 | def open_html_file 303 | File.open("#{@dir}/index.html", 'w:utf-8') do |f| 304 | @io = f 305 | yield 306 | end 307 | end 308 | 309 | def save_clean_page_image 310 | CLEAN_PAGE_IMAGE.save(clean_page_image_path) 311 | end 312 | 313 | def clean_page_image_path 314 | @clean_page_image_path ||= "#{@dir}/#{CLEAN_PAGE_IMAGE_BASE_NAME}" 315 | end 316 | 317 | def start_of_document 318 | @io.printf %Q{\n} 319 | @io.printf %Q{\n} 320 | @io.printf %Q{\tHeap visualizer\n} 321 | @io.printf %Q{\t\n} 322 | @io.printf %Q{\n} 323 | @io.printf %Q{\n} 324 | 325 | File.open("#{@dir}/stylesheet.css", 'w:utf-8') do |f| 326 | f.write(STYLESHEET) 327 | end 328 | end 329 | 330 | def start_of_heap(heap) 331 | maybe_dirty_pages = heap.maybe_dirty_pages 332 | clean_pages = heap.clean_pages 333 | 334 | @io.printf %Q{\n} 335 | @io.printf %Q{ 336 | 337 |

Heap %d — 0x%08x

338 | 339 | 340 | 341 | 342 | 343 | 344 | 345 | 346 | 347 | 348 | 349 | 350 | 351 | 352 | 353 | 354 | 355 | 356 |
Virtual%.1f MB%d pages
Dirty%.1f MB%d pages%d%%
Clean%.1f MB%d pages%d%%
357 | 358 | }, heap.number, heap.addr, 359 | 360 | heap.size / 1024.0 / 1024, heap.pages.size, 361 | 362 | maybe_dirty_pages.size * PAGE_SIZE / 1024.0 / 1024, 363 | maybe_dirty_pages.size, 364 | maybe_dirty_pages.size * 100 / heap.pages.size, 365 | 366 | clean_pages.size * PAGE_SIZE / 1024.0 / 1024, 367 | clean_pages.size, 368 | clean_pages.size * 100 / heap.pages.size 369 | @io.printf %Q{\t} 370 | end 371 | 372 | def start_of_page(page) 373 | @io.printf %Q{} 374 | @io.printf %Q{%08x-%08x}, 375 | page.addr, page.addr + page.size 376 | end 377 | 378 | def write_page_image(page) 379 | title = "0x#{page.addr.to_s(16)}" 380 | if page.maybe_dirty? 381 | image = ChunkyPNG::Image.new( 382 | NUM_BLOCKS_1D * BLOCK_SCALE, 383 | NUM_BLOCKS_1D * BLOCK_SCALE, 384 | PAGE_BG_COLOR) 385 | page.blocks.each_with_index do |block, block_index| 386 | x = (block_index % NUM_BLOCKS_1D) * BLOCK_SCALE 387 | y = (block_index / NUM_BLOCKS_1D) * BLOCK_SCALE 388 | color = color_for_block(block) 389 | BLOCK_SCALE.times do |scale_x_index| 390 | BLOCK_SCALE.times do |scale_y_index| 391 | image[x + scale_x_index, y + scale_y_index] = color 392 | end 393 | end 394 | end 395 | 396 | path, basename = path_for_page_image(page) 397 | image.save(path) 398 | @io.printf %Q{} 399 | else 400 | @io.printf %Q{} 401 | end 402 | end 403 | 404 | def color_for_block(block) 405 | if block.used? 406 | USED_BLOCK_COLORS[block.chunk.number % USED_BLOCK_COLORS.size] 407 | else 408 | FREE_BLOCK_COLORS[block.chunk.number % USED_BLOCK_COLORS.size] 409 | end 410 | end 411 | 412 | def path_for_page_image(page) 413 | basename = "page-#{page.addr.to_s(16)}.png" 414 | ["#{@dir}/#{basename}", basename] 415 | end 416 | 417 | def end_of_page(page) 418 | @io.printf %Q{} 419 | end 420 | 421 | def end_of_heap(heap) 422 | @io.printf %Q{\n} 423 | @io.printf %Q{\n} 424 | end 425 | 426 | def end_of_document 427 | @io.printf %Q{\n} 428 | @io.printf %Q{\n} 429 | end 430 | end 431 | end 432 | 433 | if ARGV.size != 2 434 | abort "Usage: ./visualize_heap.rb " 435 | end 436 | 437 | puts "Parsing file" 438 | parser = HeapVisualizer::HeapChunksLogParser.new(ARGV[0]).parse 439 | puts "Splitting heap chunks" 440 | splitter = HeapVisualizer::ChunkSplitter.new(parser.heaps).perform 441 | heaps = splitter.heaps 442 | 443 | puts "Garbage collecting" 444 | GC.start 445 | 446 | puts "Writing output" 447 | HeapVisualizer::HtmlVisualizer.new(heaps, ARGV[1]).perform 448 | --------------------------------------------------------------------------------