├── .gitignore ├── .travis.yml ├── ChangeLog ├── README.md ├── binding.gyp ├── examples ├── basic_heapdiff.js ├── do_nothing_server.js └── slightly_leaky.js ├── include.js ├── package.json ├── src ├── heapdiff.cc ├── heapdiff.hh ├── init.cc ├── memwatch.cc ├── memwatch.hh ├── platformcompat.hh ├── util.cc └── util.hh └── tests.js /.gitignore: -------------------------------------------------------------------------------- 1 | /build 2 | /node_modules 3 | *~ 4 | 5 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: node_js 2 | 3 | node_js: 4 | - 0.6 5 | - 0.8 6 | - 0.10 7 | 8 | notifications: 9 | email: 10 | - lloyd@hilaiel.com 11 | -------------------------------------------------------------------------------- /ChangeLog: -------------------------------------------------------------------------------- 1 | v0.2.2 - 2 | * don't crash when a user accidentally allocates a HeapDiff without new (ala new require('memwatch').HeapDiff()) #30 3 | 4 | v0.2.1 - 5 | * 0.10.0 support (thanks @rvagg and @tmuellerleile) 6 | * improved windows build support (no longer requires sed in path, thanks @mscdex) 7 | * work around a windows specific crash due to upstream "bug" in libuv (https://github.com/joyent/libuv/pull/629) 8 | 9 | v0.2.0 - 10 | * fix memory leak of snapshots in HeapDiff #15 11 | * HeapDiff.end() throws an exception if invoked more than once. 12 | * aggressively clean up snapshots, at end() rather than next gc 13 | 14 | v0.1.5 - 15 | * compiles on windows (thanks @jmatthewsr-ms! sorry to make you wait) 16 | 17 | v0.1.4 - 18 | * migrate to node-gyp (thanks @jhaynie for getting it started) 19 | 20 | v0.1.3 - 21 | * node 0.8 support 22 | 23 | v0.1.2 - 24 | 25 | * Addition of unit tests (running on travis) 26 | * fix bug whereby events would not be emitted when listeners use .once() 27 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | `node-memwatch`: Leak Detection and Heap Diffing for Node.JS 2 | ============================================================ 3 | 4 | [![Build Status](https://secure.travis-ci.org/lloyd/node-memwatch.png)](http://travis-ci.org/lloyd/node-memwatch) 5 | 6 | `node-memwatch` is here to help you detect and find memory leaks in 7 | Node.JS code. It provides: 8 | 9 | - A `leak` event, emitted when it appears your code is leaking memory. 10 | 11 | - A `stats` event, emitted occasionally, giving you 12 | data describing your heap usage and trends over time. 13 | 14 | - A `HeapDiff` class that lets you compare the state of your heap between 15 | two points in time, telling you what has been allocated, and what 16 | has been released. 17 | 18 | 19 | Installation 20 | ------------ 21 | 22 | - `npm install memwatch` 23 | 24 | or 25 | 26 | - `git clone git://github.com/lloyd/node-memwatch.git` 27 | 28 | 29 | Description 30 | ----------- 31 | 32 | There are a growing number of tools for debugging and profiling memory 33 | usage in Node.JS applications, but there is still a need for a 34 | platform-independent native module that requires no special 35 | instrumentation. This module attempts to satisfy that need. 36 | 37 | To get started, import `node-memwatch` like so: 38 | 39 | ```javascript 40 | var memwatch = require('memwatch'); 41 | ``` 42 | 43 | ### Leak Detection 44 | 45 | You can then subscribe to `leak` events. A `leak` event will be 46 | emitted when your heap usage has increased for five consecutive 47 | garbage collections: 48 | 49 | ```javascript 50 | memwatch.on('leak', function(info) { ... }); 51 | ``` 52 | 53 | The `info` object will look something like: 54 | 55 | ```javascript 56 | { start: Fri, 29 Jun 2012 14:12:13 GMT, 57 | end: Fri, 29 Jun 2012 14:12:33 GMT, 58 | growth: 67984, 59 | reason: 'heap growth over 5 consecutive GCs (20s) - 11.67 mb/hr' } 60 | ``` 61 | 62 | 63 | ### Heap Usage 64 | 65 | The best way to evaluate your memory footprint is to look at heap 66 | usage right aver V8 performs garbage collection. `memwatch` does 67 | exactly this - it checks heap usage only after GC to give you a stable 68 | baseline of your actual memory usage. 69 | 70 | When V8 performs a garbage collection (technically, we're talking 71 | about a full GC with heap compaction), `memwatch` will emit a `stats` 72 | event. 73 | 74 | ```javascript 75 | memwatch.on('stats', function(stats) { ... }); 76 | ``` 77 | 78 | The `stats` data will look something like this: 79 | 80 | ```javascript 81 | { 82 | "num_full_gc": 17, 83 | "num_inc_gc": 8, 84 | "heap_compactions": 8, 85 | "estimated_base": 2592568, 86 | "current_base": 2592568, 87 | "min": 2499912, 88 | "max": 2592568, 89 | "usage_trend": 0 90 | } 91 | ``` 92 | 93 | `estimated_base` and `usage_trend` are tracked over time. If usage 94 | trend is consistently positive, it indicates that your base heap size 95 | is continuously growing and you might have a leak. 96 | 97 | V8 has its own idea of when it's best to perform a GC, and under a 98 | heavy load, it may defer this action for some time. To aid in 99 | speedier debugging, `memwatch` provides a `gc()` method to force V8 to 100 | do a full GC and heap compaction. 101 | 102 | 103 | ### Heap Diffing 104 | 105 | So far we have seen how `memwatch` can aid in leak detection. For 106 | leak isolation, it provides a `HeapDiff` class that takes two snapshots 107 | and computes a diff between them. For example: 108 | 109 | ```javascript 110 | // Take first snapshot 111 | var hd = new memwatch.HeapDiff(); 112 | 113 | // do some things ... 114 | 115 | // Take the second snapshot and compute the diff 116 | var diff = hd.end(); 117 | ``` 118 | 119 | The contents of `diff` will look something like: 120 | 121 | ```javascript 122 | { 123 | "before": { "nodes": 11625, "size_bytes": 1869904, "size": "1.78 mb" }, 124 | "after": { "nodes": 21435, "size_bytes": 2119136, "size": "2.02 mb" }, 125 | "change": { "size_bytes": 249232, "size": "243.39 kb", "freed_nodes": 197, 126 | "allocated_nodes": 10007, 127 | "details": [ 128 | { "what": "String", 129 | "size_bytes": -2120, "size": "-2.07 kb", "+": 3, "-": 62 130 | }, 131 | { "what": "Array", 132 | "size_bytes": 66687, "size": "65.13 kb", "+": 4, "-": 78 133 | }, 134 | { "what": "LeakingClass", 135 | "size_bytes": 239952, "size": "234.33 kb", "+": 9998, "-": 0 136 | } 137 | ] 138 | } 139 | ``` 140 | 141 | The diff shows that during the sample period, the total number of 142 | allocated `String` and `Array` classes decreased, but `Leaking Class` 143 | grew by 9998 allocations. Hmmm. 144 | 145 | You can use `HeapDiff` in your `on('stats')` callback; even though it 146 | takes a memory snapshot, which triggers a V8 GC, it will not trigger 147 | the `stats` event itself. Because that would be silly. 148 | 149 | 150 | Future Work 151 | ----------- 152 | 153 | Please see the Issues to share suggestions and contribute! 154 | 155 | 156 | License 157 | ------- 158 | 159 | http://wtfpl.org 160 | -------------------------------------------------------------------------------- /binding.gyp: -------------------------------------------------------------------------------- 1 | { 2 | 'targets': [ 3 | { 4 | 'target_name': 'memwatch', 5 | 'include_dirs': [ 6 | ], 7 | 'sources': [ 8 | 'src/heapdiff.cc', 9 | 'src/init.cc', 10 | 'src/memwatch.cc', 11 | 'src/util.cc' 12 | ], 13 | } 14 | ] 15 | } 16 | -------------------------------------------------------------------------------- /examples/basic_heapdiff.js: -------------------------------------------------------------------------------- 1 | const 2 | memwatch = require('..'), 3 | url = require('url'); 4 | 5 | function LeakingClass() { 6 | } 7 | 8 | memwatch.gc(); 9 | 10 | var arr = []; 11 | 12 | var hd = new memwatch.HeapDiff(); 13 | 14 | for (var i = 0; i < 10000; i++) arr.push(new LeakingClass); 15 | 16 | var hde = hd.end(); 17 | 18 | console.log(JSON.stringify(hde, null, 2)); 19 | 20 | 21 | 22 | 23 | -------------------------------------------------------------------------------- /examples/do_nothing_server.js: -------------------------------------------------------------------------------- 1 | // a trivial process that does nothing except 2 | // trigger GC and output the present base memory 3 | // usage every second. this example is intended to 4 | // demonstrate that memwatch itself does not leak. 5 | 6 | var memwatch = require('../'); 7 | 8 | memwatch.on('gc', function(d) { 9 | if (d.compacted) { 10 | console.log('current base memory usage:', memwatch.stats().current_base); 11 | } 12 | }); 13 | 14 | setInterval(function() { 15 | memwatch.gc(); 16 | }, 1000); 17 | -------------------------------------------------------------------------------- /examples/slightly_leaky.js: -------------------------------------------------------------------------------- 1 | // a trivial process that does nothing except 2 | // trigger GC and output the present base memory 3 | // usage every second. this example is intended to 4 | // demonstrate that memwatch itself does not leak. 5 | 6 | var http = require('http'); 7 | 8 | var start = new Date(); 9 | function msFromStart() { 10 | return new Date() - start; 11 | } 12 | 13 | var leak = []; 14 | 15 | // every second, this program "leaks" a little bit 16 | setInterval(function() { 17 | for (var i = 0; i < 10; i++) { 18 | var str = i.toString() + " on a stick, short and stout!"; 19 | leak.push(str); 20 | } 21 | }, 1000); 22 | 23 | // meantime, the program is busy, doing *lots* of http requests 24 | var http = require('http'); 25 | http.createServer(function (req, res) { 26 | res.writeHead(200, {'Content-Type': 'text/plain'}); 27 | res.end('Hello World\n'); 28 | }).listen(1337, '127.0.0.1'); 29 | 30 | function doHTTPRequest() { 31 | var options = { 32 | host: '127.0.0.1', 33 | port: 1337, 34 | path: '/index.html' 35 | }; 36 | 37 | http.get(options, function(res) { 38 | setTimeout(doHTTPRequest, 300); 39 | }).on('error', function(e) { 40 | setTimeout(doHTTPRequest, 300); 41 | }); 42 | } 43 | 44 | doHTTPRequest(); 45 | doHTTPRequest(); 46 | 47 | var memwatch = require('../'); 48 | 49 | // report to console postgc heap size 50 | memwatch.on('stats', function(d) { 51 | console.log("postgc:", msFromStart(), d.current_base); 52 | }); 53 | 54 | memwatch.on('leak', function(d) { 55 | console.log("LEAK:", d); 56 | }); 57 | 58 | // also report periodic heap size (every 10s) 59 | setInterval(function() { 60 | console.log("naive:", msFromStart(), process.memoryUsage().heapUsed); 61 | }, 5000); 62 | -------------------------------------------------------------------------------- /include.js: -------------------------------------------------------------------------------- 1 | const 2 | magic = require('./build/Release/memwatch'), 3 | events = require('events'); 4 | 5 | module.exports = new events.EventEmitter(); 6 | 7 | module.exports.gc = magic.gc; 8 | module.exports.HeapDiff = magic.HeapDiff; 9 | 10 | magic.upon_gc(function(has_listeners, event, data) { 11 | if (has_listeners) { 12 | return (module.exports.listeners('stats').length > 0); 13 | } else { 14 | return module.exports.emit(event, data); 15 | } 16 | }); 17 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "memwatch", 3 | "description": "Keep an eye on your memory usage, and discover and isolate leaks.", 4 | "version": "0.2.2", 5 | "author": "Lloyd Hilaiel (http://lloyd.io)", 6 | "engines": { "node": ">= 0.6.0" }, 7 | "repository": { 8 | "type": "git", 9 | "url": "https://github.com/lloyd/node-memwatch.git" 10 | }, 11 | "main": "include.js", 12 | "licenses": [ { "type": "wtfpl" } ], 13 | "bugs": { 14 | "url" : "https://github.com/lloyd/node-memwatch/issues" 15 | }, 16 | "scripts": { 17 | "install": "node-gyp rebuild", 18 | "test": "mocha tests" 19 | }, 20 | "devDependencies": { 21 | "mocha": "1.2.2", 22 | "should": "0.6.3", 23 | "node-gyp": "0.5.7" 24 | }, 25 | "contributors": [ 26 | "Jed Parsons (@jedp)", 27 | "Jeff Haynie (@jhaynie)", 28 | "Justin Matthews (@jmatthewsr-ms)" 29 | ] 30 | } 31 | -------------------------------------------------------------------------------- /src/heapdiff.cc: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #include "heapdiff.hh" 6 | #include "util.hh" 7 | 8 | #include 9 | 10 | #include 11 | #include 12 | #include 13 | #include 14 | 15 | #include // abs() 16 | #include // time() 17 | 18 | using namespace v8; 19 | using namespace node; 20 | using namespace std; 21 | 22 | static bool s_inProgress = false; 23 | static time_t s_startTime; 24 | 25 | bool heapdiff::HeapDiff::InProgress() 26 | { 27 | return s_inProgress; 28 | } 29 | 30 | heapdiff::HeapDiff::HeapDiff() : ObjectWrap(), before(NULL), after(NULL), 31 | ended(false) 32 | { 33 | } 34 | 35 | heapdiff::HeapDiff::~HeapDiff() 36 | { 37 | if (before) { 38 | ((HeapSnapshot *) before)->Delete(); 39 | before = NULL; 40 | } 41 | 42 | if (after) { 43 | ((HeapSnapshot *) after)->Delete(); 44 | after = NULL; 45 | } 46 | } 47 | 48 | void 49 | heapdiff::HeapDiff::Initialize ( v8::Handle target ) 50 | { 51 | v8::HandleScope scope; 52 | v8::Local t = v8::FunctionTemplate::New(New); 53 | t->InstanceTemplate()->SetInternalFieldCount(1); 54 | t->SetClassName(String::NewSymbol("HeapDiff")); 55 | 56 | NODE_SET_PROTOTYPE_METHOD(t, "end", End); 57 | 58 | target->Set(v8::String::NewSymbol( "HeapDiff"), t->GetFunction()); 59 | } 60 | 61 | v8::Handle 62 | heapdiff::HeapDiff::New (const v8::Arguments& args) 63 | { 64 | // Don't blow up when the caller says "new require('memwatch').HeapDiff()" 65 | // issue #30 66 | // stolen from: https://github.com/kkaefer/node-cpp-modules/commit/bd9432026affafd8450ecfd9b49b7dc647b6d348 67 | if (!args.IsConstructCall()) { 68 | return ThrowException( 69 | Exception::TypeError( 70 | String::New("Use the new operator to create instances of this object."))); 71 | } 72 | 73 | v8::HandleScope scope; 74 | 75 | // allocate the underlying c++ class and wrap it up in the this pointer 76 | HeapDiff * self = new HeapDiff(); 77 | self->Wrap(args.This()); 78 | 79 | // take a snapshot and save a pointer to it 80 | s_inProgress = true; 81 | s_startTime = time(NULL); 82 | self->before = v8::HeapProfiler::TakeSnapshot(v8::String::New("")); 83 | s_inProgress = false; 84 | 85 | return args.This(); 86 | } 87 | 88 | static string handleToStr(const Handle & str) 89 | { 90 | String::Utf8Value utfString(str->ToString()); 91 | return *utfString; 92 | } 93 | 94 | static void 95 | buildIDSet(set * seen, const HeapGraphNode* cur, int & s) 96 | { 97 | v8::HandleScope scope; 98 | 99 | // cycle detection 100 | if (seen->find(cur->GetId()) != seen->end()) { 101 | return; 102 | } 103 | // always ignore HeapDiff related memory 104 | if (cur->GetType() == HeapGraphNode::kObject && 105 | handleToStr(cur->GetName()).compare("HeapDiff") == 0) 106 | { 107 | return; 108 | } 109 | 110 | // update memory usage as we go 111 | s += cur->GetSelfSize(); 112 | 113 | seen->insert(cur->GetId()); 114 | 115 | for (int i=0; i < cur->GetChildrenCount(); i++) { 116 | buildIDSet(seen, cur->GetChild(i)->GetToNode(), s); 117 | } 118 | } 119 | 120 | typedef set idset; 121 | 122 | // why doesn't STL work? 123 | // XXX: improve this algorithm 124 | void setDiff(idset a, idset b, vector &c) 125 | { 126 | for (idset::iterator i = a.begin(); i != a.end(); i++) { 127 | if (b.find(*i) == b.end()) c.push_back(*i); 128 | } 129 | } 130 | 131 | 132 | class example 133 | { 134 | public: 135 | HeapGraphEdge::Type context; 136 | HeapGraphNode::Type type; 137 | std::string name; 138 | std::string value; 139 | std::string heap_value; 140 | int self_size; 141 | int retained_size; 142 | int retainers; 143 | 144 | example() : context(HeapGraphEdge::kHidden), 145 | type(HeapGraphNode::kHidden), 146 | self_size(0), retained_size(0), retainers(0) { }; 147 | }; 148 | 149 | class change 150 | { 151 | public: 152 | long int size; 153 | long int added; 154 | long int released; 155 | std::vector examples; 156 | 157 | change() : size(0), added(0), released(0) { } 158 | }; 159 | 160 | typedef std::mapchangeset; 161 | 162 | static void manageChange(changeset & changes, const HeapGraphNode * node, bool added) 163 | { 164 | std::string type; 165 | 166 | switch(node->GetType()) { 167 | case HeapGraphNode::kArray: 168 | type.append("Array"); 169 | break; 170 | case HeapGraphNode::kString: 171 | type.append("String"); 172 | break; 173 | case HeapGraphNode::kObject: 174 | type.append(handleToStr(node->GetName())); 175 | break; 176 | case HeapGraphNode::kCode: 177 | type.append("Code"); 178 | break; 179 | case HeapGraphNode::kClosure: 180 | type.append("Closure"); 181 | break; 182 | case HeapGraphNode::kRegExp: 183 | type.append("RegExp"); 184 | break; 185 | case HeapGraphNode::kHeapNumber: 186 | type.append("Number"); 187 | break; 188 | case HeapGraphNode::kNative: 189 | type.append("Native"); 190 | break; 191 | case HeapGraphNode::kHidden: 192 | default: 193 | return; 194 | } 195 | 196 | if (changes.find(type) == changes.end()) { 197 | changes[type] = change(); 198 | } 199 | 200 | changeset::iterator i = changes.find(type); 201 | 202 | i->second.size += node->GetSelfSize() * (added ? 1 : -1); 203 | if (added) i->second.added++; 204 | else i->second.released++; 205 | 206 | // XXX: example 207 | 208 | return; 209 | } 210 | 211 | static Handle changesetToObject(changeset & changes) 212 | { 213 | v8::HandleScope scope; 214 | Local a = Array::New(); 215 | 216 | for (changeset::iterator i = changes.begin(); i != changes.end(); i++) { 217 | Local d = Object::New(); 218 | d->Set(String::New("what"), String::New(i->first.c_str())); 219 | d->Set(String::New("size_bytes"), Integer::New(i->second.size)); 220 | d->Set(String::New("size"), String::New(mw_util::niceSize(i->second.size).c_str())); 221 | d->Set(String::New("+"), Integer::New(i->second.added)); 222 | d->Set(String::New("-"), Integer::New(i->second.released)); 223 | a->Set(a->Length(), d); 224 | } 225 | 226 | return scope.Close(a); 227 | } 228 | 229 | 230 | static v8::Handle 231 | compare(const v8::HeapSnapshot * before, const v8::HeapSnapshot * after) 232 | { 233 | v8::HandleScope scope; 234 | int s, diffBytes; 235 | 236 | Local o = Object::New(); 237 | 238 | // first let's append summary information 239 | Local b = Object::New(); 240 | b->Set(String::New("nodes"), Integer::New(before->GetNodesCount())); 241 | b->Set(String::New("time"), NODE_UNIXTIME_V8(s_startTime)); 242 | o->Set(String::New("before"), b); 243 | 244 | Local a = Object::New(); 245 | a->Set(String::New("nodes"), Integer::New(after->GetNodesCount())); 246 | a->Set(String::New("time"), NODE_UNIXTIME_V8(time(NULL))); 247 | o->Set(String::New("after"), a); 248 | 249 | // now let's get allocations by name 250 | set beforeIDs, afterIDs; 251 | s = 0; 252 | buildIDSet(&beforeIDs, before->GetRoot(), s); 253 | b->Set(String::New("size_bytes"), Integer::New(s)); 254 | b->Set(String::New("size"), String::New(mw_util::niceSize(s).c_str())); 255 | 256 | diffBytes = s; 257 | s = 0; 258 | buildIDSet(&afterIDs, after->GetRoot(), s); 259 | a->Set(String::New("size_bytes"), Integer::New(s)); 260 | a->Set(String::New("size"), String::New(mw_util::niceSize(s).c_str())); 261 | 262 | diffBytes = s - diffBytes; 263 | 264 | Local c = Object::New(); 265 | c->Set(String::New("size_bytes"), Integer::New(diffBytes)); 266 | c->Set(String::New("size"), String::New(mw_util::niceSize(diffBytes).c_str())); 267 | o->Set(String::New("change"), c); 268 | 269 | // before - after will reveal nodes released (memory freed) 270 | vector changedIDs; 271 | setDiff(beforeIDs, afterIDs, changedIDs); 272 | c->Set(String::New("freed_nodes"), Integer::New(changedIDs.size())); 273 | 274 | // here's where we'll collect all the summary information 275 | changeset changes; 276 | 277 | // for each of these nodes, let's aggregate the change information 278 | for (unsigned long i = 0; i < changedIDs.size(); i++) { 279 | const HeapGraphNode * n = before->GetNodeById(changedIDs[i]); 280 | manageChange(changes, n, false); 281 | } 282 | 283 | changedIDs.clear(); 284 | 285 | // after - before will reveal nodes added (memory allocated) 286 | setDiff(afterIDs, beforeIDs, changedIDs); 287 | 288 | c->Set(String::New("allocated_nodes"), Integer::New(changedIDs.size())); 289 | 290 | for (unsigned long i = 0; i < changedIDs.size(); i++) { 291 | const HeapGraphNode * n = after->GetNodeById(changedIDs[i]); 292 | manageChange(changes, n, true); 293 | } 294 | 295 | c->Set(String::New("details"), changesetToObject(changes)); 296 | 297 | return scope.Close(o); 298 | } 299 | 300 | v8::Handle 301 | heapdiff::HeapDiff::End( const Arguments& args ) 302 | { 303 | // take another snapshot and compare them 304 | v8::HandleScope scope; 305 | 306 | HeapDiff *t = Unwrap( args.This() ); 307 | 308 | // How shall we deal with double .end()ing? The only reasonable 309 | // approach seems to be an exception, cause nothing else makes 310 | // sense. 311 | if (t->ended) { 312 | return v8::ThrowException( 313 | v8::Exception::Error( 314 | v8::String::New("attempt to end() a HeapDiff that was " 315 | "already ended"))); 316 | } 317 | t->ended = true; 318 | 319 | s_inProgress = true; 320 | t->after = v8::HeapProfiler::TakeSnapshot(v8::String::New("")); 321 | s_inProgress = false; 322 | 323 | v8::Handle comparison = compare(t->before, t->after); 324 | // free early, free often. I mean, after all, this process we're in is 325 | // probably having memory problems. We want to help her. 326 | ((HeapSnapshot *) t->before)->Delete(); 327 | t->before = NULL; 328 | ((HeapSnapshot *) t->after)->Delete(); 329 | t->after = NULL; 330 | 331 | return scope.Close(comparison); 332 | } 333 | -------------------------------------------------------------------------------- /src/heapdiff.hh: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #ifndef __HEADDIFF_H 6 | #define __HEADDIFF_H 7 | 8 | #include 9 | #include 10 | #include 11 | 12 | namespace heapdiff 13 | { 14 | class HeapDiff : public node::ObjectWrap 15 | { 16 | public: 17 | static void Initialize ( v8::Handle target ); 18 | 19 | static v8::Handle New( const v8::Arguments& args ); 20 | static v8::Handle End( const v8::Arguments& args ); 21 | static bool InProgress(); 22 | 23 | protected: 24 | HeapDiff(); 25 | ~HeapDiff(); 26 | private: 27 | const v8::HeapSnapshot * before; 28 | const v8::HeapSnapshot * after; 29 | bool ended; 30 | }; 31 | }; 32 | 33 | #endif 34 | -------------------------------------------------------------------------------- /src/init.cc: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|do what the fuck you want to 3 | */ 4 | 5 | #include 6 | #include 7 | 8 | #include "heapdiff.hh" 9 | #include "memwatch.hh" 10 | 11 | extern "C" { 12 | void init (v8::Handle target) 13 | { 14 | v8::HandleScope scope; 15 | heapdiff::HeapDiff::Initialize(target); 16 | 17 | NODE_SET_METHOD(target, "upon_gc", memwatch::upon_gc); 18 | NODE_SET_METHOD(target, "gc", memwatch::trigger_gc); 19 | 20 | v8::V8::AddGCEpilogueCallback(memwatch::after_gc); 21 | } 22 | 23 | NODE_MODULE(memwatch, init); 24 | }; 25 | -------------------------------------------------------------------------------- /src/memwatch.cc: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #include "platformcompat.hh" 6 | #include "memwatch.hh" 7 | #include "heapdiff.hh" 8 | #include "util.hh" 9 | 10 | #include 11 | #include 12 | 13 | #include 14 | #include 15 | #include 16 | 17 | #include // for pow 18 | #include // for time 19 | 20 | using namespace v8; 21 | using namespace node; 22 | 23 | Handle g_context; 24 | Handle g_cb; 25 | 26 | struct Baton { 27 | uv_work_t req; 28 | size_t heapUsage; 29 | GCType type; 30 | GCCallbackFlags flags; 31 | }; 32 | 33 | static const unsigned int RECENT_PERIOD = 10; 34 | static const unsigned int ANCIENT_PERIOD = 120; 35 | 36 | static struct 37 | { 38 | // counts of different types of gc events 39 | unsigned int gc_full; 40 | unsigned int gc_inc; 41 | unsigned int gc_compact; 42 | 43 | // last base heap size as measured *right* after GC 44 | unsigned int last_base; 45 | 46 | // the estimated "base memory" usage of the javascript heap 47 | // over the RECENT_PERIOD number of GC runs 48 | unsigned int base_recent; 49 | 50 | // the estimated "base memory" usage of the javascript heap 51 | // over the ANCIENT_PERIOD number of GC runs 52 | unsigned int base_ancient; 53 | 54 | // the most extreme values we've seen for base heap size 55 | unsigned int base_max; 56 | unsigned int base_min; 57 | 58 | // leak detection! 59 | 60 | // the period from which this leak analysis starts 61 | time_t leak_time_start; 62 | // the base memory for the detection period 63 | time_t leak_base_start; 64 | // the number of consecutive compactions for which we've grown 65 | unsigned int consecutive_growth; 66 | } s_stats; 67 | 68 | static Handle getLeakReport(size_t heapUsage) 69 | { 70 | HandleScope scope; 71 | 72 | size_t growth = heapUsage - s_stats.leak_base_start; 73 | int now = time(NULL); 74 | int delta = now - s_stats.leak_time_start; 75 | 76 | Local leakReport = Object::New(); 77 | leakReport->Set(String::New("start"), NODE_UNIXTIME_V8(s_stats.leak_time_start)); 78 | leakReport->Set(String::New("end"), NODE_UNIXTIME_V8(now)); 79 | leakReport->Set(String::New("growth"), Integer::New(growth)); 80 | 81 | std::stringstream ss; 82 | ss << "heap growth over 5 consecutive GCs (" 83 | << mw_util::niceDelta(delta) << ") - " 84 | << mw_util::niceSize(growth / ((double) delta / (60.0 * 60.0))) << "/hr"; 85 | 86 | leakReport->Set(String::New("reason"), String::New(ss.str().c_str())); 87 | 88 | return scope.Close(leakReport); 89 | } 90 | 91 | static void AsyncMemwatchAfter(uv_work_t* request) { 92 | HandleScope scope; 93 | 94 | Baton * b = (Baton *) request->data; 95 | 96 | // do the math in C++, permanent 97 | // record the type of GC event that occured 98 | if (b->type == kGCTypeMarkSweepCompact) s_stats.gc_full++; 99 | else s_stats.gc_inc++; 100 | 101 | if ( 102 | #if NODE_VERSION_AT_LEAST(0,8,0) 103 | b->type == kGCTypeMarkSweepCompact 104 | #else 105 | b->flags == kGCCallbackFlagCompacted 106 | #endif 107 | ) { 108 | // leak detection code. has the heap usage grown? 109 | if (s_stats.last_base < b->heapUsage) { 110 | if (s_stats.consecutive_growth == 0) { 111 | s_stats.leak_time_start = time(NULL); 112 | s_stats.leak_base_start = b->heapUsage; 113 | } 114 | 115 | s_stats.consecutive_growth++; 116 | 117 | // consecutive growth over 5 GCs suggests a leak 118 | if (s_stats.consecutive_growth >= 5) { 119 | // reset to zero 120 | s_stats.consecutive_growth = 0; 121 | 122 | // emit a leak report! 123 | Handle argv[3]; 124 | argv[0] = Boolean::New(false); 125 | // the type of event to emit 126 | argv[1] = String::New("leak"); 127 | argv[2] = getLeakReport(b->heapUsage); 128 | g_cb->Call(g_context, 3, argv); 129 | } 130 | } else { 131 | s_stats.consecutive_growth = 0; 132 | } 133 | 134 | // update last_base 135 | s_stats.last_base = b->heapUsage; 136 | 137 | // update compaction count 138 | s_stats.gc_compact++; 139 | 140 | // the first ten compactions we'll use a different algorithm to 141 | // dampen out wider memory fluctuation at startup 142 | if (s_stats.gc_compact < RECENT_PERIOD) { 143 | double decay = pow(s_stats.gc_compact / RECENT_PERIOD, 2.5); 144 | decay *= s_stats.gc_compact; 145 | if (ISINF(decay) || ISNAN(decay)) decay = 0; 146 | s_stats.base_recent = ((s_stats.base_recent * decay) + 147 | s_stats.last_base) / (decay + 1); 148 | 149 | decay = pow(s_stats.gc_compact / RECENT_PERIOD, 2.4); 150 | decay *= s_stats.gc_compact; 151 | s_stats.base_ancient = ((s_stats.base_ancient * decay) + 152 | s_stats.last_base) / (1 + decay); 153 | 154 | } else { 155 | s_stats.base_recent = ((s_stats.base_recent * (RECENT_PERIOD - 1)) + 156 | s_stats.last_base) / RECENT_PERIOD; 157 | double decay = FMIN(ANCIENT_PERIOD, s_stats.gc_compact); 158 | s_stats.base_ancient = ((s_stats.base_ancient * (decay - 1)) + 159 | s_stats.last_base) / decay; 160 | } 161 | 162 | // only record min/max after 3 gcs to let initial instability settle 163 | if (s_stats.gc_compact >= 3) { 164 | if (!s_stats.base_min || s_stats.base_min > s_stats.last_base) { 165 | s_stats.base_min = s_stats.last_base; 166 | } 167 | 168 | if (!s_stats.base_max || s_stats.base_max < s_stats.last_base) { 169 | s_stats.base_max = s_stats.last_base; 170 | } 171 | } 172 | 173 | // if there are any listeners, it's time to emit! 174 | if (!g_cb.IsEmpty()) { 175 | Handle argv[3]; 176 | // magic argument to indicate to the callback all we want to know is whether there are 177 | // listeners (here we don't) 178 | argv[0] = Boolean::New(true); 179 | 180 | Handle haveListeners = g_cb->Call(g_context, 1, argv); 181 | 182 | if (haveListeners->BooleanValue()) { 183 | double ut= 0.0; 184 | if (s_stats.base_ancient) { 185 | ut = (double) ROUND(((double) (s_stats.base_recent - s_stats.base_ancient) / 186 | (double) s_stats.base_ancient) * 1000.0) / 10.0; 187 | } 188 | 189 | // ok, there are listeners, we actually must serialize and emit this stats event 190 | Local stats = Object::New(); 191 | stats->Set(String::New("num_full_gc"), Integer::New(s_stats.gc_full)); 192 | stats->Set(String::New("num_inc_gc"), Integer::New(s_stats.gc_inc)); 193 | stats->Set(String::New("heap_compactions"), Integer::New(s_stats.gc_compact)); 194 | stats->Set(String::New("usage_trend"), Number::New(ut)); 195 | stats->Set(String::New("estimated_base"), Integer::New(s_stats.base_recent)); 196 | stats->Set(String::New("current_base"), Integer::New(s_stats.last_base)); 197 | stats->Set(String::New("min"), Integer::New(s_stats.base_min)); 198 | stats->Set(String::New("max"), Integer::New(s_stats.base_max)); 199 | argv[0] = Boolean::New(false); 200 | // the type of event to emit 201 | argv[1] = String::New("stats"); 202 | argv[2] = stats; 203 | g_cb->Call(g_context, 3, argv); 204 | } 205 | } 206 | } 207 | 208 | delete b; 209 | } 210 | 211 | static void noop_work_func(uv_work_t *) { } 212 | 213 | void memwatch::after_gc(GCType type, GCCallbackFlags flags) 214 | { 215 | if (heapdiff::HeapDiff::InProgress()) return; 216 | 217 | HandleScope scope; 218 | 219 | Baton * baton = new Baton; 220 | v8::HeapStatistics hs; 221 | 222 | v8::V8::GetHeapStatistics(&hs); 223 | 224 | baton->heapUsage = hs.used_heap_size(); 225 | baton->type = type; 226 | baton->flags = flags; 227 | baton->req.data = (void *) baton; 228 | 229 | // schedule our work to run in a moment, once gc has fully completed. 230 | // 231 | // here we pass a noop work function to work around a flaw in libuv, 232 | // uv_queue_work on unix works fine, but will will crash on 233 | // windows. see: https://github.com/joyent/libuv/pull/629 234 | uv_queue_work(uv_default_loop(), &(baton->req), 235 | noop_work_func, (uv_after_work_cb)AsyncMemwatchAfter); 236 | 237 | scope.Close(Undefined()); 238 | } 239 | 240 | Handle memwatch::upon_gc(const Arguments& args) { 241 | HandleScope scope; 242 | if (args.Length() >= 1 && args[0]->IsFunction()) { 243 | g_cb = Persistent::New(Handle::Cast(args[0])); 244 | g_context = Persistent::New(Context::GetCalling()->Global()); 245 | } 246 | return scope.Close(Undefined()); 247 | } 248 | 249 | Handle memwatch::trigger_gc(const Arguments& args) { 250 | HandleScope scope; 251 | while(!V8::IdleNotification()) {}; 252 | return scope.Close(Undefined()); 253 | } 254 | -------------------------------------------------------------------------------- /src/memwatch.hh: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #ifndef __MEMWATCH_HH 6 | #define __MEMWATCH_HH 7 | 8 | #include 9 | 10 | namespace memwatch 11 | { 12 | v8::Handle upon_gc(const v8::Arguments& args); 13 | v8::Handle trigger_gc(const v8::Arguments& args); 14 | void after_gc(v8::GCType type, v8::GCCallbackFlags flags); 15 | }; 16 | 17 | #endif 18 | -------------------------------------------------------------------------------- /src/platformcompat.hh: -------------------------------------------------------------------------------- 1 | #ifndef __PLATFORMCOMPAT_H 2 | #define __PLATFORMCOMPAT_H 3 | 4 | #include // round() 5 | 6 | #if defined(_MSC_VER) 7 | #include //isinf, isnan 8 | #include //min 9 | #define ISINF _finite 10 | #define ISNAN _isnan 11 | #define FMIN __min 12 | #define ROUND(x) floor(x + 0.5) 13 | #else 14 | #define ISINF isinf 15 | #define ISNAN isnan 16 | #define FMIN fmin 17 | #define ROUND round 18 | #endif 19 | 20 | #endif -------------------------------------------------------------------------------- /src/util.cc: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #include "platformcompat.hh" 6 | #include "util.hh" 7 | 8 | #include 9 | 10 | #include // abs() 11 | 12 | std::string 13 | mw_util::niceSize(int bytes) 14 | { 15 | std::stringstream ss; 16 | 17 | if (abs(bytes) > 1024 * 1024) { 18 | ss << ROUND(bytes / (((double) 1024 * 1024 ) / 100)) / (double) 100 << " mb"; 19 | } else if (abs(bytes) > 1024) { 20 | ss << ROUND(bytes / (((double) 1024 ) / 100)) / (double) 100 << " kb"; 21 | } else { 22 | ss << bytes << " bytes"; 23 | } 24 | 25 | return ss.str(); 26 | } 27 | 28 | std::string 29 | mw_util::niceDelta(int seconds) 30 | { 31 | std::stringstream ss; 32 | 33 | if (seconds > (60*60)) { 34 | ss << (seconds / (60*60)) << "h "; 35 | seconds %= (60*60); 36 | } 37 | 38 | if (seconds > (60)) { 39 | ss << (seconds / (60)) << "m "; 40 | seconds %= (60); 41 | } 42 | 43 | ss << seconds << "s"; 44 | 45 | return ss.str(); 46 | } 47 | -------------------------------------------------------------------------------- /src/util.hh: -------------------------------------------------------------------------------- 1 | /* 2 | * 2012|lloyd|http://wtfpl.org 3 | */ 4 | 5 | #include 6 | 7 | namespace mw_util { 8 | // given a size in bytes, return a human readable representation of the 9 | // string 10 | std::string niceSize(int bytes); 11 | 12 | // given a delta in seconds, return a human redable representation 13 | std::string niceDelta(int seconds); 14 | }; 15 | 16 | 17 | -------------------------------------------------------------------------------- /tests.js: -------------------------------------------------------------------------------- 1 | const 2 | should = require('should'), 3 | memwatch = require('./'); 4 | 5 | describe('the library', function() { 6 | it('should export a couple functions', function(done) { 7 | should.exist(memwatch.gc); 8 | should.exist(memwatch.on); 9 | should.exist(memwatch.once); 10 | should.exist(memwatch.removeAllListeners); 11 | should.exist(memwatch.HeapDiff); 12 | done(); 13 | }); 14 | }); 15 | describe('calling .gc()', function() { 16 | it('should cause a stats() event to be emitted', function(done) { 17 | memwatch.once('stats', function(s) { 18 | s.should.be.a('object'); 19 | done(); 20 | }); 21 | memwatch.gc(); 22 | }); 23 | }); 24 | 25 | describe('HeapDiff', function() { 26 | it('should detect allocations', function(done) { 27 | function LeakingClass() {}; 28 | var arr = []; 29 | var hd = new memwatch.HeapDiff(); 30 | for (var i = 0; i < 100; i++) arr.push(new LeakingClass()); 31 | var diff = hd.end(); 32 | (Array.isArray(diff.change.details)).should.be.ok; 33 | diff.change.details.should.be.an.instanceOf(Array); 34 | // find the LeakingClass elem 35 | var leakingReport; 36 | diff.change.details.forEach(function(d) { 37 | if (d.what === 'LeakingClass') 38 | leakingReport = d; 39 | }); 40 | should.exist(leakingReport); 41 | ((leakingReport['+'] - leakingReport['-']) > 0).should.be.ok; 42 | done(); 43 | 44 | }); 45 | }); 46 | 47 | describe('HeapDiff', function() { 48 | it('double end should throw', function(done) { 49 | var hd = new memwatch.HeapDiff(); 50 | var arr = []; 51 | (function() { hd.end(); }).should.not.throw(); 52 | (function() { hd.end(); }).should.throw(); 53 | done(); 54 | }); 55 | }); 56 | 57 | describe('improper HeapDiff allocation', function() { 58 | it('should throw an exception', function(done) { 59 | // equivalent to "new require('memwatch').HeapDiff()" 60 | // see issue #30 61 | (function() { new (memwatch.HeapDiff()); }).should.throw(); 62 | done(); 63 | }); 64 | }); 65 | --------------------------------------------------------------------------------