├── doc ├── images │ ├── chunked-seq-realized1.png │ ├── chunked-seq-realized33.png │ ├── chunked-seq-unrealized.png │ ├── lazy-fib-seq-realized1.png │ ├── lazy-fib-seq-realized2.png │ ├── lazy-fib-seq-unrealized.png │ ├── chunked-seq-realized1-no-longs.png │ ├── lazy-fib-seq-vector-of-nthrest.png │ ├── chunked-seq-and-nthrest-20-realized20.png │ ├── chunked-seq-and-nthrest-40-realized33.png │ ├── arr10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png │ ├── vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png │ ├── my-map-macos-10.13.6-oraclejdk-1.8.0_192-clj-1.10.1.png │ ├── unboxed-vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png │ ├── strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-11.0.26-clj-1.12.0.png │ ├── strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-1.8.0_442-clj-1.12.0.png │ ├── strings-8-bit-and-not-Mac-OS-X-10.13.6-jdk-Oracle-1.8.0_192-clj-1.10.1.png │ ├── compressed-oops-disabled │ │ ├── map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png │ │ └── map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot │ ├── compressed-oops-enabled │ │ ├── map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png │ │ └── map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot │ ├── arr10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot │ ├── unboxed-vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot │ ├── strings-8-bit-and-not-Mac-OS-X-10.13.6-jdk-Oracle-1.8.0_192-clj-1.10.1.dot │ ├── lazy-fib-seq-unrealized.dot │ ├── chunked-seq-unrealized.dot │ ├── lazy-fib-seq-realized1.dot │ ├── strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-11.0.26-clj-1.12.0.dot │ ├── strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-1.8.0_442-clj-1.12.0.dot │ ├── chunked-seq-realized1-no-longs.dot │ ├── lazy-fib-seq-realized2.dot │ ├── vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot │ ├── chunked-seq-realized33.dot │ ├── chunked-seq-and-nthrest-20-realized20.dot │ ├── lazy-fib-seq-vector-of-nthrest.dot │ ├── chunked-seq-and-nthrest-40-realized33.dot │ └── chunked-seq-realized1.dot ├── README.md ├── maybe-jol-size-bug │ ├── run.sh │ └── MaybeBug.java ├── problem-dot-files │ ├── s00-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot │ ├── s09-30-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot │ ├── s09-31-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot │ ├── s00-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot │ ├── s09-30-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot │ ├── s09-31-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot │ └── README.md ├── generate-report.sh ├── generate-images.sh ├── jol-dev-ideas │ ├── README.md │ ├── jol-0-9-plus-parseInstanceIds.diff │ └── jol-0-9-plus-parseInstanceIds-plus-andy-dev-changes.diff ├── sample-classgraph-use.clj ├── README-performance.md ├── field-data-results │ ├── report-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.txt │ ├── report-Linux-4.15.0-54-jdk-Oracle-12.0.1-clj-1.10.1.txt │ └── report-Linux-4.15.0-54-jdk-Oracle-9.0.4-clj-1.10.1.txt └── README-clojure-map-size.md ├── .gitignore ├── src ├── clj │ └── cljol │ │ ├── jdk8_and_earlier.clj │ │ ├── version_info.clj │ │ ├── jdk9_and_later.clj │ │ ├── misc.clj │ │ ├── performance.clj │ │ └── object_walk.clj └── generate │ └── gen │ └── generate.clj ├── project.clj ├── scripts ├── test-cljol.sh └── summarize-warnings.py ├── test └── cljol │ └── dig_test.clj ├── deps.edn ├── LICENSE └── README.md /doc/images/chunked-seq-realized1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-realized1.png -------------------------------------------------------------------------------- /doc/images/chunked-seq-realized33.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-realized33.png -------------------------------------------------------------------------------- /doc/images/chunked-seq-unrealized.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-unrealized.png -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-realized1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/lazy-fib-seq-realized1.png -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-realized2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/lazy-fib-seq-realized2.png -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-unrealized.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/lazy-fib-seq-unrealized.png -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /target 2 | /classes 3 | /checkouts 4 | pom.xml 5 | pom.xml.asc 6 | *.jar 7 | *.class 8 | /.lein-* 9 | /.nrepl-port 10 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-realized1-no-longs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-realized1-no-longs.png -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-vector-of-nthrest.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/lazy-fib-seq-vector-of-nthrest.png -------------------------------------------------------------------------------- /doc/images/chunked-seq-and-nthrest-20-realized20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-and-nthrest-20-realized20.png -------------------------------------------------------------------------------- /doc/images/chunked-seq-and-nthrest-40-realized33.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/chunked-seq-and-nthrest-40-realized33.png -------------------------------------------------------------------------------- /doc/images/arr10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/arr10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/my-map-macos-10.13.6-oraclejdk-1.8.0_192-clj-1.10.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/my-map-macos-10.13.6-oraclejdk-1.8.0_192-clj-1.10.1.png -------------------------------------------------------------------------------- /doc/images/unboxed-vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/unboxed-vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-11.0.26-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-11.0.26-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-1.8.0_442-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-1.8.0_442-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Mac-OS-X-10.13.6-jdk-Oracle-1.8.0_192-clj-1.10.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/strings-8-bit-and-not-Mac-OS-X-10.13.6-jdk-Oracle-1.8.0_192-clj-1.10.1.png -------------------------------------------------------------------------------- /doc/images/compressed-oops-disabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/compressed-oops-disabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png -------------------------------------------------------------------------------- /doc/images/compressed-oops-enabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jafingerhut/cljol/HEAD/doc/images/compressed-oops-enabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.png -------------------------------------------------------------------------------- /src/clj/cljol/jdk8_and_earlier.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.jdk8-and-earlier 2 | (:import (java.lang.reflect Field))) 3 | 4 | (set! *warn-on-reflection* true) 5 | 6 | 7 | (defn obj-field-value [obj ^Field fld _inaccessible-field-val-sentinel] 8 | (. fld setAccessible true) 9 | (.get fld obj)) 10 | -------------------------------------------------------------------------------- /doc/README.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | See the [gallery](README-gallery.md) for examples of figures created 4 | by this library that demonstrate aspects of the Java VM or Clojure's 5 | implementation that I find interesting. 6 | 7 | [README-dev.md](README-dev.md) contains a few notes on the 8 | implementation of the `cljol` library itself. 9 | -------------------------------------------------------------------------------- /doc/images/arr10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18831105472" ["reachable-only-from"=18831105472,"scc-num-nodes"=1,style="filled","my-unique-total-size"=96,"my-unique-num-reachable-nodes"=1,label="96 bytes 4 | 1 object, 96 bytes reachable 5 | this object in no reference cycles 6 | array of 10 long 7 | 8 | [0 1 2 3 4 5 6 7 8 9]",shape="box"]; 9 | } -------------------------------------------------------------------------------- /doc/maybe-jol-size-bug/run.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | set -ex 4 | 5 | uname -a 6 | java -version 7 | 8 | M2="${HOME}/.m2/repository" 9 | JOL_CORE_0_9_JAR="${M2}/org/openjdk/jol/jol-core/0.9/jol-core-0.9.jar" 10 | 11 | javac -classpath "${JOL_CORE_0_9_JAR}" MaybeBug.java 12 | #echo "${PWD}" 13 | java -classpath "${PWD}:${JOL_CORE_0_9_JAR}" MaybeBug 14 | #sudo java -classpath "${PWD}:${JOL_CORE_0_9_JAR}" MaybeBug 15 | -------------------------------------------------------------------------------- /doc/problem-dot-files/s00-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "31787676800" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | "]; 9 | "31787676992" [shape=box,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | array of 1 char 12 | 13 | [\\]"]; 14 | "31787676800" -> "31787676992" ["field-name"=value,label=value]; 15 | } -------------------------------------------------------------------------------- /doc/problem-dot-files/s09-30-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "31809961472" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | ￾"]; 9 | "31809961664" [shape=box,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | array of 1 char 12 | 13 | [\\￾]"]; 14 | "31809961472" -> "31809961664" ["field-name"=value,label=value]; 15 | } -------------------------------------------------------------------------------- /doc/problem-dot-files/s09-31-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "31811383936" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | ￿"]; 9 | "31811384128" [shape=box,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | array of 1 char 12 | 13 | [\\￿]"]; 14 | "31811383936" -> "31811384128" ["field-name"=value,label=value]; 15 | } -------------------------------------------------------------------------------- /doc/problem-dot-files/s00-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "26560179712" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | 20: coder (byte) 0 9 | "]; 10 | "26560179904" [shape=box,label="24 bytes 11 | 1 object, 24 bytes reachable 12 | array of 1 byte 13 | 14 | [0]"]; 15 | "26560179712" -> "26560179904" ["field-name"=value,label=value]; 16 | } -------------------------------------------------------------------------------- /doc/problem-dot-files/s09-30-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "26490815168" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | 20: coder (byte) 1 9 | ￾"]; 10 | "26490815552" [shape=box,label="24 bytes 11 | 1 object, 24 bytes reachable 12 | array of 2 byte 13 | 14 | [-2 -1]"]; 15 | "26490815168" -> "26490815552" ["field-name"=value,label=value]; 16 | } -------------------------------------------------------------------------------- /doc/problem-dot-files/s09-31-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout=dot,rankdir=LR]; 3 | "26475463296" [shape=box,label="24 bytes 4 | 2 objects, 48 bytes reachable 5 | j.l.String 6 | 12: value (ref) -> 7 | 16: hash (int) 0 8 | 20: coder (byte) 1 9 | ￿"]; 10 | "26475463680" [shape=box,label="24 bytes 11 | 1 object, 24 bytes reachable 12 | array of 2 byte 13 | 14 | [-1 -1]"]; 15 | "26475463296" -> "26475463680" ["field-name"=value,label=value]; 16 | } -------------------------------------------------------------------------------- /doc/generate-report.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | # I do not understand why yet, but for this combination of versions: 4 | # report-Mac-OS-X-10.13.6-jdk-AdoptOpenJDK-11.0.3-clj-1.10.1.txt 5 | 6 | # I get no size mismatches in 'inst-size-diffs' part of output when I 7 | # do not use the :priv2 alias, but a size mismatch on every class when 8 | # I use the :priv2 alias. 9 | 10 | # The :priv2 alias is defined this way when I observed this: 11 | # :priv2 {:jvm-opts ["-Djdk.attach.allowAttachSelf"]} 12 | 13 | #clojure -A:classgraph:jamm -e "(require,'[cljol.reflection-test-helpers,:as,t]),(t/report)" 14 | clojure -A:classgraph:jamm:priv -e "(require,'[cljol.reflection-test-helpers,:as,t]),(t/report)" 15 | #clojure -A:classgraph:jamm:priv2 -e "(require,'[cljol.reflection-test-helpers,:as,t]),(t/report)" 16 | -------------------------------------------------------------------------------- /doc/generate-images.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | # Remember the current directory when the script was started: 4 | INSTALL_DIR="${PWD}" 5 | 6 | # Found here: 7 | # 8 | # https://stackoverflow.com/questions/4774054/reliable-way-for-a-bash-script-to-get-the-full-path-to-itself/20265654 9 | # 10 | # I tested it on Ubuntu 18.04 Linux and macOS 10.13.6 and it worked 11 | # for both, so at least for now I am going for it. 12 | SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )" 13 | #echo ":${SCRIPTPATH}:" 14 | #exit 0 15 | 16 | OUTPUT_DIR="${SCRIPTPATH}/tryout-images" 17 | mkdir -p "${OUTPUT_DIR}" 18 | 19 | ALIAS="-A:generate" 20 | #ALIAS="${ALIAS}:disablecompressedoops" 21 | clojure $ALIAS -m gen.generate "${OUTPUT_DIR}" 22 | 23 | cd "${OUTPUT_DIR}" 24 | for j in *.dot 25 | do 26 | echo "Generating images from $j ..." 27 | for format in png pdf 28 | do 29 | dot -T${format} "$j" -o `basename $j .dot`.${format} 30 | done 31 | done 32 | -------------------------------------------------------------------------------- /src/clj/cljol/version_info.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.version-info 2 | (:require [clojure.string :as str])) 3 | 4 | (set! *warn-on-reflection* true) 5 | 6 | 7 | (defn clean-os-version [s] 8 | (str/replace s #"-generic$" "")) 9 | 10 | 11 | (defn clean-vendor [s] 12 | (str/replace s #" Corporation$" "")) 13 | 14 | 15 | (defn calc-version-data [] 16 | (let [props (into (sorted-map) (System/getProperties)) 17 | os-desc (str (props "os.name") 18 | "-" (clean-os-version (props "os.version"))) 19 | jvm-desc (str "jdk-" (clean-vendor (props "java.vendor")) 20 | "-" (props "java.version")) 21 | clj-desc (str "clj-" (clojure-version))] 22 | {:os-desc os-desc 23 | :jvm-desc jvm-desc 24 | :clj-desc clj-desc 25 | :stack-desc (str/replace (str os-desc "-" jvm-desc "-" clj-desc) 26 | " " "-")})) 27 | 28 | 29 | (def version-data (delay (calc-version-data))) 30 | -------------------------------------------------------------------------------- /project.clj: -------------------------------------------------------------------------------- 1 | (defproject cljol "0.3.0" 2 | :description "Analyze Java objects in memory using the Java Object Layout library" 3 | :url "http://github.com/jafingerhut/cljol" 4 | :license {:name "Eclipse Public License" 5 | :url "http://www.eclipse.org/legal/epl-v10.html"} 6 | :source-paths ["src/clj"] 7 | :dependencies [[org.clojure/clojure "1.10.1"] 8 | [com.fingerhutpress.cljol_jvm_support/cljol_jvm_support "1.0"] 9 | [ubergraph "0.8.2" 10 | :exclusions [tailrecursion/cljs-priority-map]]] 11 | :test-selectors {:perf-focus :perf-focus} 12 | :profiles {:test {:dependencies [[medley "1.2.0"] 13 | [criterium "0.4.5"]]} 14 | :uberlocal {:dependencies [[ubergraph "0.5.4-andy-mods"]]} 15 | :master {:dependencies [[org.clojure/clojure "1.11.0-master-SNAPSHOT"]]} 16 | } 17 | :jvm-opts ^:replace [ 18 | ;;"-XX:+PrintGC" 19 | ;;"-XX:+PrintGCDetails" 20 | ;;"-Djol.tryWithSudo=true" 21 | ] 22 | ) 23 | -------------------------------------------------------------------------------- /src/clj/cljol/jdk9_and_later.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.jdk9-and-later 2 | (:import (java.lang.reflect Field InaccessibleObjectException))) 3 | 4 | (set! *warn-on-reflection* true) 5 | 6 | (def packages-to-consider-opening (atom #{})) 7 | 8 | (defn get-exc-message [exc] 9 | (-> exc Throwable->map :via (nth 0) :message)) 10 | 11 | (defn get-package-info [msg] 12 | (let [m (re-find #"accessible: module (.*) does not \"opens (\S+)\" to unnamed" msg)] 13 | (if m 14 | {:module-name (m 1), :package-name (m 2)} 15 | nil))) 16 | 17 | (defn obj-field-value [obj ^Field fld inaccessible-field-val-sentinel] 18 | (try 19 | (. fld setAccessible true) 20 | (.get fld obj) 21 | (catch InaccessibleObjectException e 22 | (let [msg (get-exc-message e) 23 | package-info (get-package-info msg)] 24 | (if package-info 25 | (let [[old new] (swap-vals! packages-to-consider-opening 26 | conj package-info)] 27 | (when (not= old new) 28 | (printf "ERROR: Add these JVM command line options to avoid errors determining field values of objects: --add-opens %s/%s=ALL-UNNAMED\n" 29 | (:module-name package-info) 30 | (:package-name package-info)))) 31 | (printf "ERROR: Could not find package name in exception message: %s" 32 | msg))) 33 | inaccessible-field-val-sentinel))) 34 | -------------------------------------------------------------------------------- /doc/jol-dev-ideas/README.md: -------------------------------------------------------------------------------- 1 | These files contain 2 very similar patches, the first of which I 2 | proposed as some relatively small additions to the existing JOL 3 | project maintainers as a patch for consideration. I want to leave 4 | that file as is for a while, to give them time to respond to the idea, 5 | if they are interested. 6 | 7 | * jol-0-9-plus-parseInstanceIds.diff 8 | 9 | This is a minor variation of the above, with a few more changes that 10 | make it easier for me to test the changes on my dev machine: 11 | 12 | * jol-0-9-plus-parseInstanceIds-plus-andy-dev-changes.diff 13 | 14 | I have made some slightly bigger changes, that adds '2' variants of 4 15 | different classes, starting with GraphPathRecord2 that has fewer 16 | fields, only the ones I actually use in the cljol project, and then 17 | working up to GraphLayout2.java which has the IdentityHashMap return 18 | value, and no addresses 'map' return value at all, because it is not 19 | really necessary for `cljol`. 20 | 21 | These source files are checked into the Github project 22 | [cljol-jvm-support](https://github.com/jafingerhut/cljol-jvm-support) 23 | and [a JAR 24 | file](https://clojars.org/com.fingerhutpress.cljol_jvm_support/cljol_jvm_support) 25 | has been deployed to Clojars.org. 26 | 27 | 28 | Notes on building the jol-core library from Java source code: 29 | 30 | If you do not have javadoc installed in a system, you can still build 31 | and test JOL with this command: 32 | 33 | ```bash 34 | $ mvn -Dmaven.javadoc.skip=true clean install 35 | ``` 36 | -------------------------------------------------------------------------------- /doc/images/unboxed-vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "17779777088" ["reachable-only-from"=18831109952,"scc-num-nodes"=1,label="144 bytes 4 | 1 object, 144 bytes reachable 5 | this object in no reference cycles 6 | array of 32 j.l.Object 7 | 8 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 9 | "18831109952" ["reachable-only-from"=18831109952,"scc-num-nodes"=1,style="filled","my-unique-total-size"=320,"my-unique-num-reachable-nodes"=5,label="40 bytes 10 | 5 objects, 320 bytes reachable 11 | this object in no reference cycles 12 | clojure.core.Vec 13 | 12: cnt (int) 10 14 | 16: shift (int) 5 15 | 20: am (ref) -> 16 | 24: root (ref) -> 17 | 28: tail (ref) -> 18 | 32: _meta (ref) nil 19 | clojure.core.Vec@9ebadac6",shape="box"]; 20 | "17785018368" ["reachable-only-from"=18831109952,"scc-num-nodes"=1,label="16 bytes 21 | 1 object, 16 bytes reachable 22 | this object in no reference cycles 23 | clojure.core$reify__8419 24 | 12: __meta (ref) nil 25 | val maybe realizes if str'ed",shape="box"]; 26 | "18831110272" ["reachable-only-from"=18831109952,"scc-num-nodes"=1,label="96 bytes 27 | 1 object, 96 bytes reachable 28 | this object in no reference cycles 29 | array of 10 long 30 | 31 | [0 1 2 3 4 5 6 7 8 9]",shape="box"]; 32 | "17779776896" ["reachable-only-from"=18831109952,"scc-num-nodes"=1,label="24 bytes 33 | 2 objects, 168 bytes reachable 34 | this object in no reference cycles 35 | clojure.core.VecNode 36 | 12: edit (ref) nil 37 | 16: arr (ref) -> 38 | val maybe realizes if str'ed",shape="box"]; 39 | "18831109952" -> "17785018368" ["field-name"="am",label="am"]; 40 | "18831109952" -> "17779776896" ["field-name"="root",label="root"]; 41 | "18831109952" -> "18831110272" ["field-name"="tail",label="tail"]; 42 | "17779776896" -> "17779777088" ["field-name"="arr",label="arr"]; 43 | } -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Mac-OS-X-10.13.6-jdk-Oracle-1.8.0_192-clj-1.10.1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph[dpi=100, rankdir=LR] 3 | node[fontname="Monospace"] 4 | edge[fontname="Monospace"] 5 | 6 | node686[shape="box", label="80 bytes 7 | array of 30 char 8 | [\\f \\ሴ \\o \\d \\space \\h \\a \\s \\space \\n \\o \\n \\- \\8 ..."] 9 | node687[shape="box", label="24 bytes 10 | j.l.String 11 | 12: value (ref) -> 12 | 16: hash (int) 511352126 13 | food has only 8-bit characters"] 14 | node688[shape="box", label="40 bytes 15 | c.l.PersistentVector 16 | 12: _hash (int) 0 17 | 16: _hasheq (int) 0 18 | 20: cnt (int) 2 19 | 24: shift (int) 5 20 | 28: root (ref) -> 21 | 32: tail (ref) -> 22 | 36: _meta (ref) nil 23 | [\"food has only 8-bit characters\" \"fሴod has non-8- ..."] 24 | node689[shape="box", label="24 bytes 25 | array of 2 j.l.Object 26 | [\"food has only 8-bit characters\" \"fሴod has non-8- ..."] 27 | node690[shape="box", label="80 bytes 28 | array of 30 char 29 | [\\f \\o \\o \\d \\space \\h \\a \\s \\space \\o \\n \\l \\y \\s ..."] 30 | node691[shape="box", label="24 bytes 31 | c.l.PersistentVector$Node 32 | 12: edit (ref) -> 33 | 16: array (ref) -> 34 | clojure.lang.PersistentVector$Node@61e3a1fd"] 35 | node692[shape="box", label="24 bytes 36 | j.l.String 37 | 12: value (ref) -> 38 | 16: hash (int) -938149940 39 | fሴod has non-8-bit characters!"] 40 | node693[shape="box", label="144 bytes 41 | array of 32 j.l.Object 42 | [nil nil nil nil nil nil nil nil nil nil nil nil n ..."] 43 | node694[shape="box", label="16 bytes 44 | j.u.concurrent.atomic.AtomicReference 45 | 12: value (ref) nil 46 | null"] 47 | node687 -> node690[label="value"] 48 | node688 -> node691[label="root"] 49 | node688 -> node689[label="tail"] 50 | node689 -> node687[label="[0]"] 51 | node689 -> node692[label="[1]"] 52 | node691 -> node694[label="edit"] 53 | node691 -> node693[label="array"] 54 | node692 -> node686[label="value"] 55 | } 56 | -------------------------------------------------------------------------------- /doc/problem-dot-files/README.md: -------------------------------------------------------------------------------- 1 | The following files were found to cause errors with the following 2 | `dot` command: 3 | 4 | ```bash 5 | j=name of the file 6 | k=`basename $F .dot`.pdf 7 | dot -Tpdf $F -o $G 8 | ``` 9 | 10 | with this version of dot shown below, installed on the given version 11 | of Ubuntu desktop Linux via the command `sudo apt-get install 12 | graphviz`: 13 | 14 | ``` 15 | $ dot -V 16 | dot - graphviz version 2.40.1 (20161225.0304) 17 | 18 | $ lsb_release -a 19 | No LSB modules are available. 20 | Distributor ID: Ubuntu 21 | Description: Ubuntu 18.04.2 LTS 22 | Release: 18.04 23 | Codename: bionic 24 | ``` 25 | 26 | ``` 27 | s00-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot 28 | s00-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot 29 | s09-30-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot 30 | s09-30-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot 31 | s09-31-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.dot 32 | s09-31-Linux-4.15.0-54-jdk-Oracle-1.8.0_192-clj-1.10.1.dot 33 | ``` 34 | 35 | 36 | With the version of dot below, installed on the given version of 37 | Ubuntu desktop Linux, only the first two files give an error when 38 | running dot on them. The other 4 cause no errors, and seem to 39 | generate PDF files. 40 | 41 | ``` 42 | $ dot -V 43 | dot - graphviz version 2.38.0 (20140413.2041) 44 | 45 | $ lsb_release -a 46 | No LSB modules are available. 47 | Distributor ID: Ubuntu 48 | Description: Ubuntu 16.04.6 LTS 49 | Release: 16.04 50 | Codename: xenial 51 | ``` 52 | 53 | Running on macOS 10.13.6, I get an error running the version of dot 54 | shown below on all 6 files. That version of dot was installed via 55 | `sudo port install graphviz`. 56 | 57 | ```bash 58 | $ dot -V 59 | dot - graphviz version 2.40.1 (20161225.0304) 60 | 61 | $ uname -a 62 | Darwin JAFINGER-M-W0SR 17.7.0 Darwin Kernel Version 17.7.0: Wed Apr 24 21:17:24 PDT 2019; root:xnu-4570.71.45~1/RELEASE_X86_64 x86_64 63 | ``` 64 | -------------------------------------------------------------------------------- /src/clj/cljol/misc.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.misc) 2 | 3 | 4 | ;; I wrote these at one point during cljol development, and probably 5 | ;; used them at some point, but they do not appear to be used now. 6 | ;; Keeping them around for a bit in case I find a use for them. 7 | 8 | 9 | (defn find-first-or-last 10 | "Find and return the first item of coll such that (pred item) returns 11 | logical true. If there is no such item, but there is at least one 12 | item in coll, return the last item. If coll is empty, return the 13 | not-found value." 14 | [pred coll not-found] 15 | (letfn [(step [s n] 16 | (let [f (first s)] 17 | (if (pred f) 18 | [f n] 19 | (if-let [nxt (next s)] 20 | (recur nxt (inc n)) 21 | [f n]))))] 22 | (if-let [s (seq coll)] 23 | (step s 1) 24 | [not-found -1]))) 25 | 26 | (comment 27 | (= [6 3] (find-first-or-last #(>= % 5) [2 4 6 8] :not-found)) 28 | (= [8 4] (find-first-or-last #(>= % 10) [2 4 6 8] :not-found)) 29 | (= [:not-found -1] (find-first-or-last #(>= % 10) [] :not-found)) 30 | ) 31 | 32 | 33 | (defn last-and-count 34 | "Return a vector of 2 elements, taking linear time in the size of 35 | coll, and traversing through its elements only once. The first 36 | element of the returned vector is the last item in coll, or nil if 37 | coll is empty. The second element of the returned vector is the 38 | number of elements in coll, 0 if coll is empty." 39 | [coll] 40 | (letfn [(step [s count] 41 | (if (next s) 42 | (recur (next s) (inc count)) 43 | [(first s) count]))] 44 | (if-let [s (seq coll)] 45 | (step s 1) 46 | [nil 0]))) 47 | 48 | (comment 49 | (last-and-count [:a :b :c]) 50 | ;; => [:c 3] 51 | (last-and-count (take 0 (range 100))) 52 | ;; => [nil 0] 53 | (last-and-count (take 1 (range 100))) 54 | ;; => [0 1] 55 | (last-and-count (take 5 (range 100))) 56 | ;; => [4 5] 57 | ) 58 | -------------------------------------------------------------------------------- /doc/sample-classgraph-use.clj: -------------------------------------------------------------------------------- 1 | ;; Short sample use of classgraph library from Clojure 2 | 3 | ;; https://github.com/classgraph/classgraph 4 | 5 | ;; I do not know why, but when I tried this with AdoptOpenJDK 8 and 6 | ;; Zulu JDK 11 on Ubuntu 18.04 Linux, the class java.lang.Object was 7 | ;; not included in the thousands of classes returned, so it must not 8 | ;; be a list of all classes, but only those that can be found by the 9 | ;; way I call the method. Reading more of the classgraph library 10 | ;; documentation will probably make it clear why, and what might be 11 | ;; done to get a more complete list. 12 | 13 | 14 | ;; Sample command to invoke using Clojure CLI tools from the shell: 15 | 16 | ;; clojure -Sdeps '{:deps {io.github.classgraph/classgraph {:mvn/version "4.8.90"}}}' 17 | 18 | (import '(io.github.classgraph ClassGraph ClassInfo)) 19 | (require '[clojure.reflect :as refl]) 20 | 21 | (defn all-class-infos [] 22 | (let [scan-result (.. (ClassGraph.) enableAllInfo scan)] 23 | (into {} (.getAllClassesAsMap scan-result)))) 24 | 25 | ;; Classes found by the classgraph library are not necessarily loaded 26 | ;; at the time they are found. The loadClass method tries attempts to 27 | ;; load a class, if it is not already. 28 | 29 | (defn try-load-class [class-info] 30 | (try 31 | {:err nil :klass (. class-info loadClass)} 32 | (catch Exception e 33 | {:err e}))) 34 | 35 | (defn get-class [class-infos class-name-string] 36 | (let [class-info (get class-infos class-name-string) 37 | {:keys [err klass]} (try-load-class class-info)] 38 | (if err 39 | nil 40 | klass))) 41 | 42 | (def x (all-class-infos)) 43 | 44 | (count x) 45 | 46 | ;; Print some info about a couple of classes, chosen arbitrarily 47 | (pprint (nth (seq x) 0)) 48 | (pprint (nth (seq x) 100)) 49 | 50 | (refl/type-reflect (get-class x "clojure.core$when_first")) 51 | (refl/type-reflect (get-class x "clojure.lang.PersistentVector")) 52 | (refl/type-reflect (get-class x "java.lang.Object")) 53 | 54 | ;; Show all class names as strings, sorted 55 | (->> (keys x) 56 | sort 57 | pprint) 58 | -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-unrealized.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18757450240" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,style="filled","my-unique-total-size"=152,"my-unique-num-reachable-nodes"=6,label="32 bytes 4 | 6 objects, 152 bytes reachable 5 | this object in no reference cycles 6 | c.l.LazySeq 7 | 12: _meta (ref) nil 8 | 16: fn (ref) -> 9 | 20: sv (ref) nil 10 | 24: s (ref) nil 11 | 28: lock (ref) -> 12 | val maybe realizes if str'ed",shape="box"]; 13 | "18757450688" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,label="16 bytes 14 | 2 objects, 48 bytes reachable 15 | this object in no reference cycles 16 | j.u.c.locks.ReentrantLock 17 | 12: sync (ref) -> 18 | val maybe realizes if str'ed",shape="box"]; 19 | "34349647488" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,label="24 bytes 20 | 1 object, 24 bytes reachable 21 | this object in no reference cycles 22 | j.l.Long 23 | 16: value (long) 0 24 | 0",shape="box"]; 25 | "34349647680" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,label="24 bytes 26 | 1 object, 24 bytes reachable 27 | this object in no reference cycles 28 | j.l.Long 29 | 16: value (long) 1 30 | 1",shape="box"]; 31 | "18757450496" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,label="24 bytes 32 | 3 objects, 72 bytes reachable 33 | this object in no reference cycles 34 | user$fib_fn$fn__10167 35 | 12: __methodImplCache (ref) nil 36 | 16: a (ref) -> 37 | 20: b (ref) -> 38 | val maybe realizes if str'ed",shape="box"]; 39 | "18757450816" ["reachable-only-from"=18757450240,"scc-num-nodes"=1,label="32 bytes 40 | 1 object, 32 bytes reachable 41 | this object in no reference cycles 42 | j.u.c.locks.ReentrantLock$NonfairSync 43 | 12: exclusiveOwnerThread (ref) nil 44 | 16: state (int) 0 45 | 20: head (ref) nil 46 | 24: tail (ref) nil 47 | val maybe realizes if str'ed",shape="box"]; 48 | "18757450240" -> "18757450496" ["field-name"="fn",label="fn"]; 49 | "18757450240" -> "18757450688" ["field-name"="lock",label="lock"]; 50 | "18757450688" -> "18757450816" ["field-name"="sync",label="sync"]; 51 | "18757450496" -> "34349647488" ["field-name"="a",label="a"]; 52 | "18757450496" -> "34349647680" ["field-name"="b",label="b"]; 53 | } -------------------------------------------------------------------------------- /scripts/test-cljol.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | JDK_INSTALL_DIR="$HOME/p4-docs/jdks" 4 | 5 | opts="$1" 6 | 7 | CLJOL_DEP='{:deps {cljol/cljol {:git/url "https://github.com/jafingerhut/cljol" :sha "dc17a8e02f5abf7aacf6c1962c627fe7b19993d0"}}}' 8 | EVAL_EXPRS1="(require '[cljol.dig9 :as d]) (def my-map {\"a\" 1 \"foobar\" 3.5}) (d/write-dot-file [my-map] \"my-map.dot\") (System/exit 0)" 9 | EVAL_EXPRS2="(require '[cljol.dig9 :as d]) (def my-map {\"a\" 1 \"foobar\" 3.5 \"b\" (java.util.TreeSet. [])}) (d/write-dot-file [my-map] \"my-map.dot\") (System/exit 0)" 10 | ADD_OPENS_JAVA_LANG="-J--add-opens -Jjava.base/java.lang=ALL-UNNAMED" 11 | ADD_OPENS_JAVA_UTIL="-J--add-opens -Jjava.base/java.util=ALL-UNNAMED" 12 | 13 | for ver in 8 11 16 17 18 19 20 21 22 23 24 14 | do 15 | echo "" 16 | if [ -e ${JDK_INSTALL_DIR}/setup-jdk-${ver}.bash ] 17 | then 18 | source ${JDK_INSTALL_DIR}/setup-jdk-${ver}.bash 19 | else 20 | continue 21 | fi 22 | java -version | head -n 1 23 | case ${opts} in 24 | 1) 25 | set -x 26 | clojure -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS1}" 27 | set +x 28 | ;; 29 | 2) 30 | set -x 31 | clojure -J-Djdk.attach.allowAttachSelf -J-Djol.tryWithSudo=true -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS1}" 32 | set +x 33 | ;; 34 | 3) 35 | set -x 36 | clojure ${ADD_OPENS_JAVA_LANG} -J-Djdk.attach.allowAttachSelf -J-Djol.tryWithSudo=true -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS1}" 37 | set +x 38 | ;; 39 | 4) 40 | set -x 41 | clojure ${ADD_OPENS_JAVA_LANG} -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS1}" 42 | set +x 43 | ;; 44 | 5) 45 | set -x 46 | clojure ${ADD_OPENS_JAVA_LANG} -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS2}" 47 | set +x 48 | ;; 49 | 6) 50 | set -x 51 | clojure ${ADD_OPENS_JAVA_LANG} ${ADD_OPENS_JAVA_UTIL} -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS2}" 52 | set +x 53 | ;; 54 | 7) 55 | set -x 56 | clojure ${ADD_OPENS_JAVA_LANG} -J-Djdk.attach.allowAttachSelf -J-Djol.tryWithSudo=true -J-XX:+EnableDynamicAgentLoading -Sdeps "${CLJOL_DEP}" -M --eval "${EVAL_EXPRS1}" 57 | set +x 58 | ;; 59 | *) 60 | 1>&2 echo "Unknown command line option: ${opts}" 61 | exit 1 62 | ;; 63 | esac 64 | set +x 65 | mv my-map.dot my-map-jdk-${ver}.dot 66 | done 67 | 68 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-unrealized.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18128863040" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,label="56 bytes 4 | 1 object, 56 bytes reachable 5 | this object in no reference cycles 6 | c.l.LongRange 7 | 12: _meta (ref) nil 8 | 16: _hash (int) 0 9 | 20: _hasheq (int) 0 10 | 24: start (long) 0 11 | 32: end (long) 1000 12 | 40: step (long) 1 13 | 48: count (int) 1000 14 | val maybe realizes if str'ed",shape="box"]; 15 | "18128862592" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,style="filled","my-unique-total-size"=176,"my-unique-num-reachable-nodes"=6,label="32 bytes 16 | 6 objects, 176 bytes reachable 17 | this object in no reference cycles 18 | c.l.LazySeq 19 | 12: _meta (ref) nil 20 | 16: fn (ref) -> 21 | 20: sv (ref) nil 22 | 24: s (ref) nil 23 | 28: lock (ref) -> 24 | val maybe realizes if str'ed",shape="box"]; 25 | "18128863616" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,label="16 bytes 26 | 2 objects, 48 bytes reachable 27 | this object in no reference cycles 28 | j.u.c.locks.ReentrantLock 29 | 12: sync (ref) -> 30 | val maybe realizes if str'ed",shape="box"]; 31 | "18128862848" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,label="24 bytes 32 | 3 objects, 96 bytes reachable 33 | this object in no reference cycles 34 | clojure.core$map$fn__5954 35 | 12: __methodImplCache (ref) nil 36 | 16: coll (ref) -> 37 | 20: f (ref) -> 38 | val maybe realizes if str'ed",shape="box"]; 39 | "18128863488" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,label="16 bytes 40 | 1 object, 16 bytes reachable 41 | this object in no reference cycles 42 | user$fn__10278 43 | 12: __methodImplCache (ref) nil 44 | val maybe realizes if str'ed",shape="box"]; 45 | "18128863744" ["reachable-only-from"=18128862592,"scc-num-nodes"=1,label="32 bytes 46 | 1 object, 32 bytes reachable 47 | this object in no reference cycles 48 | j.u.c.locks.ReentrantLock$NonfairSync 49 | 12: exclusiveOwnerThread (ref) nil 50 | 16: state (int) 0 51 | 20: head (ref) nil 52 | 24: tail (ref) nil 53 | val maybe realizes if str'ed",shape="box"]; 54 | "18128862592" -> "18128862848" ["field-name"="fn",label="fn"]; 55 | "18128862592" -> "18128863616" ["field-name"="lock",label="lock"]; 56 | "18128863616" -> "18128863744" ["field-name"="sync",label="sync"]; 57 | "18128862848" -> "18128863040" ["field-name"="coll",label="coll"]; 58 | "18128862848" -> "18128863488" ["field-name"="f",label="f"]; 59 | } -------------------------------------------------------------------------------- /src/clj/cljol/performance.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.performance 2 | (:refer-clojure :exclude [time]) 3 | (:import (java.lang.management ManagementFactory GarbageCollectorMXBean))) 4 | 5 | (set! *warn-on-reflection* true) 6 | 7 | 8 | (defn gc-collection-stats [] 9 | (let [mxbeans (ManagementFactory/getGarbageCollectorMXBeans)] 10 | (apply merge-with + 11 | (for [^GarbageCollectorMXBean gc mxbeans] 12 | {:gc-collection-count (max 0 (. gc getCollectionCount)) 13 | :gc-collection-time-msec (max 0 (. gc getCollectionTime))})))) 14 | 15 | 16 | (defn gc-collection-stats-delta [start-stats end-stats] 17 | (merge-with - end-stats start-stats)) 18 | 19 | 20 | (defn print-perf-stats [perf-stats] 21 | (let [{:keys [time-nsec gc-stats]} perf-stats 22 | {:keys [gc-collection-count gc-collection-time-msec]} gc-stats 23 | time-msec (/ time-nsec 1000000.0)] 24 | (println (if (< time-msec 1) (str time-msec) (format "%.1f" time-msec)) "msec," 25 | gc-collection-count "gc-count," 26 | gc-collection-time-msec "gc-time-msec"))) 27 | 28 | 29 | (defmacro time [expr] 30 | `(let [start-nsec# (. System (nanoTime)) 31 | start-gc-stats# (gc-collection-stats) 32 | ret# ~expr] 33 | {:time-nsec (- (. System (nanoTime)) start-nsec#) 34 | :gc-stats (gc-collection-stats-delta start-gc-stats# 35 | (gc-collection-stats)) 36 | :ret ret#})) 37 | 38 | 39 | (defmacro time-record-results [expr a] 40 | `(let [ret# (time ~expr)] 41 | (swap! ~a conj (dissoc ret# :ret)) 42 | ret#)) 43 | 44 | 45 | (defn add-times 46 | [{time-nsec1 :time-nsec 47 | {count1 :gc-collection-count 48 | time-msec1 :gc-collection-time-msec} :gc-stats} 49 | {time-nsec2 :time-nsec 50 | {count2 :gc-collection-count 51 | time-msec2 :gc-collection-time-msec} :gc-stats}] 52 | {:time-nsec (+ time-nsec1 time-nsec2) 53 | :gc-stats {:gc-collection-count (+ count1 count2) 54 | :gc-collection-time-msec (+ time-msec1 time-msec2)}}) 55 | 56 | (defn subtract-times 57 | [{time-nsec1 :time-nsec 58 | {count1 :gc-collection-count 59 | time-msec1 :gc-collection-time-msec} :gc-stats} 60 | {time-nsec2 :time-nsec 61 | {count2 :gc-collection-count 62 | time-msec2 :gc-collection-time-msec} :gc-stats}] 63 | {:time-nsec (- time-nsec1 time-nsec2) 64 | :gc-stats {:gc-collection-count (- count1 count2) 65 | :gc-collection-time-msec (- time-msec1 time-msec2)}}) 66 | -------------------------------------------------------------------------------- /doc/images/compressed-oops-enabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18497576576" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 4 | 1 object, 24 bytes reachable 5 | this object in no reference cycles 6 | j.l.Double 7 | 16: value (double) 3.5 8 | 3.5",shape="box"]; 9 | "18497576768" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 10 | 2 objects, 48 bytes reachable 11 | this object in no reference cycles 12 | j.l.String 13 | 12: hash (int) -1268878963 14 | 16: coder (byte) 0 15 | 17: hashIsZero (boolean) false 16 | 20: value (ref) -> 17 | foobar",shape="box"]; 18 | "18497576320" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="32 bytes 19 | 7 objects, 176 bytes reachable 20 | this object in no reference cycles 21 | array of 4 j.l.Object 22 | 23 | [\"a\" 1 \"foobar\" 3.5]",shape="box"]; 24 | "34343005824" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 25 | 2 objects, 48 bytes reachable 26 | this object in no reference cycles 27 | j.l.String 28 | 12: hash (int) 97 29 | 16: coder (byte) 0 30 | 17: hashIsZero (boolean) false 31 | 20: value (ref) -> 32 | a",shape="box"]; 33 | "34349647680" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 34 | 1 object, 24 bytes reachable 35 | this object in no reference cycles 36 | j.l.Long 37 | 16: value (long) 1 38 | 1",shape="box"]; 39 | "18497576960" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 40 | 1 object, 24 bytes reachable 41 | this object in no reference cycles 42 | array of 6 byte 43 | 44 | [102 111 111 98 97 114]",shape="box"]; 45 | "34343006016" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,label="24 bytes 46 | 1 object, 24 bytes reachable 47 | this object in no reference cycles 48 | array of 1 byte 49 | 50 | [97]",shape="box"]; 51 | "18497576064" ["reachable-only-from"=18497576064,"scc-num-nodes"=1,style="filled","my-unique-total-size"=208,"my-unique-num-reachable-nodes"=8,label="32 bytes 52 | 8 objects, 208 bytes reachable 53 | this object in no reference cycles 54 | c.l.PersistentArrayMap 55 | 12: _hash (int) 0 56 | 16: _hasheq (int) 0 57 | 20: array (ref) -> 58 | 24: _meta (ref) nil 59 | {\"a\" 1, \"foobar\" 3.5}",shape="box"]; 60 | "18497576768" -> "18497576960" ["field-name"="value",label="value"]; 61 | "18497576320" -> "34343005824" ["field-name"="[0]",label="[0]"]; 62 | "18497576320" -> "34349647680" ["field-name"="[1]",label="[1]"]; 63 | "18497576320" -> "18497576768" ["field-name"="[2]",label="[2]"]; 64 | "18497576320" -> "18497576576" ["field-name"="[3]",label="[3]"]; 65 | "34343005824" -> "34343006016" ["field-name"="value",label="value"]; 66 | "18497576064" -> "18497576320" ["field-name"="array",label="array"]; 67 | } -------------------------------------------------------------------------------- /doc/maybe-jol-size-bug/MaybeBug.java: -------------------------------------------------------------------------------- 1 | //package org.openjdk.jol.samples; 2 | 3 | import java.util.IdentityHashMap; 4 | import org.openjdk.jol.info.ClassLayout; 5 | import org.openjdk.jol.vm.VM; 6 | 7 | import static java.lang.System.out; 8 | 9 | public class MaybeBug { 10 | 11 | public static boolean showSizes(Object obj) { 12 | Class c = obj.getClass(); 13 | 14 | ClassLayout parsedInst = ClassLayout.parseInstance(obj); 15 | ClassLayout parsedCls = ClassLayout.parseClass(c); 16 | long vmSizeOf = VM.current().sizeOf(obj); 17 | long sizeFromInst = parsedInst.instanceSize(); 18 | long sizeFromCls = parsedCls.instanceSize(); 19 | 20 | out.println("\n------------------------------------------------------------"); 21 | out.println("obj= " + obj); 22 | out.println("obj.getClass()= " + c); 23 | out.println("\ntoPrintable of parseInstance ret value:"); 24 | out.println(parsedInst.toPrintable()); 25 | 26 | out.println("\ntoPrintable of parseClass ret value:"); 27 | out.println(parsedCls.toPrintable()); 28 | 29 | out.println("obj= " + obj); 30 | out.println("obj.getClass()= " + c); 31 | out.println("VM.current().sizeOf(obj)= " + vmSizeOf); 32 | out.println("parsedInst.instanceSize()= " + sizeFromInst); 33 | out.println("parsedCls.instanceSize()= " + sizeFromCls); 34 | return ((vmSizeOf == sizeFromInst) && (vmSizeOf == sizeFromCls)); 35 | } 36 | 37 | public static void main(String[] args) throws Exception { 38 | StringBuilder s = new StringBuilder(); 39 | String str = "bar"; 40 | boolean good; 41 | IdentityHashMap ihm = new IdentityHashMap(); 42 | 43 | out.println(VM.current().details()); 44 | 45 | final Long i = Long.valueOf(5); 46 | good = showSizes(i); 47 | s.append("\n" + (good ? "ok " : "bad") + " " + i.getClass() + " obj=" + i); 48 | 49 | good = showSizes(str); 50 | s.append("\n" + (good ? "ok " : "bad") + " " + str.getClass() + " obj=" + str); 51 | 52 | good = showSizes(ihm); 53 | s.append("\n" + (good ? "ok " : "bad") + " " + ihm.getClass() + " obj=" + ihm); 54 | 55 | final Class c = Class.forName("java.lang.Class"); 56 | good = showSizes(c); 57 | s.append("\n" + (good ? "ok " : "bad") + " " + c.getClass() + " obj=" + c); 58 | 59 | final Class c2 = Class.forName("java.lang.Long"); 60 | good = showSizes(c2); 61 | s.append("\n" + (good ? "ok " : "bad") + " " + c2.getClass() + " obj=" + c2); 62 | 63 | final Class c3 = Class.forName("java.lang.String"); 64 | good = showSizes(c3); 65 | s.append("\n" + (good ? "ok " : "bad") + " " + c3.getClass() + " obj=" + c3); 66 | 67 | final Class c4 = Class.forName("java.util.IdentityHashMap"); 68 | good = showSizes(c4); 69 | s.append("\n" + (good ? "ok " : "bad") + " " + c4.getClass() + " obj=" + c4); 70 | 71 | out.println("\n------------------------------------------------------------"); 72 | out.println(s.toString()); 73 | } 74 | } 75 | -------------------------------------------------------------------------------- /doc/images/compressed-oops-disabled/map2-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "270458824976096" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="48 bytes 4 | 7 objects, 208 bytes reachable 5 | this object in no reference cycles 6 | array of 4 j.l.Object 7 | 8 | [\"a\" 1 \"foobar\" 3.5]",shape="box"]; 9 | "270460740704544" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | this object in no reference cycles 12 | array of 1 byte 13 | 14 | [97]",shape="box"]; 15 | "270458824976144" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="24 bytes 16 | 1 object, 24 bytes reachable 17 | this object in no reference cycles 18 | j.l.Double 19 | 16: value (double) 3.5 20 | 3.5",shape="box"]; 21 | "270458677322912" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="24 bytes 22 | 1 object, 24 bytes reachable 23 | this object in no reference cycles 24 | j.l.Long 25 | 16: value (long) 1 26 | 1",shape="box"]; 27 | "270458824976200" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="24 bytes 28 | 1 object, 24 bytes reachable 29 | this object in no reference cycles 30 | array of 6 byte 31 | 32 | [102 111 111 98 97 114]",shape="box"]; 33 | "270460740704512" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="32 bytes 34 | 2 objects, 56 bytes reachable 35 | this object in no reference cycles 36 | j.l.String 37 | 12: hash (int) 97 38 | 16: coder (byte) 0 39 | 17: hashIsZero (boolean) false 40 | 24: value (ref) -> 41 | a",shape="box"]; 42 | "270458824976056" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,style="filled","my-unique-total-size"=248,"my-unique-num-reachable-nodes"=8,label="40 bytes 43 | 8 objects, 248 bytes reachable 44 | this object in no reference cycles 45 | c.l.PersistentArrayMap 46 | 12: _hash (int) 0 47 | 16: _hasheq (int) 0 48 | 24: array (ref) -> 49 | 32: _meta (ref) nil 50 | {\"a\" 1, \"foobar\" 3.5}",shape="box"]; 51 | "270458824976168" ["reachable-only-from"=270458824976056,"scc-num-nodes"=1,label="32 bytes 52 | 2 objects, 56 bytes reachable 53 | this object in no reference cycles 54 | j.l.String 55 | 12: hash (int) -1268878963 56 | 16: coder (byte) 0 57 | 17: hashIsZero (boolean) false 58 | 24: value (ref) -> 59 | foobar",shape="box"]; 60 | "270458824976096" -> "270460740704512" ["field-name"="[0]",label="[0]"]; 61 | "270458824976096" -> "270458677322912" ["field-name"="[1]",label="[1]"]; 62 | "270458824976096" -> "270458824976168" ["field-name"="[2]",label="[2]"]; 63 | "270458824976096" -> "270458824976144" ["field-name"="[3]",label="[3]"]; 64 | "270460740704512" -> "270460740704544" ["field-name"="value",label="value"]; 65 | "270458824976056" -> "270458824976096" ["field-name"="array",label="array"]; 66 | "270458824976168" -> "270458824976200" ["field-name"="value",label="value"]; 67 | } -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-realized1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "17920231808" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 4 | 5 objects, 128 bytes reachable 5 | this object in no reference cycles 6 | c.l.LazySeq 7 | 12: _meta (ref) nil 8 | 16: fn (ref) -> 9 | 20: sv (ref) nil 10 | 24: s (ref) nil 11 | 28: lock (ref) -> 12 | val maybe realizes if str'ed",shape="box"]; 13 | "17920232064" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 14 | 2 objects, 48 bytes reachable 15 | this object in no reference cycles 16 | user$fib_fn$fn__10167 17 | 12: __methodImplCache (ref) nil 18 | 16: a (ref) -> 19 | 20: b (ref) -> 20 | val maybe realizes if str'ed",shape="box"]; 21 | "17920232256" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="16 bytes 22 | 2 objects, 48 bytes reachable 23 | this object in no reference cycles 24 | j.u.c.locks.ReentrantLock 25 | 12: sync (ref) -> 26 | val maybe realizes if str'ed",shape="box"]; 27 | "17920231296" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,style="filled","my-unique-total-size"=216,"my-unique-num-reachable-nodes"=8,label="32 bytes 28 | 8 objects, 216 bytes reachable 29 | this object in no reference cycles 30 | c.l.LazySeq 31 | 12: _meta (ref) nil 32 | 16: fn (ref) nil 33 | 20: sv (ref) nil 34 | 24: s (ref) -> 35 | 28: lock (ref) nil 36 | val maybe realizes if str'ed",shape="box"]; 37 | "17920232384" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 38 | 1 object, 32 bytes reachable 39 | this object in no reference cycles 40 | j.u.c.locks.ReentrantLock$NonfairSync 41 | 12: exclusiveOwnerThread (ref) nil 42 | 16: state (int) 0 43 | 20: head (ref) nil 44 | 24: tail (ref) nil 45 | val maybe realizes if str'ed",shape="box"]; 46 | "34349647488" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 47 | 1 object, 24 bytes reachable 48 | this object in no reference cycles 49 | j.l.Long 50 | 16: value (long) 0 51 | 0",shape="box"]; 52 | "17920231552" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 53 | 7 objects, 184 bytes reachable 54 | this object in no reference cycles 55 | c.l.Cons 56 | 12: _meta (ref) nil 57 | 16: _hash (int) 0 58 | 20: _hasheq (int) 0 59 | 24: _first (ref) -> 60 | 28: _more (ref) -> 61 | val maybe realizes if str'ed",shape="box"]; 62 | "34349647680" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 63 | 1 object, 24 bytes reachable 64 | this object in no reference cycles 65 | j.l.Long 66 | 16: value (long) 1 67 | 1",shape="box"]; 68 | "17920231808" -> "17920232064" ["field-name"="fn",label="fn"]; 69 | "17920231808" -> "17920232256" ["field-name"="lock",label="lock"]; 70 | "17920232064" -> "34349647680" ["field-name"="b",label="b"]; 71 | "17920232064" -> "34349647680" ["field-name"="a",label="a"]; 72 | "17920232256" -> "17920232384" ["field-name"="sync",label="sync"]; 73 | "17920231296" -> "17920231552" ["field-name"="s",label="s"]; 74 | "17920231552" -> "34349647488" ["field-name"="_first",label="_first"]; 75 | "17920231552" -> "17920231808" ["field-name"="_more",label="_more"]; 76 | } -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-11.0.26-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "2345092440" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="48 bytes 4 | 1 object, 48 bytes reachable 5 | this object in no reference cycles 6 | array of 30 byte 7 | 8 | [102 111 111 100 32 104 97 115 32 111 110 108 121 ...",shape="box"]; 9 | "2345105016" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,style="filled","my-unique-total-size"=424,"my-unique-num-reachable-nodes"=9,label="40 bytes 10 | 9 objects, 424 bytes reachable 11 | this object in no reference cycles 12 | c.l.PersistentVector 13 | 12: _hash (int) 0 14 | 16: _hasheq (int) 0 15 | 20: cnt (int) 2 16 | 24: shift (int) 5 17 | 28: root (ref) -> 18 | 32: tail (ref) -> 19 | 36: _meta (ref) nil 20 | [\"food has only 8-bit characters\" \"fሴod has non-8- ...",shape="box"]; 21 | "2345092840" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="80 bytes 22 | 1 object, 80 bytes reachable 23 | this object in no reference cycles 24 | array of 60 byte 25 | 26 | [102 0 52 18 111 0 100 0 32 0 104 0 97 0 115 0 32 ...",shape="box"]; 27 | "2345104992" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="24 bytes 28 | 5 objects, 200 bytes reachable 29 | this object in no reference cycles 30 | array of 2 j.l.Object 31 | 32 | [\"food has only 8-bit characters\" \"fሴod has non-8- ...",shape="box"]; 33 | "2345092816" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="24 bytes 34 | 2 objects, 104 bytes reachable 35 | this object in no reference cycles 36 | j.l.String 37 | 12: value (ref) -> 38 | 16: hash (int) -938149940 39 | 20: coder (byte) 1 40 | fሴod has non-8-bit characters!",shape="box"]; 41 | "2220576048" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="24 bytes 42 | 3 objects, 184 bytes reachable 43 | this object in no reference cycles 44 | c.l.PersistentVector$Node 45 | 12: edit (ref) -> 46 | 16: array (ref) -> 47 | val maybe realizes if str'ed",shape="box"]; 48 | "2220576072" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="16 bytes 49 | 1 object, 16 bytes reachable 50 | this object in no reference cycles 51 | j.u.c.atomic.AtomicReference 52 | 12: value (ref) nil 53 | val maybe realizes if str'ed",shape="box"]; 54 | "2345092416" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="24 bytes 55 | 2 objects, 72 bytes reachable 56 | this object in no reference cycles 57 | j.l.String 58 | 12: value (ref) -> 59 | 16: hash (int) 511352126 60 | 20: coder (byte) 0 61 | food has only 8-bit characters",shape="box"]; 62 | "2220576088" ["reachable-only-from"=2345105016,"scc-num-nodes"=1,label="144 bytes 63 | 1 object, 144 bytes reachable 64 | this object in no reference cycles 65 | array of 32 j.l.Object 66 | 67 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 68 | "2345105016" -> "2220576048" ["field-name"="root",label="root"]; 69 | "2345105016" -> "2345104992" ["field-name"="tail",label="tail"]; 70 | "2345104992" -> "2345092416" ["field-name"="[0]",label="[0]"]; 71 | "2345104992" -> "2345092816" ["field-name"="[1]",label="[1]"]; 72 | "2345092816" -> "2345092840" ["field-name"="value",label="value"]; 73 | "2220576048" -> "2220576072" ["field-name"="edit",label="edit"]; 74 | "2220576048" -> "2220576088" ["field-name"="array",label="array"]; 75 | "2345092416" -> "2345092440" ["field-name"="value",label="value"]; 76 | } -------------------------------------------------------------------------------- /doc/images/strings-8-bit-and-not-Linux-6.8.0-56-jdk-openjdk-1.8.0_442-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "29516619072" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,style="filled","my-unique-total-size"=456,"my-unique-num-reachable-nodes"=9,label="40 bytes 4 | 9 objects, 456 bytes reachable 5 | this object in no reference cycles 6 | c.l.PersistentVector 7 | 12: _hash (int) 0 8 | 16: _hasheq (int) 0 9 | 20: cnt (int) 2 10 | 24: shift (int) 5 11 | 28: root (ref) -> 12 | 32: tail (ref) -> 13 | 36: _meta (ref) nil 14 | [\"food has only 8-bit characters\" \"fሴod has non-8- ...",shape="box"]; 15 | "17766377984" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="144 bytes 16 | 1 object, 144 bytes reachable 17 | this object in no reference cycles 18 | array of 32 j.l.Object 19 | 20 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 21 | "17766377664" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="24 bytes 22 | 3 objects, 184 bytes reachable 23 | this object in no reference cycles 24 | c.l.PersistentVector$Node 25 | 12: edit (ref) -> 26 | 16: array (ref) -> 27 | val maybe realizes if str'ed",shape="box"]; 28 | "29516528448" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="80 bytes 29 | 1 object, 80 bytes reachable 30 | this object in no reference cycles 31 | array of 30 char 32 | 33 | [\\f \\o \\o \\d \\space \\h \\a \\s \\space \\o \\n \\l \\y \\s ...",shape="box"]; 34 | "29516528256" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="24 bytes 35 | 2 objects, 104 bytes reachable 36 | this object in no reference cycles 37 | j.l.String 38 | 12: value (ref) -> 39 | 16: hash (int) 511352126 40 | food has only 8-bit characters",shape="box"]; 41 | "29516531264" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="80 bytes 42 | 1 object, 80 bytes reachable 43 | this object in no reference cycles 44 | array of 30 char 45 | 46 | [\\f \\ሴ \\o \\d \\space \\h \\a \\s \\space \\n \\o \\n \\- \\8 ...",shape="box"]; 47 | "29516531072" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="24 bytes 48 | 2 objects, 104 bytes reachable 49 | this object in no reference cycles 50 | j.l.String 51 | 12: value (ref) -> 52 | 16: hash (int) -938149940 53 | fሴod has non-8-bit characters!",shape="box"]; 54 | "29516618880" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="24 bytes 55 | 5 objects, 232 bytes reachable 56 | this object in no reference cycles 57 | array of 2 j.l.Object 58 | 59 | [\"food has only 8-bit characters\" \"fሴod has non-8- ...",shape="box"]; 60 | "17766377856" ["reachable-only-from"=29516619072,"scc-num-nodes"=1,label="16 bytes 61 | 1 object, 16 bytes reachable 62 | this object in no reference cycles 63 | j.u.c.atomic.AtomicReference 64 | 12: value (ref) nil 65 | val maybe realizes if str'ed",shape="box"]; 66 | "29516619072" -> "17766377664" ["field-name"="root",label="root"]; 67 | "29516619072" -> "29516618880" ["field-name"="tail",label="tail"]; 68 | "17766377664" -> "17766377856" ["field-name"="edit",label="edit"]; 69 | "17766377664" -> "17766377984" ["field-name"="array",label="array"]; 70 | "29516528256" -> "29516528448" ["field-name"="value",label="value"]; 71 | "29516531072" -> "29516531264" ["field-name"="value",label="value"]; 72 | "29516618880" -> "29516528256" ["field-name"="[0]",label="[0]"]; 73 | "29516618880" -> "29516531072" ["field-name"="[1]",label="[1]"]; 74 | } -------------------------------------------------------------------------------- /doc/README-performance.md: -------------------------------------------------------------------------------- 1 | # Performance improvements from ubergraph 0.5.3 to 0.6.0 2 | 3 | Some modest reductions in compute time of some steps on large graphs. 4 | Definitely improvements worth having. 5 | 6 | macOS 10.13.6 7 | 8 | $ java -version 9 | java version "1.8.0_192" 10 | Java(TM) SE Runtime Environment (build 1.8.0_192-b12) 11 | Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode) 12 | 13 | git SHA of cljol: 9c38e79e93e7acc191c27b5fb6e426e81c88f53e 14 | 15 | ``` 16 | ---------------------------------------------------------------------- 17 | [ ... some output elided if the ubergraph version did not affect the 18 | performance ...] 19 | 20 | converted 129398 objmaps into ubergraph with 273382 edges: 2277.190175 msec, 1 gc-count, 26 gc-time-msec 21 | The scc-graph has 107087 nodes and 179726 edges, took: 3582.82373 msec, 1 gc-count, 63 gc-time-msec 22 | Calculated num-reachable-nodes and total-size for scc-graph in: 3679.876264 msec, 2 gc-count, 88 gc-time-msec 23 | 24 | calculated :bounded total sizes: 11931.822018 msec, 4 gc-count, 195 gc-time-msec 25 | added graphviz attributes: 8187.715493 msec, 5 gc-count, 134 gc-time-msec 26 | 129398 objects 27 | 273382 references between them 28 | 4987408 bytes total in all objects 29 | 25958 leaf objects (no references to other objects) 30 | 31 | ---------------------------------------------------------------------- 32 | converted 129400 objmaps into ubergraph with 273384 edges: 2447.445549 msec, 1 gc-count, 77 gc-time-msec 33 | The scc-graph has 107089 nodes and 179729 edges, took: 4302.79296 msec, 2 gc-count, 735 gc-time-msec 34 | Calculated num-reachable-nodes and total-size for scc-graph in: 3458.973393 msec, 1 gc-count, 53 gc-time-msec 35 | 36 | calculated :bounded total sizes: 12639.766494 msec, 5 gc-count, 961 gc-time-msec 37 | added graphviz attributes: 8667.186223 msec, 6 gc-count, 175 gc-time-msec 38 | 129400 objects 39 | 273384 references between them 40 | 4987448 bytes total in all objects 41 | 25958 leaf objects (no references to other objects) 42 | ``` 43 | 44 | 45 | Same as above, except one line change to deps.edn to use ubergraph 46 | 0.6.0 instead of 0.5.3: 47 | 48 | ``` 49 | ---------------------------------------------------------------------- 50 | converted 129636 objmaps into ubergraph with 273713 edges: 2265.629436 msec, 1 gc-count, 35 gc-time-msec 51 | The scc-graph has 107319 nodes and 180028 edges, took: 3955.709981 msec, 2 gc-count, 906 gc-time-msec 52 | Calculated num-reachable-nodes and total-size for scc-graph in: 2206.444629 msec, 1 gc-count, 121 gc-time-msec 53 | 54 | calculated :bounded total sizes: 10918.40454 msec, 4 gc-count, 1083 gc-time-msec 55 | added graphviz attributes: 9037.733732 msec, 7 gc-count, 415 gc-time-msec 56 | 129636 objects 57 | 273713 references between them 58 | 4995888 bytes total in all objects 59 | 26015 leaf objects (no references to other objects) 60 | 61 | ---------------------------------------------------------------------- 62 | converted 129636 objmaps into ubergraph with 273713 edges: 2272.416103 msec, 1 gc-count, 86 gc-time-msec 63 | The scc-graph has 107319 nodes and 180028 edges, took: 3796.13578 msec, 2 gc-count, 766 gc-time-msec 64 | Calculated num-reachable-nodes and total-size for scc-graph in: 1934.077117 msec, 1 gc-count, 47 gc-time-msec 65 | 66 | calculated :bounded total sizes: 10577.804244 msec, 5 gc-count, 1015 gc-time-msec 67 | added graphviz attributes: 8912.418918 msec, 6 gc-count, 435 gc-time-msec 68 | 129636 objects 69 | 273713 references between them 70 | 4996272 bytes total in all objects 71 | 26015 leaf objects (no references to other objects) 72 | ``` 73 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-realized1-no-longs.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18094456128" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="56 bytes 4 | 1 object, 56 bytes reachable 5 | this object in no reference cycles 6 | c.l.LongRange 7 | 12: _meta (ref) nil 8 | 16: _hash (int) 0 9 | 20: _hasheq (int) 0 10 | 24: start (long) 32 11 | 32: end (long) 1000 12 | 40: step (long) 1 13 | 48: count (int) 968 14 | val maybe realizes if str'ed",shape="box"]; 15 | "18094454080" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 16 | 41 objects, 1144 bytes reachable 17 | this object in no reference cycles 18 | c.l.ChunkedCons 19 | 12: _meta (ref) nil 20 | 16: _hash (int) 0 21 | 20: _hasheq (int) 0 22 | 24: chunk (ref) -> 23 | 28: _more (ref) -> 24 | val maybe realizes if str'ed",shape="box"]; 25 | "18094455680" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 26 | 6 objects, 176 bytes reachable 27 | this object in no reference cycles 28 | c.l.LazySeq 29 | 12: _meta (ref) nil 30 | 16: fn (ref) -> 31 | 20: sv (ref) nil 32 | 24: s (ref) nil 33 | 28: lock (ref) -> 34 | val maybe realizes if str'ed",shape="box"]; 35 | "18094454528" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="144 bytes 36 | 33 objects, 912 bytes reachable 37 | this object in no reference cycles 38 | array of 32 j.l.Object 39 | 40 | [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 ...",shape="box"]; 41 | "18094456704" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="16 bytes 42 | 2 objects, 48 bytes reachable 43 | this object in no reference cycles 44 | j.u.c.locks.ReentrantLock 45 | 12: sync (ref) -> 46 | val maybe realizes if str'ed",shape="box"]; 47 | "18094456832" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 48 | 1 object, 32 bytes reachable 49 | this object in no reference cycles 50 | j.u.c.locks.ReentrantLock$NonfairSync 51 | 12: exclusiveOwnerThread (ref) nil 52 | 16: state (int) 0 53 | 20: head (ref) nil 54 | 24: tail (ref) nil 55 | val maybe realizes if str'ed",shape="box"]; 56 | "18094453824" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,style="filled","my-unique-total-size"=1176,"my-unique-num-reachable-nodes"=42,label="32 bytes 57 | 42 objects, 1176 bytes reachable 58 | this object in no reference cycles 59 | c.l.LazySeq 60 | 12: _meta (ref) nil 61 | 16: fn (ref) nil 62 | 20: sv (ref) nil 63 | 24: s (ref) -> 64 | 28: lock (ref) nil 65 | val maybe realizes if str'ed",shape="box"]; 66 | "18094456576" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="16 bytes 67 | 1 object, 16 bytes reachable 68 | this object in no reference cycles 69 | user$fn__10278 70 | 12: __methodImplCache (ref) nil 71 | val maybe realizes if str'ed",shape="box"]; 72 | "18094455936" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 73 | 3 objects, 96 bytes reachable 74 | this object in no reference cycles 75 | clojure.core$map$fn__5954 76 | 12: __methodImplCache (ref) nil 77 | 16: coll (ref) -> 78 | 20: f (ref) -> 79 | val maybe realizes if str'ed",shape="box"]; 80 | "18094454336" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 81 | 34 objects, 936 bytes reachable 82 | this object in no reference cycles 83 | c.l.ArrayChunk 84 | 12: off (int) 0 85 | 16: end (int) 32 86 | 20: array (ref) -> 87 | val maybe realizes if str'ed",shape="box"]; 88 | "18094454080" -> "18094454336" ["field-name"="chunk",label="chunk"]; 89 | "18094454080" -> "18094455680" ["field-name"="_more",label="_more"]; 90 | "18094455680" -> "18094455936" ["field-name"="fn",label="fn"]; 91 | "18094455680" -> "18094456704" ["field-name"="lock",label="lock"]; 92 | "18094456704" -> "18094456832" ["field-name"="sync",label="sync"]; 93 | "18094453824" -> "18094454080" ["field-name"="s",label="s"]; 94 | "18094455936" -> "18094456128" ["field-name"="coll",label="coll"]; 95 | "18094455936" -> "18094456576" ["field-name"="f",label="f"]; 96 | "18094454336" -> "18094454528" ["field-name"="array",label="array"]; 97 | } -------------------------------------------------------------------------------- /scripts/summarize-warnings.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | 3 | import fileinput 4 | import re 5 | 6 | for line in fileinput.input(): 7 | line = line.rstrip() 8 | match = re.search(r"""WARNING: A Java agent has been loaded dynamically \(/tmp/jolAgent\d+.jar\)""", line) 9 | if match: 10 | print("WARN1") 11 | continue 12 | 13 | match = re.search(r"""WARNING: If a serviceability tool is in use, please run with -XX:\+EnableDynamicAgentLoading to hide this warning""", line) 14 | if match: 15 | print("WARN2") 16 | continue 17 | match = re.search(r"""WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information""", line) 18 | if match: 19 | print("WARN3") 20 | continue 21 | match = re.search(r"""WARNING: Dynamic loading of agents will be disallowed by default in a future release""", line) 22 | if match: 23 | print("WARN4") 24 | continue 25 | match = re.search(r"""# WARNING: Unable to attach Serviceability Agent. sun.jvm.hotspot.memory.Universe.getNarrowOopBase\(\)""", line) 26 | if match: 27 | print("WARN5") 28 | continue 29 | match = re.search(r"""WARNING: A terminally deprecated method in sun.misc.Unsafe has been called""", line) 30 | if match: 31 | print("WARN6") 32 | continue 33 | match = re.search(r"""WARNING: sun.misc.Unsafe::arrayBaseOffset has been called by org.openjdk.jol.vm.HotspotUnsafe \(file:.*.m2/repository/org/openjdk/jol/jol-core/0.9/jol-core-0.9.jar\)""", line) 34 | if match: 35 | print("WARN-Unsafe::arrayBaseOffset-called") 36 | continue 37 | match = re.search(r"""WARNING: Please consider reporting this to the maintainers of (.*)""", line) 38 | if match: 39 | print("WARN-report-%s" % (match.group(1))) 40 | continue 41 | match = re.search(r"""WARNING: sun.misc.Unsafe::arrayBaseOffset will be removed in a future release""", line) 42 | if match: 43 | print("WARN-Unsafe::arrayBaseOffset-will-be-removed") 44 | continue 45 | match = re.search(r"""Boxed math warning, cljol/ubergraph_extras.clj:\d+:\d+ - call: public static java.lang.Number clojure.lang.Numbers.divide\(java.lang.Object,long\).""", line) 46 | if match: 47 | print("WARN-BoxedMath1") 48 | continue 49 | match = re.search(r"""Boxed math warning, cljol/ubergraph_extras.clj:128:9 - call: public static boolean clojure.lang.Numbers.lt\(long,java.lang.Object\).""", line) 50 | if match: 51 | print("WARN-BoxedMath2") 52 | continue 53 | match = re.search(r"""# WARNING: Unable to get Instrumentation. Dynamic Attach failed. You may add this JAR as -javaagent manually, or supply -Djdk.attach.allowAttachSelf""", line) 54 | if match: 55 | print("WARN7") 56 | continue 57 | match = re.search(r"""# WARNING: Unable to attach Serviceability Agent. You can try again with escalated privileges. Two options: a\) use -Djol.tryWithSudo=true to try with sudo; b\) echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope""", line) 58 | if match: 59 | print("WARN8") 60 | continue 61 | match = re.search(r"""WARNING: An illegal reflective access operation has occurred""", line) 62 | if match: 63 | print("WARN9") 64 | continue 65 | match = re.search(r"""WARNING: Illegal reflective access by org.openjdk.jol.util.ObjectUtils \(file:.*/.m2/repository/org/openjdk/jol/jol-core/0.9/jol-core-0.9.jar\) to field (.*)""", line) 66 | if match: 67 | print("WARN10-%s" % (match.group(1))) 68 | continue 69 | match = re.search(r"""WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations""", line) 70 | if match: 71 | print("WARN11") 72 | continue 73 | match = re.search(r"""WARNING: All illegal access operations will be denied in a future release""", line) 74 | if match: 75 | print("WARN12") 76 | continue 77 | # match = re.search(r""" 78 | # """, line) 79 | # if match: 80 | # print("WARN") 81 | # continue 82 | print(line) 83 | 84 | 85 | -------------------------------------------------------------------------------- /test/cljol/dig_test.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.dig-test 2 | (:require [clojure.test :refer :all] 3 | [clojure.java.io :as io] 4 | [clojure.edn :as edn] 5 | [loom.alg :as lalg] 6 | [ubergraph.core :as uber] 7 | [cljol.performance :as perf] 8 | [cljol.graph :as gr] 9 | [cljol.ubergraph-extras :as ubere] 10 | [cljol.dig9 :refer :all])) 11 | 12 | (def ref-array (object-array 5)) 13 | (def prim-array (int-array 4 [2 4 6 8])) 14 | (def two-dee-array (to-array-2d [[1 2 3] [4 5 6] [7 8 9]])) 15 | 16 | (deftest a-test 17 | (are [x] (let [ex (reachable-objmaps [x] {})] 18 | (and (nil? (validate-obj-graph ex)) 19 | (not (any-objects-overlap? ex)) 20 | (not (any-object-moved? ex)))) 21 | #{} 22 | #{5 7 12 13} 23 | #{nil} 24 | {5 7, 12 13} 25 | {5 false, nil 13} 26 | ref-array 27 | prim-array 28 | two-dee-array 29 | {{5 false, nil 13} "first-kv", 30 | "second-kv" {8 9, 10 11}} 31 | )) 32 | 33 | 34 | (deftest ubergraph-successors-test 35 | ;; The current implementation of cljol.graph/edge-vectors assumes 36 | ;; that ubergraph.core/successors returns each successor node at 37 | ;; most once, even if there are multiple parallel edges to it. Make 38 | ;; a test to check this. 39 | (let [g (uber/multidigraph [1 2] 40 | [1 2] 41 | [1 3] 42 | [1 4] 43 | [4 2] 44 | [4 2])] 45 | (is (= false (nil? (seq (uber/successors g 1))))) 46 | (is (= #{2 3 4} (set (uber/successors g 1)))) 47 | (is (= true (apply distinct? (uber/successors g 1)))) 48 | (is (= false (apply distinct? (map uber/dest (uber/out-edges g 1))))) 49 | (is (= false (nil? (seq (uber/predecessors g 2))))) 50 | (is (= #{1 4} (set (uber/predecessors g 2)))) 51 | (is (= true (apply distinct? (uber/predecessors g 2)))) 52 | (is (= false (apply distinct? (map uber/src (uber/in-edges g 2))))))) 53 | 54 | 55 | (defn compare-sccs-with-perf [g] 56 | (println "Comparing SCCs found multiple ways for graph with" 57 | (uber/count-nodes g) "nodes," 58 | (uber/count-edges g) "edges") 59 | (let [{loom-scc :ret :as p} (perf/time (lalg/scc g)) 60 | _ (do (print "Using loom.alg/scc, found" (count loom-scc) "SCCs in: ") 61 | (perf/print-perf-stats p)) 62 | {scc-data :ret :as p} (perf/time (ubere/scc-tarjan g)) 63 | _ (do (print "Using cljol.graph/scc-tarjan, found" 64 | (count (:components scc-data)) "SCCs in: ") 65 | (perf/print-perf-stats p))] 66 | (is (= (set (map set loom-scc)) 67 | (set (:components scc-data)))))) 68 | 69 | 70 | (deftest unique-owners-tests 71 | (is (= (uniquely-owned-values {:a [1 2] :b [2 3] :c [4 5]}) 72 | {:owners {1 :a, 2 :cljol.dig9/multiple-owners, 3 :b, 4 :c, 5 :c}, 73 | :uniquely-owned {:a #{1}, :b #{3}, :c #{4 5}}})) 74 | (is (= (uniquely-owned-values {:a [1 2] :b [2 3] :c [4 5] :d [3 4]}) 75 | {:owners 76 | {1 :a, 77 | 2 :cljol.dig9/multiple-owners, 78 | 3 :cljol.dig9/multiple-owners, 79 | 4 :cljol.dig9/multiple-owners, 80 | 5 :c}, 81 | :uniquely-owned {:a #{1}, :b #{}, :c #{5}, :d #{}}})) 82 | (is (= (uniquely-owned-values {:a [1 2]}) 83 | {:owners {1 :a, 2 :a}, 84 | :uniquely-owned {:a [1 2]}}))) 85 | 86 | 87 | (deftest scc-tests 88 | (let [g1 (uber/multidigraph 89 | [1 2] 90 | [3 4] 91 | [5 6] 92 | [5 7] 93 | [8 4] 94 | [8 2] 95 | [9 5] 96 | [9 8] 97 | [2 10] 98 | [2 3] 99 | [4 10] 100 | [4 1]) 101 | fname2 "resources/dimultigraph-129k-nodes-272k-edges.edn" 102 | g2 (ubere/read-ubergraph-as-edges fname2)] 103 | (is (= (set (map set (lalg/scc g1))) 104 | (set (:components (ubere/scc-tarjan g1))))) 105 | (compare-sccs-with-perf g1) 106 | (compare-sccs-with-perf g2))) 107 | -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-realized2.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18777430336" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 4 | 1 object, 32 bytes reachable 5 | this object in no reference cycles 6 | j.u.c.locks.ReentrantLock$NonfairSync 7 | 12: exclusiveOwnerThread (ref) nil 8 | 16: state (int) 0 9 | 20: head (ref) nil 10 | 24: tail (ref) nil 11 | val maybe realizes if str'ed",shape="box"]; 12 | "34349647488" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 13 | 1 object, 24 bytes reachable 14 | this object in no reference cycles 15 | j.l.Long 16 | 16: value (long) 0 17 | 0",shape="box"]; 18 | "18777430208" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="16 bytes 19 | 2 objects, 48 bytes reachable 20 | this object in no reference cycles 21 | j.u.c.locks.ReentrantLock 22 | 12: sync (ref) -> 23 | val maybe realizes if str'ed",shape="box"]; 24 | "18777430016" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 25 | 3 objects, 72 bytes reachable 26 | this object in no reference cycles 27 | user$fib_fn$fn__10167 28 | 12: __methodImplCache (ref) nil 29 | 16: a (ref) -> 30 | 20: b (ref) -> 31 | val maybe realizes if str'ed",shape="box"]; 32 | "17920231296" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,style="filled","my-unique-total-size"=304,"my-unique-num-reachable-nodes"=11,label="32 bytes 33 | 11 objects, 304 bytes reachable 34 | this object in no reference cycles 35 | c.l.LazySeq 36 | 12: _meta (ref) nil 37 | 16: fn (ref) nil 38 | 20: sv (ref) nil 39 | 24: s (ref) -> 40 | 28: lock (ref) nil 41 | val maybe realizes if str'ed",shape="box"]; 42 | "17920231552" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 43 | 10 objects, 272 bytes reachable 44 | this object in no reference cycles 45 | c.l.Cons 46 | 12: _meta (ref) nil 47 | 16: _hash (int) 0 48 | 20: _hasheq (int) 0 49 | 24: _first (ref) -> 50 | 28: _more (ref) -> 51 | val maybe realizes if str'ed",shape="box"]; 52 | "17920231808" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 53 | 8 objects, 216 bytes reachable 54 | this object in no reference cycles 55 | c.l.LazySeq 56 | 12: _meta (ref) nil 57 | 16: fn (ref) nil 58 | 20: sv (ref) nil 59 | 24: s (ref) -> 60 | 28: lock (ref) nil 61 | val maybe realizes if str'ed",shape="box"]; 62 | "18777429760" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 63 | 6 objects, 152 bytes reachable 64 | this object in no reference cycles 65 | c.l.LazySeq 66 | 12: _meta (ref) nil 67 | 16: fn (ref) -> 68 | 20: sv (ref) nil 69 | 24: s (ref) nil 70 | 28: lock (ref) -> 71 | val maybe realizes if str'ed",shape="box"]; 72 | "34349647872" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 73 | 1 object, 24 bytes reachable 74 | this object in no reference cycles 75 | j.l.Long 76 | 16: value (long) 2 77 | 2",shape="box"]; 78 | "18777430592" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="32 bytes 79 | 7 objects, 184 bytes reachable 80 | this object in no reference cycles 81 | c.l.Cons 82 | 12: _meta (ref) nil 83 | 16: _hash (int) 0 84 | 20: _hasheq (int) 0 85 | 24: _first (ref) -> 86 | 28: _more (ref) -> 87 | val maybe realizes if str'ed",shape="box"]; 88 | "34349647680" ["reachable-only-from"=17920231296,"scc-num-nodes"=1,label="24 bytes 89 | 1 object, 24 bytes reachable 90 | this object in no reference cycles 91 | j.l.Long 92 | 16: value (long) 1 93 | 1",shape="box"]; 94 | "18777430208" -> "18777430336" ["field-name"="sync",label="sync"]; 95 | "18777430016" -> "34349647680" ["field-name"="a",label="a"]; 96 | "18777430016" -> "34349647872" ["field-name"="b",label="b"]; 97 | "17920231296" -> "17920231552" ["field-name"="s",label="s"]; 98 | "17920231552" -> "34349647488" ["field-name"="_first",label="_first"]; 99 | "17920231552" -> "17920231808" ["field-name"="_more",label="_more"]; 100 | "17920231808" -> "18777430592" ["field-name"="s",label="s"]; 101 | "18777429760" -> "18777430016" ["field-name"="fn",label="fn"]; 102 | "18777429760" -> "18777430208" ["field-name"="lock",label="lock"]; 103 | "18777430592" -> "34349647680" ["field-name"="_first",label="_first"]; 104 | "18777430592" -> "18777429760" ["field-name"="_more",label="_more"]; 105 | } -------------------------------------------------------------------------------- /deps.edn: -------------------------------------------------------------------------------- 1 | {:paths ["src/clj" "test"] 2 | :deps {com.fingerhutpress.cljol_jvm_support/cljol_jvm_support {:mvn/version "1.0"} 3 | medley/medley {:mvn/version "1.2.0"} 4 | ;; This version of dorothy library has a small fix for 5 | ;; this issue: https://github.com/daveray/dorothy/issues/18 6 | dorothy/dorothy {:git/url "https://github.com/daveray/dorothy" 7 | :sha "207570804dfda2162a15b9ee55b5e76ec6e1ecfa"} 8 | ;; This forked version of ubergraph library has a small update 9 | ;; to enable it to use dorothy version 0.7, which the forked 10 | ;; version above is based upon. dorothy simply moved a couple 11 | ;; of functions to a different namespace, which changes how 12 | ;; ubergraph needs to call those functions. 13 | ubergraph/ubergraph {:git/url "https://github.com/jafingerhut/ubergraph" 14 | :sha "df0d5498a4d833c3bfb70a6a39caa808b35692cc"} 15 | ;; This is the version of ubergraph depended upon by cljol 16 | ;; before the fix for the dorothy issue described above was 17 | ;; created. 18 | ;;{:mvn/version "0.8.2" :exclusions [tailrecursion/cljs-priority-map]} 19 | } 20 | :aliases 21 | { 22 | ;; Common alias to use for all Clojure/Java commands 23 | :clj {:jvm-opts ["-XX:-OmitStackTraceInFastThrow"]} 24 | 25 | ;; - start a Clojure/Java Socket REPL on port 50505: 26 | :socket 27 | {:jvm-opts 28 | ["-Dclojure.server.repl={:port,50505,:accept,clojure.core.server/repl}"]} 29 | 30 | ;; pull in specific versions of Clojure: 31 | ;; cljol uses Ubergraph, which uses Loom, and at least in the 32 | ;; current versions of those projects that cljol depends upon, Loom 33 | ;; uses .cljc files, which were new with Clojure 1.7.0. 34 | :1.7 {:override-deps {org.clojure/clojure {:mvn/version "1.7.0"}}} 35 | :1.8 {:override-deps {org.clojure/clojure {:mvn/version "1.8.0"}}} 36 | :1.9 {:override-deps {org.clojure/clojure {:mvn/version "1.9.0"}}} 37 | :1.10 {:override-deps {org.clojure/clojure {:mvn/version "1.10.0"}}} 38 | :1.10.1 {:override-deps {org.clojure/clojure {:mvn/version "1.10.1"}}} 39 | :master {:override-deps {org.clojure/clojure {:mvn/version "1.11.0-master-SNAPSHOT"}}} 40 | 41 | ;; Extra dependency I found useful for getting a list of all classes 42 | ;; defined in a running JVM. I used that to have lots of test cases 43 | ;; to compare the results of lists of object fields returned via 44 | ;; clojure.reflect/type-reflect vs. JOL's ClassData. Except for a 45 | ;; few minor differences in class names, e.g. "$" characters instead 46 | ;; of ".", and "<>" instead of "[]" in array type names, they were 47 | ;; identical. 48 | :classgraph {:extra-deps {io.github.classgraph/classgraph 49 | {:mvn/version "4.8.42"}}} 50 | :jamm {:extra-deps {com.clojure-goes-fast/clj-memory-meter 51 | {:mvn/version "0.1.2"}}} 52 | :jammlocal {:extra-deps {com.clojure-goes-fast/clj-memory-meter 53 | {:mvn/version "0.1.3-SNAPSHOT"}}} 54 | :eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.3.5"}} 55 | :main-opts ["-m" "eastwood.lint" 56 | "{:source-paths,[\"src/clj\"],:test-paths,[\"test\"],:add-linters,[:unused-fn-args,:unused-locals,:unused-namespaces,:unused-private-vars],:exclude-linters,[:implicit-dependencies],:exclude-namespaces,[cljol.jdk9-and-later]}"]} 57 | :loomlocal {:override-deps {aysylu/loom {:mvn/version "1.0.3-SNAPSHOT"}}} 58 | :uberlocal {:override-deps {ubergraph/ubergraph {:mvn/version "0.6.1-andy-mods"}}} 59 | :jollocal {:override-deps {org.openjdk.jol/jol-core {:mvn/version "0.9.1"}}} 60 | :generate {:extra-paths ["src/generate"] 61 | :extra-deps {org.clojure/data.int-map {:mvn/version "0.2.4"}}} 62 | :priv {:jvm-opts ["-Djdk.attach.allowAttachSelf" "-Djol.tryWithSudo=true"]} 63 | :priv2 {:jvm-opts ["-Djdk.attach.allowAttachSelf"]} 64 | :bigmem {:jvm-opts ["-Xms4g" "-Xmx8g"]} 65 | :cap {;; recommended options from README of 66 | ;; https://github.com/clojure-goes-fast/clj-async-profiler 67 | :jvm-opts ["-Djdk.attach.allowAttachSelf" 68 | ;; I have trouble entering password for this from 69 | ;; clj REPL. Maybe clojure command instead of clj 70 | ;; is better for this? 71 | "-Djol.tryWithSudo=true" 72 | "-XX:+UnlockDiagnosticVMOptions" 73 | "-XX:+DebugNonSafepoints"] 74 | :extra-deps {com.clojure-goes-fast/clj-async-profiler 75 | {:mvn/version "0.4.0"}}} 76 | :printcomp {:jvm-opts ["-XX:+PrintCompilation"]} 77 | :disablecompressedoops {:jvm-opts ["-XX:-UseCompressedOops"]} 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /doc/images/vec10-Linux-6.8.0-56-jdk-openjdk-21.0.6-clj-1.12.0.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18840849984" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 4 | 3 objects, 184 bytes reachable 5 | this object in no reference cycles 6 | c.l.PersistentVector$Node 7 | 12: edit (ref) -> 8 | 16: array (ref) -> 9 | val maybe realizes if str'ed",shape="box"]; 10 | "34349647488" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 11 | 1 object, 24 bytes reachable 12 | this object in no reference cycles 13 | j.l.Long 14 | 16: value (long) 0 15 | 0",shape="box"]; 16 | "18840849664" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,style="filled","my-unique-total-size"=520,"my-unique-num-reachable-nodes"=15,label="40 bytes 17 | 15 objects, 520 bytes reachable 18 | this object in no reference cycles 19 | c.l.PersistentVector 20 | 12: _hash (int) 0 21 | 16: _hasheq (int) 0 22 | 20: cnt (int) 10 23 | 24: shift (int) 5 24 | 28: root (ref) -> 25 | 32: tail (ref) -> 26 | 36: _meta (ref) nil 27 | [0 1 2 3 4 5 6 7 8 9]",shape="box"]; 28 | "18840850176" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="16 bytes 29 | 1 object, 16 bytes reachable 30 | this object in no reference cycles 31 | j.u.c.atomic.AtomicReference 32 | 12: value (ref) .setAccessible failed 33 | val maybe realizes if str'ed",shape="box"]; 34 | "18840850304" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="144 bytes 35 | 1 object, 144 bytes reachable 36 | this object in no reference cycles 37 | array of 32 j.l.Object 38 | 39 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 40 | "34349648640" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 41 | 1 object, 24 bytes reachable 42 | this object in no reference cycles 43 | j.l.Long 44 | 16: value (long) 6 45 | 6",shape="box"]; 46 | "34349649024" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 47 | 1 object, 24 bytes reachable 48 | this object in no reference cycles 49 | j.l.Long 50 | 16: value (long) 8 51 | 8",shape="box"]; 52 | "34349648256" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 53 | 1 object, 24 bytes reachable 54 | this object in no reference cycles 55 | j.l.Long 56 | 16: value (long) 4 57 | 4",shape="box"]; 58 | "34349648064" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 59 | 1 object, 24 bytes reachable 60 | this object in no reference cycles 61 | j.l.Long 62 | 16: value (long) 3 63 | 3",shape="box"]; 64 | "34349648832" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 65 | 1 object, 24 bytes reachable 66 | this object in no reference cycles 67 | j.l.Long 68 | 16: value (long) 7 69 | 7",shape="box"]; 70 | "34349649216" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 71 | 1 object, 24 bytes reachable 72 | this object in no reference cycles 73 | j.l.Long 74 | 16: value (long) 9 75 | 9",shape="box"]; 76 | "18840851456" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="56 bytes 77 | 11 objects, 296 bytes reachable 78 | this object in no reference cycles 79 | array of 10 j.l.Object 80 | 81 | [0 1 2 3 4 5 6 7 8 9]",shape="box"]; 82 | "34349648448" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 83 | 1 object, 24 bytes reachable 84 | this object in no reference cycles 85 | j.l.Long 86 | 16: value (long) 5 87 | 5",shape="box"]; 88 | "34349647872" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 89 | 1 object, 24 bytes reachable 90 | this object in no reference cycles 91 | j.l.Long 92 | 16: value (long) 2 93 | 2",shape="box"]; 94 | "34349647680" ["reachable-only-from"=18840849664,"scc-num-nodes"=1,label="24 bytes 95 | 1 object, 24 bytes reachable 96 | this object in no reference cycles 97 | j.l.Long 98 | 16: value (long) 1 99 | 1",shape="box"]; 100 | "18840849984" -> "18840850176" ["field-name"="edit",label="edit"]; 101 | "18840849984" -> "18840850304" ["field-name"="array",label="array"]; 102 | "18840849664" -> "18840849984" ["field-name"="root",label="root"]; 103 | "18840849664" -> "18840851456" ["field-name"="tail",label="tail"]; 104 | "18840851456" -> "34349647488" ["field-name"="[0]",label="[0]"]; 105 | "18840851456" -> "34349648640" ["field-name"="[6]",label="[6]"]; 106 | "18840851456" -> "34349649024" ["field-name"="[8]",label="[8]"]; 107 | "18840851456" -> "34349648256" ["field-name"="[4]",label="[4]"]; 108 | "18840851456" -> "34349648064" ["field-name"="[3]",label="[3]"]; 109 | "18840851456" -> "34349648832" ["field-name"="[7]",label="[7]"]; 110 | "18840851456" -> "34349649216" ["field-name"="[9]",label="[9]"]; 111 | "18840851456" -> "34349648448" ["field-name"="[5]",label="[5]"]; 112 | "18840851456" -> "34349647872" ["field-name"="[2]",label="[2]"]; 113 | "18840851456" -> "34349647680" ["field-name"="[1]",label="[1]"]; 114 | } -------------------------------------------------------------------------------- /doc/field-data-results/report-Linux-4.15.0-54-jdk-Oracle-11.0.3-clj-1.10.1.txt: -------------------------------------------------------------------------------- 1 | Scan for classes found 8793 2 | of which 8770 we will attempt to load and compare, but 3 | 23 we expect would cause problems in loading or comparison. 4 | Number of classes categorized by the phase in which an 5 | error occurred while loading and comparing (nil=no error): 6 | {nil 8769, :load-class 1} 7 | Wrote error info below after heading '# errors'. 8 | 8769 classes with no difference in their field data. 9 | Wrote details about differences for 0 classes below after heading '# differences'. 10 | Found 0 classes with different per-instance field lists according to different APIs. 11 | Wrote differences below after heading '# pif-diffs'. 12 | Found 0 classes with different sizes according to different JOL APIs. 13 | Wrote differences below after heading '# inst-size-diffs'. 14 | 15 | ############################################################ 16 | # errors 17 | ############################################################ 18 | {:load-class 19 | [{:class-name-str "org.apache.tools.ant.Task", 20 | :klass nil, 21 | :err-phase :load-class, 22 | :err #error { 23 | :cause "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 24 | :via 25 | [{:type java.lang.IllegalArgumentException 26 | :message "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 27 | :at [io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171]}] 28 | :trace 29 | [[io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171] 30 | [io.github.classgraph.ScanResultObject loadClass "ScanResultObject.java" 206] 31 | [io.github.classgraph.ClassInfo loadClass "ClassInfo.java" 2623] 32 | [jdk.internal.reflect.GeneratedMethodAccessor13 invoke nil -1] 33 | [jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43] 34 | [java.lang.reflect.Method invoke "Method.java" 566] 35 | [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167] 36 | [clojure.lang.Reflector invokeNoArgInstanceMember "Reflector.java" 438] 37 | [cljol.reflection_test_helpers$try_load_class invokeStatic "reflection_test_helpers.clj" 168] 38 | [cljol.reflection_test_helpers$try_load_class invoke "reflection_test_helpers.clj" 166] 39 | [cljol.reflection_test_helpers$load_classes_and_compare_results$fn__6632 invoke "reflection_test_helpers.clj" 199] 40 | [clojure.core$mapv$fn__8445 invoke "core.clj" 6912] 41 | [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49] 42 | [clojure.core.protocols$fn__8140 invokeStatic "protocols.clj" 75] 43 | [clojure.core.protocols$fn__8140 invoke "protocols.clj" 75] 44 | [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13] 45 | [clojure.core$reduce invokeStatic "core.clj" 6828] 46 | [clojure.core$mapv invokeStatic "core.clj" 6903] 47 | [clojure.core$mapv invoke "core.clj" 6903] 48 | [cljol.reflection_test_helpers$load_classes_and_compare_results invokeStatic "reflection_test_helpers.clj" 198] 49 | [cljol.reflection_test_helpers$load_classes_and_compare_results invoke "reflection_test_helpers.clj" 197] 50 | [cljol.reflection_test_helpers$report invokeStatic "reflection_test_helpers.clj" 253] 51 | [cljol.reflection_test_helpers$report invoke "reflection_test_helpers.clj" 242] 52 | [user$eval6689 invokeStatic "NO_SOURCE_FILE" 1] 53 | [user$eval6689 invoke "NO_SOURCE_FILE" 1] 54 | [clojure.lang.Compiler eval "Compiler.java" 7177] 55 | [clojure.lang.Compiler eval "Compiler.java" 7132] 56 | [clojure.core$eval invokeStatic "core.clj" 3214] 57 | [clojure.core$eval invoke "core.clj" 3210] 58 | [clojure.main$repl$read_eval_print__9086$fn__9089 invoke "main.clj" 437] 59 | [clojure.main$repl$read_eval_print__9086 invoke "main.clj" 437] 60 | [clojure.main$repl$fn__9095 invoke "main.clj" 458] 61 | [clojure.main$repl invokeStatic "main.clj" 458] 62 | [clojure.main$repl_opt invokeStatic "main.clj" 522] 63 | [clojure.main$main invokeStatic "main.clj" 667] 64 | [clojure.main$main doInvoke "main.clj" 616] 65 | [clojure.lang.RestFn invoke "RestFn.java" 397] 66 | [clojure.lang.AFn applyToHelper "AFn.java" 152] 67 | [clojure.lang.RestFn applyTo "RestFn.java" 132] 68 | [clojure.lang.Var applyTo "Var.java" 705] 69 | [clojure.main main "main.java" 40]]}, 70 | :diffs nil}]} 71 | 72 | ############################################################ 73 | # differences 74 | ############################################################ 75 | 76 | ############################################################ 77 | # pif-diffs 78 | ############################################################ 79 | 80 | ############################################################ 81 | # inst-size-diffs 82 | ############################################################ 83 | -------------------------------------------------------------------------------- /doc/field-data-results/report-Linux-4.15.0-54-jdk-Oracle-12.0.1-clj-1.10.1.txt: -------------------------------------------------------------------------------- 1 | Scan for classes found 8793 2 | of which 8770 we will attempt to load and compare, but 3 | 23 we expect would cause problems in loading or comparison. 4 | Number of classes categorized by the phase in which an 5 | error occurred while loading and comparing (nil=no error): 6 | {nil 8769, :load-class 1} 7 | Wrote error info below after heading '# errors'. 8 | 8769 classes with no difference in their field data. 9 | Wrote details about differences for 0 classes below after heading '# differences'. 10 | Found 0 classes with different per-instance field lists according to different APIs. 11 | Wrote differences below after heading '# pif-diffs'. 12 | Found 0 classes with different sizes according to different JOL APIs. 13 | Wrote differences below after heading '# inst-size-diffs'. 14 | 15 | ############################################################ 16 | # errors 17 | ############################################################ 18 | {:load-class 19 | [{:class-name-str "org.apache.tools.ant.Task", 20 | :klass nil, 21 | :err-phase :load-class, 22 | :err #error { 23 | :cause "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 24 | :via 25 | [{:type java.lang.IllegalArgumentException 26 | :message "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 27 | :at [io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171]}] 28 | :trace 29 | [[io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171] 30 | [io.github.classgraph.ScanResultObject loadClass "ScanResultObject.java" 206] 31 | [io.github.classgraph.ClassInfo loadClass "ClassInfo.java" 2623] 32 | [jdk.internal.reflect.GeneratedMethodAccessor13 invoke nil -1] 33 | [jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43] 34 | [java.lang.reflect.Method invoke "Method.java" 567] 35 | [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167] 36 | [clojure.lang.Reflector invokeNoArgInstanceMember "Reflector.java" 438] 37 | [cljol.reflection_test_helpers$try_load_class invokeStatic "reflection_test_helpers.clj" 168] 38 | [cljol.reflection_test_helpers$try_load_class invoke "reflection_test_helpers.clj" 166] 39 | [cljol.reflection_test_helpers$load_classes_and_compare_results$fn__6632 invoke "reflection_test_helpers.clj" 199] 40 | [clojure.core$mapv$fn__8445 invoke "core.clj" 6912] 41 | [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49] 42 | [clojure.core.protocols$fn__8140 invokeStatic "protocols.clj" 75] 43 | [clojure.core.protocols$fn__8140 invoke "protocols.clj" 75] 44 | [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13] 45 | [clojure.core$reduce invokeStatic "core.clj" 6828] 46 | [clojure.core$mapv invokeStatic "core.clj" 6903] 47 | [clojure.core$mapv invoke "core.clj" 6903] 48 | [cljol.reflection_test_helpers$load_classes_and_compare_results invokeStatic "reflection_test_helpers.clj" 198] 49 | [cljol.reflection_test_helpers$load_classes_and_compare_results invoke "reflection_test_helpers.clj" 197] 50 | [cljol.reflection_test_helpers$report invokeStatic "reflection_test_helpers.clj" 253] 51 | [cljol.reflection_test_helpers$report invoke "reflection_test_helpers.clj" 242] 52 | [user$eval6689 invokeStatic "NO_SOURCE_FILE" 1] 53 | [user$eval6689 invoke "NO_SOURCE_FILE" 1] 54 | [clojure.lang.Compiler eval "Compiler.java" 7177] 55 | [clojure.lang.Compiler eval "Compiler.java" 7132] 56 | [clojure.core$eval invokeStatic "core.clj" 3214] 57 | [clojure.core$eval invoke "core.clj" 3210] 58 | [clojure.main$repl$read_eval_print__9086$fn__9089 invoke "main.clj" 437] 59 | [clojure.main$repl$read_eval_print__9086 invoke "main.clj" 437] 60 | [clojure.main$repl$fn__9095 invoke "main.clj" 458] 61 | [clojure.main$repl invokeStatic "main.clj" 458] 62 | [clojure.main$repl_opt invokeStatic "main.clj" 522] 63 | [clojure.main$main invokeStatic "main.clj" 667] 64 | [clojure.main$main doInvoke "main.clj" 616] 65 | [clojure.lang.RestFn invoke "RestFn.java" 397] 66 | [clojure.lang.AFn applyToHelper "AFn.java" 152] 67 | [clojure.lang.RestFn applyTo "RestFn.java" 132] 68 | [clojure.lang.Var applyTo "Var.java" 705] 69 | [clojure.main main "main.java" 40]]}, 70 | :diffs nil}]} 71 | 72 | ############################################################ 73 | # differences 74 | ############################################################ 75 | 76 | ############################################################ 77 | # pif-diffs 78 | ############################################################ 79 | 80 | ############################################################ 81 | # inst-size-diffs 82 | ############################################################ 83 | -------------------------------------------------------------------------------- /doc/field-data-results/report-Linux-4.15.0-54-jdk-Oracle-9.0.4-clj-1.10.1.txt: -------------------------------------------------------------------------------- 1 | Scan for classes found 8793 2 | of which 8770 we will attempt to load and compare, but 3 | 23 we expect would cause problems in loading or comparison. 4 | Number of classes categorized by the phase in which an 5 | error occurred while loading and comparing (nil=no error): 6 | {nil 8769, :load-class 1} 7 | Wrote error info below after heading '# errors'. 8 | 8769 classes with no difference in their field data. 9 | Wrote details about differences for 0 classes below after heading '# differences'. 10 | Found 0 classes with different per-instance field lists according to different APIs. 11 | Wrote differences below after heading '# pif-diffs'. 12 | Found 0 classes with different sizes according to different JOL APIs. 13 | Wrote differences below after heading '# inst-size-diffs'. 14 | 15 | ############################################################ 16 | # errors 17 | ############################################################ 18 | {:load-class 19 | [{:class-name-str "org.apache.tools.ant.Task", 20 | :klass nil, 21 | :err-phase :load-class, 22 | :err #error { 23 | :cause "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 24 | :via 25 | [{:type java.lang.IllegalArgumentException 26 | :message "Could not load class org.apache.tools.ant.Task : java.lang.ClassNotFoundException: Could not load classfile for class org.apache.tools.ant.Task" 27 | :at [io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171]}] 28 | :trace 29 | [[io.github.classgraph.ScanResult loadClass "ScanResult.java" 1171] 30 | [io.github.classgraph.ScanResultObject loadClass "ScanResultObject.java" 206] 31 | [io.github.classgraph.ClassInfo loadClass "ClassInfo.java" 2623] 32 | [jdk.internal.reflect.GeneratedMethodAccessor13 invoke nil -1] 33 | [jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43] 34 | [java.lang.reflect.Method invoke "Method.java" 564] 35 | [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167] 36 | [clojure.lang.Reflector invokeNoArgInstanceMember "Reflector.java" 438] 37 | [cljol.reflection_test_helpers$try_load_class invokeStatic "reflection_test_helpers.clj" 168] 38 | [cljol.reflection_test_helpers$try_load_class invoke "reflection_test_helpers.clj" 166] 39 | [cljol.reflection_test_helpers$load_classes_and_compare_results$fn__6632 invoke "reflection_test_helpers.clj" 199] 40 | [clojure.core$mapv$fn__8445 invoke "core.clj" 6912] 41 | [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49] 42 | [clojure.core.protocols$fn__8140 invokeStatic "protocols.clj" 75] 43 | [clojure.core.protocols$fn__8140 invoke "protocols.clj" 75] 44 | [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13] 45 | [clojure.core$reduce invokeStatic "core.clj" 6828] 46 | [clojure.core$mapv invokeStatic "core.clj" 6903] 47 | [clojure.core$mapv invoke "core.clj" 6903] 48 | [cljol.reflection_test_helpers$load_classes_and_compare_results invokeStatic "reflection_test_helpers.clj" 198] 49 | [cljol.reflection_test_helpers$load_classes_and_compare_results invoke "reflection_test_helpers.clj" 197] 50 | [cljol.reflection_test_helpers$report invokeStatic "reflection_test_helpers.clj" 253] 51 | [cljol.reflection_test_helpers$report invoke "reflection_test_helpers.clj" 242] 52 | [user$eval6689 invokeStatic "NO_SOURCE_FILE" 1] 53 | [user$eval6689 invoke "NO_SOURCE_FILE" 1] 54 | [clojure.lang.Compiler eval "Compiler.java" 7177] 55 | [clojure.lang.Compiler eval "Compiler.java" 7132] 56 | [clojure.core$eval invokeStatic "core.clj" 3214] 57 | [clojure.core$eval invoke "core.clj" 3210] 58 | [clojure.main$repl$read_eval_print__9086$fn__9089 invoke "main.clj" 437] 59 | [clojure.main$repl$read_eval_print__9086 invoke "main.clj" 437] 60 | [clojure.main$repl$fn__9095 invoke "main.clj" 458] 61 | [clojure.main$repl invokeStatic "main.clj" 458] 62 | [clojure.main$repl_opt invokeStatic "main.clj" 522] 63 | [clojure.main$main invokeStatic "main.clj" 667] 64 | [clojure.main$main doInvoke "main.clj" 616] 65 | [clojure.lang.RestFn invoke "RestFn.java" 397] 66 | [clojure.lang.AFn applyToHelper "AFn.java" 152] 67 | [clojure.lang.RestFn applyTo "RestFn.java" 132] 68 | [clojure.lang.Var applyTo "Var.java" 705] 69 | [clojure.main main "main.java" 40]]}, 70 | :diffs nil}]} 71 | 72 | ############################################################ 73 | # differences 74 | ############################################################ 75 | 76 | ############################################################ 77 | # pif-diffs 78 | ############################################################ 79 | 80 | ############################################################ 81 | # inst-size-diffs 82 | ############################################################ 83 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-realized33.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "19212555456" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="24 bytes 4 | 34 objects, 936 bytes reachable 5 | this object in no reference cycles 6 | c.l.ArrayChunk 7 | 12: off (int) 0 8 | 16: end (int) 32 9 | 20: array (ref) -> 10 | val maybe realizes if str'ed",shape="box"]; 11 | "18626146112" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="144 bytes 12 | 33 objects, 912 bytes reachable 13 | this object in no reference cycles 14 | array of 32 j.l.Object 15 | 16 | [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 ...",shape="box"]; 17 | "19212556096" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="32 bytes 18 | 6 objects, 176 bytes reachable 19 | this object in no reference cycles 20 | c.l.LazySeq 21 | 12: _meta (ref) nil 22 | 16: fn (ref) -> 23 | 20: sv (ref) nil 24 | 24: s (ref) nil 25 | 28: lock (ref) -> 26 | val maybe realizes if str'ed",shape="box"]; 27 | "19212556352" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="24 bytes 28 | 3 objects, 96 bytes reachable 29 | this object in no reference cycles 30 | clojure.core$map$fn__5954 31 | 12: __methodImplCache (ref) nil 32 | 16: coll (ref) -> 33 | 20: f (ref) -> 34 | val maybe realizes if str'ed",shape="box"]; 35 | "19212222528" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="144 bytes 36 | 33 objects, 912 bytes reachable 37 | this object in no reference cycles 38 | array of 32 j.l.Object 39 | 40 | [33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 ...",shape="box"]; 41 | "18626145664" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="32 bytes 42 | at least 51 objects, 1488 bytes reachable 43 | this object in no reference cycles 44 | c.l.ChunkedCons 45 | 12: _meta (ref) nil 46 | 16: _hash (int) 0 47 | 20: _hasheq (int) 0 48 | 24: chunk (ref) -> 49 | 28: _more (ref) -> 50 | val maybe realizes if str'ed",shape="box"]; 51 | "19212556672" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="32 bytes 52 | 1 object, 32 bytes reachable 53 | this object in no reference cycles 54 | j.u.c.locks.ReentrantLock$NonfairSync 55 | 12: exclusiveOwnerThread (ref) nil 56 | 16: state (int) 0 57 | 20: head (ref) nil 58 | 24: tail (ref) nil 59 | val maybe realizes if str'ed",shape="box"]; 60 | "18626148160" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="16 bytes 61 | 1 object, 16 bytes reachable 62 | this object in no reference cycles 63 | user$fn__10278 64 | 12: __methodImplCache (ref) nil 65 | val maybe realizes if str'ed",shape="box"]; 66 | "19212556544" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="16 bytes 67 | 2 objects, 48 bytes reachable 68 | this object in no reference cycles 69 | j.u.c.locks.ReentrantLock 70 | 12: sync (ref) -> 71 | val maybe realizes if str'ed",shape="box"]; 72 | "18626145408" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,style="filled","my-unique-total-size"=2176,"my-unique-num-reachable-nodes"=78,label="32 bytes 73 | 78 objects, 2176 bytes reachable 74 | this object in no reference cycles 75 | c.l.LazySeq 76 | 12: _meta (ref) nil 77 | 16: fn (ref) nil 78 | 20: sv (ref) nil 79 | 24: s (ref) -> 80 | 28: lock (ref) nil 81 | val maybe realizes if str'ed",shape="box"]; 82 | "18626145920" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="24 bytes 83 | 34 objects, 936 bytes reachable 84 | this object in no reference cycles 85 | c.l.ArrayChunk 86 | 12: off (int) 0 87 | 16: end (int) 32 88 | 20: array (ref) -> 89 | val maybe realizes if str'ed",shape="box"]; 90 | "18626147264" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="32 bytes 91 | 42 objects, 1176 bytes reachable 92 | this object in no reference cycles 93 | c.l.LazySeq 94 | 12: _meta (ref) nil 95 | 16: fn (ref) nil 96 | 20: sv (ref) nil 97 | 24: s (ref) -> 98 | 28: lock (ref) nil 99 | val maybe realizes if str'ed",shape="box"]; 100 | "19212555648" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="56 bytes 101 | 1 object, 56 bytes reachable 102 | this object in no reference cycles 103 | c.l.LongRange 104 | 12: _meta (ref) nil 105 | 16: _hash (int) 0 106 | 20: _hasheq (int) 0 107 | 24: start (long) 64 108 | 32: end (long) 1000 109 | 40: step (long) 1 110 | 48: count (int) 936 111 | val maybe realizes if str'ed",shape="box"]; 112 | "19212556928" ["reachable-only-from"=18626145408,"scc-num-nodes"=1,label="32 bytes 113 | 41 objects, 1144 bytes reachable 114 | this object in no reference cycles 115 | c.l.ChunkedCons 116 | 12: _meta (ref) nil 117 | 16: _hash (int) 0 118 | 20: _hasheq (int) 0 119 | 24: chunk (ref) -> 120 | 28: _more (ref) -> 121 | val maybe realizes if str'ed",shape="box"]; 122 | "19212555456" -> "19212222528" ["field-name"="array",label="array"]; 123 | "19212556096" -> "19212556352" ["field-name"="fn",label="fn"]; 124 | "19212556096" -> "19212556544" ["field-name"="lock",label="lock"]; 125 | "19212556352" -> "19212555648" ["field-name"="coll",label="coll"]; 126 | "19212556352" -> "18626148160" ["field-name"="f",label="f"]; 127 | "18626145664" -> "18626145920" ["field-name"="chunk",label="chunk"]; 128 | "18626145664" -> "18626147264" ["field-name"="_more",label="_more"]; 129 | "19212556544" -> "19212556672" ["field-name"="sync",label="sync"]; 130 | "18626145408" -> "18626145664" ["field-name"="s",label="s"]; 131 | "18626145920" -> "18626146112" ["field-name"="array",label="array"]; 132 | "18626147264" -> "19212556928" ["field-name"="s",label="s"]; 133 | "19212556928" -> "19212555456" ["field-name"="chunk",label="chunk"]; 134 | "19212556928" -> "19212556096" ["field-name"="_more",label="_more"]; 135 | } -------------------------------------------------------------------------------- /src/clj/cljol/object_walk.clj: -------------------------------------------------------------------------------- 1 | (ns cljol.object-walk 2 | (:import (java.lang.reflect Field Method Modifier Constructor)) 3 | (:import (java.util IdentityHashMap)) 4 | (:import (org.openjdk.jol.info ClassLayout GraphLayout GraphPathRecord 5 | ClassData FieldData)) 6 | (:import (org.openjdk.jol.util ObjectUtils))) 7 | 8 | 9 | (set! *warn-on-reflection* true) 10 | 11 | 12 | ;; obj() is a private method of class GraphPathRecord, and its 13 | ;; constructor is also private. Use some Java hackery to call it 14 | ;; anyway, as long as the security policy in place allows us to. 15 | 16 | (def ^Class gpr-class (Class/forName "org.openjdk.jol.info.GraphPathRecord")) 17 | (def ^Method gpr-obj-method (.getDeclaredMethod gpr-class "obj" nil)) 18 | (.setAccessible gpr-obj-method true) 19 | (def ^Constructor gpr-ctor (first (.getDeclaredConstructors gpr-class))) 20 | (.setAccessible gpr-ctor true) 21 | 22 | 23 | (defn FieldData->map [^FieldData fd] 24 | (let [^Field ref-field (.refField fd)] 25 | {:field-name (.name fd) 26 | :type-class (.typeClass fd) 27 | :host-class (.hostClass fd) 28 | :is-contended? (.isContended fd) 29 | :contended-group (.contendedGroup fd) 30 | :ref-field ref-field 31 | :is-primitive? (. (. ref-field getType) isPrimitive) 32 | :vm-offset (.vmOffset fd)})) 33 | 34 | 35 | ;; Note that much of the information about a class returned by 36 | ;; ClassData->map can also be obtained via the Java reflection API in 37 | ;; the java.lang.reflect.* classes. At least one kind of information 38 | ;; that can be obtained from ClassData->map that cannot be gotten from 39 | ;; Java's reflection API is the :vm-offset of fields. 40 | 41 | (defn ClassData->map [^ClassData cd] 42 | (merge 43 | {:class-name (.name cd) 44 | :superclass (.superClass cd) 45 | :class-hierarchy (.classHierarchy cd) 46 | :is-array? (.isArray cd) 47 | :is-contended? (.isContended cd) 48 | :fields (->> (.fields cd) 49 | (map FieldData->map) 50 | (sort-by :vm-offset))} 51 | (if (.isArray cd) 52 | ;; info specific to array objects 53 | {:array-component-type (.arrayComponentType cd) 54 | :array-length (.arrayLength cd)} 55 | ;; info specific to non-array objects 56 | {;; oops are "Ordinary Object Pointers" according to this article: 57 | ;; https://www.baeldung.com/jvm-compressed-oops 58 | :oops-count (.oopsCount cd)}))) 59 | 60 | 61 | ;; This is intended to do what the JOL library's (version 0.9) 62 | ;; GraphLayout/parseInstance method does, except for a few 63 | ;; enhancements: 64 | 65 | ;; + Use an IdentityHashMap to track which objects have been visited 66 | ;; so far, rather than their numerical addresses. This enables 67 | ;; tracking object identities accurately, even if their addresses 68 | ;; change due to GC during the object walk. 69 | 70 | ;; + Do the walk using a stack on the heap, and not the Java call 71 | ;; stack, to perform the depth-first search of the object reference 72 | ;; graph. 73 | 74 | ;; + Add an option to prevent following references out of 75 | ;; java.lang.ref.Reference objects (and their subclasses). 76 | ;; Following such references and returning them creates a strong 77 | ;; reference to the objects found, preventing them from being 78 | ;; garbage collected, and in the case of 79 | ;; java.lang.ref.ReferenceQueue objects, many objects that likely 80 | ;; have little or no relationship to the ones that you want to see. 81 | 82 | (defn make-and-add-gpr-if-new [obj depth ^IdentityHashMap objects-found] 83 | (if (. objects-found containsKey obj) 84 | nil 85 | (let [;; I would use this if the GraphPathRecord constructor were 86 | ;; public. 87 | ;;gpr (GraphPathRecord. parent path (int depth) ^Object obj) 88 | ctor-args (object-array [nil nil (int depth) obj]) 89 | gpr (.newInstance gpr-ctor ctor-args)] 90 | (. objects-found put obj gpr) 91 | gpr))) 92 | 93 | 94 | (defn get-all-references [obj klass] 95 | (let [flds (->> (ClassData/parseClass klass) 96 | ClassData->map 97 | :fields 98 | (remove :is-primitive?) 99 | (map :ref-field))] 100 | (keep #(ObjectUtils/value obj %) flds))) 101 | 102 | 103 | (def empty-obj-array (object-array [])) 104 | 105 | 106 | (defn gpr->java-obj [^GraphPathRecord gpr] 107 | (.invoke gpr-obj-method gpr empty-obj-array)) 108 | 109 | 110 | (defn peel-references [gpr] 111 | (let [obj (gpr->java-obj gpr)] 112 | (if (nil? obj) 113 | nil 114 | (let [klass (class obj)] 115 | (if (. klass isArray) 116 | (if (. (. klass getComponentType) isPrimitive) 117 | nil 118 | (keep #(aget ^objects obj %) (range (count obj)))) 119 | (get-all-references obj klass)))))) 120 | 121 | 122 | (defn apply-conj! [transient-coll s] 123 | (loop [c transient-coll 124 | s (seq s)] 125 | (if-let [s (seq s)] 126 | (recur (conj! c (first s)) (rest s)) 127 | c))) 128 | 129 | 130 | (defn parse-instance-ids [stop-fn roots] 131 | (if (nil? roots) 132 | (throw (IllegalArgumentException. "Roots are null"))) 133 | (doseq [root roots] 134 | (if (nil? root) 135 | (throw (IllegalArgumentException. "Some root is null")))) 136 | (let [data (GraphLayout. (object-array roots)) 137 | objects-found (java.util.IdentityHashMap.)] 138 | (loop [cur-layer (keep #(make-and-add-gpr-if-new % 0 objects-found) roots) 139 | depth 1 140 | next-layer (transient [])] 141 | (if-let [s (seq cur-layer)] 142 | (let [gpr (first s) 143 | new-gprs (keep #(make-and-add-gpr-if-new % depth objects-found) 144 | (peel-references gpr))] 145 | (recur (rest s) depth (apply-conj! next-layer new-gprs))) 146 | ;; cur-layer is empty. Is next-layer non-empty? 147 | (let [nl (persistent! next-layer)] 148 | (if (seq nl) 149 | (recur nl (inc depth) (transient [])) 150 | ;; else both cur-layer and next-layer are empty 151 | ;; tbd: return something 152 | nil)))) 153 | {:data data 154 | :objects-found objects-found})) 155 | 156 | 157 | (comment 158 | 159 | (import '(org.openjdk.jol.info ClassLayout GraphLayout GraphPathRecord 160 | ClassData FieldData)) 161 | 162 | (def data (GraphLayout. (object-array [1 2 3]))) 163 | (type data) 164 | 165 | ) 166 | -------------------------------------------------------------------------------- /doc/README-clojure-map-size.md: -------------------------------------------------------------------------------- 1 | Machine tested on is macOS 10.14.6 with Oracle JDK 8 installed. 2 | 3 | ```bash 4 | $ java -version 5 | java version "1.8.0_192" 6 | Java(TM) SE Runtime Environment (build 1.8.0_192-b12) 7 | Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode) 8 | 9 | $ wc dark-corpus-2.edn 10 | 0 22982424 150383643 dark-corpus-2.edn 11 | ``` 12 | 13 | Link to EDN file dark-corpus-2.edn obtained from here on 2020-Oct-27: 14 | https://www.dropbox.com/s/urby2ahcwp58l4f/dark-corpus-2.edn?dl=0 15 | 16 | Found in this Github issue for the nippy project: 17 | https://github.com/ptaoussanis/nippy/issues/136 18 | 19 | Note: I recommend that you do _not_ try to run `d/sum` or `d/view` on 20 | the entire map `x1` created below. It would be very slow and consume 21 | a lot of memory, at least 5 times than the memory required by `x1` 22 | itself. These functions are not really designed for analyzing such 23 | large data structures. They are quick at analyzing smaller data 24 | structures, as demonstrated below. 25 | 26 | 27 | ```bash 28 | $ clj -Sdeps '{:deps {cljol/cljol {:git/url "https://github.com/jafingerhut/cljol" :sha "ff33d97f8375b4a0aaf758295e0aef7185ef9d6e"}}}' 29 | 30 | Clojure 1.10.1 31 | ``` 32 | 33 | ```clojure 34 | user=> (def fname "dark-corpus-2.edn") 35 | #'user/fname 36 | (require '[clojure.edn :as edn] 37 | '[clojure.java.io :as io]) 38 | nil 39 | user=> (require '[cljol.dig9 :as d]) 40 | Boxed math warning, cljol/ubergraph_extras.clj:128:27 - call: public static java.lang.Number clojure.lang.Numbers.divide(java.lang.Object,long). 41 | Boxed math warning, cljol/ubergraph_extras.clj:128:9 - call: public static boolean clojure.lang.Numbers.lt(long,java.lang.Object). 42 | nil 43 | user=> (def cljol-opts {:summary-options #{:size-breakdown :class-breakdown}}) 44 | #'user/cljol-opts 45 | 46 | ;; Read the EDN file 47 | 48 | user=> (def x1 (edn/read (java.io.PushbackReader. (io/reader fname)))) 49 | #'user/x1 50 | 51 | ;; Printed representation as EDN is about 150 million chars long 52 | user=> (count (str x1)) 53 | 150383643 54 | 55 | ;; Examine first key/value pair 56 | 57 | user=> (def kv0 (nth (seq x1) 0)) 58 | #'user/kv0 59 | 60 | user=> kv0 61 | [("profanity" "unholy") {"its" 2}] 62 | 63 | ;; 34 chars long as EDN 64 | 65 | user=> (count (str kv0)) 66 | 34 67 | 68 | ;; d/sum shows that it occupies 12 JVM objects in memory, totaling 360 69 | ;; bytes of memory 70 | 71 | user=> (def s1 (d/sum [kv0] cljol-opts)) 72 | # WARNING: Unable to attach Serviceability Agent. You can try again with escalated privileges. Two options: a) use -Djol.tryWithSudo=true to try with sudo; b) echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope 73 | 12 objects 74 | 11 references between them 75 | 360 bytes total in all objects 76 | no cycles 77 | number of objects of each size in bytes: 78 | ({:size-bytes 24, :num-objects 6, :total-size 144} 79 | {:size-bytes 32, :num-objects 3, :total-size 96} 80 | {:size-bytes 40, :num-objects 3, :total-size 120}) 81 | number and size of objects of each class: 82 | ({:total-size 24, :num-objects 1, :class "j.l.Long"} 83 | {:total-size 24, :num-objects 1, :class "[Ljava.lang.Object;"} 84 | {:total-size 32, :num-objects 1, :class "c.l.MapEntry"} 85 | {:total-size 32, :num-objects 1, :class "c.l.PersistentArrayMap"} 86 | {:total-size 72, :num-objects 3, :class "j.l.String"} 87 | {:total-size 80, :num-objects 2, :class "c.l.PersistentList"} 88 | {:total-size 96, :num-objects 3, :class "[C"}) 89 | 90 | 4 leaf objects (no references to other objects) 91 | 1 root nodes (no reference to them from other objects _in this graph_) 92 | #'user/s1 93 | 94 | ;; d/view will show a picture of these 12 objects, the size and class 95 | ;; of each, the value of their fields, and a printed representation of 96 | ;; each one's value. 97 | 98 | user=> (d/view [kv0]) 99 | nil 100 | 101 | ;; The number of memory bytes occupied by these objects divided by the 102 | ;; number of characters in the EDN representation averages to 10.6 103 | ;; memory bytes per character. 104 | 105 | user=> (/ 360.0 34) 106 | 10.588235294117647 107 | 108 | ;; Let us create a map that contains only the first 100 keys of the 109 | ;; full map. 110 | 111 | user=> (def x1-100 (select-keys x1 (take 100 (keys x1)))) 112 | #'user/x1-100 113 | 114 | ;; Its EDN representation is 4304 characters. It is represented in 115 | ;; memory by 1382 JVM objects, totaling 41,656 bytes. 116 | 117 | user=> (count (str x1-100)) 118 | 4304 119 | user=> (def s1 (d/sum [x1-100] cljol-opts)) 120 | 1382 objects 121 | 1640 references between them 122 | 41656 bytes total in all objects 123 | no cycles 124 | number of objects of each size in bytes: 125 | ({:size-bytes 16, :num-objects 8, :total-size 128} 126 | {:size-bytes 24, :num-objects 777, :total-size 18648} 127 | {:size-bytes 32, :num-objects 278, :total-size 8896} 128 | {:size-bytes 40, :num-objects 243, :total-size 9720} 129 | {:size-bytes 48, :num-objects 59, :total-size 2832} 130 | {:size-bytes 56, :num-objects 5, :total-size 280} 131 | {:size-bytes 64, :num-objects 3, :total-size 192} 132 | {:size-bytes 72, :num-objects 2, :total-size 144} 133 | {:size-bytes 80, :num-objects 2, :total-size 160} 134 | {:size-bytes 112, :num-objects 2, :total-size 224} 135 | {:size-bytes 144, :num-objects 3, :total-size 432}) 136 | number and size of objects of each class: 137 | ({:total-size 72, 138 | :num-objects 3, 139 | :class "c.l.PersistentHashMap$ArrayNode"} 140 | {:total-size 128, 141 | :num-objects 8, 142 | :class "j.u.c.atomic.AtomicReference"} 143 | {:total-size 200, :num-objects 5, :class "c.l.PersistentHashMap"} 144 | {:total-size 312, :num-objects 13, :class "j.l.Long"} 145 | {:total-size 432, 146 | :num-objects 3, 147 | :class "[Lclojure.lang.PersistentHashMap$INode;"} 148 | {:total-size 2064, 149 | :num-objects 86, 150 | :class "c.l.PersistentHashMap$BitmapIndexedNode"} 151 | {:total-size 3072, :num-objects 96, :class "c.l.PersistentArrayMap"} 152 | {:total-size 6696, :num-objects 182, :class "[Ljava.lang.Object;"} 153 | {:total-size 8000, :num-objects 200, :class "c.l.PersistentList"} 154 | {:total-size 9432, :num-objects 393, :class "j.l.String"} 155 | {:total-size 11248, :num-objects 393, :class "[C"}) 156 | 157 | 416 leaf objects (no references to other objects) 158 | 1 root nodes (no reference to them from other objects _in this graph_) 159 | #'user/s1 160 | 161 | ;; Average of 9.7 memory bytes per character of EDN for this map 162 | 163 | user=> (/ 41656.0 4304) 164 | 9.678438661710038 165 | ``` 166 | 167 | If we extrapolate for the entire map, whose EDN representation is 168 | 150,383,643 characters, times an average of 9.7 memory bytes per 169 | character, that is about 1.5 GBytes of memory. 170 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-and-nthrest-20-realized20.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18113330240" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="144 bytes 4 | 33 objects, 912 bytes reachable 5 | this object in no reference cycles 6 | array of 32 j.l.Object 7 | 8 | [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 ...",shape="box"]; 9 | "18113331392" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="32 bytes 10 | 6 objects, 176 bytes reachable 11 | this object in no reference cycles 12 | c.l.LazySeq 13 | 12: _meta (ref) nil 14 | 16: fn (ref) -> 15 | 20: sv (ref) nil 16 | 24: s (ref) nil 17 | 28: lock (ref) -> 18 | val maybe realizes if str'ed",shape="box"]; 19 | "18113332288" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="16 bytes 20 | 1 object, 16 bytes reachable 21 | this object in no reference cycles 22 | user$fn__10278 23 | 12: __methodImplCache (ref) nil 24 | val maybe realizes if str'ed",shape="box"]; 25 | "19212643008" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="24 bytes 26 | 45 objects, 1256 bytes reachable 27 | this object in no reference cycles 28 | array of 2 j.l.Object 29 | 30 | val maybe realizes if str'ed",shape="box"]; 31 | "17758699968" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="24 bytes 32 | 3 objects, 184 bytes reachable 33 | this object in no reference cycles 34 | c.l.PersistentVector$Node 35 | 12: edit (ref) -> 36 | 16: array (ref) -> 37 | val maybe realizes if str'ed",shape="box"]; 38 | "18113331840" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="56 bytes 39 | 1 object, 56 bytes reachable 40 | this object in no reference cycles 41 | c.l.LongRange 42 | 12: _meta (ref) nil 43 | 16: _hash (int) 0 44 | 20: _hasheq (int) 0 45 | 24: start (long) 32 46 | 32: end (long) 1000 47 | 40: step (long) 1 48 | 48: count (int) 968 49 | val maybe realizes if str'ed",shape="box"]; 50 | "18113329536" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="32 bytes 51 | 42 objects, 1176 bytes reachable 52 | this object in no reference cycles 53 | c.l.LazySeq 54 | 12: _meta (ref) nil 55 | 16: fn (ref) nil 56 | 20: sv (ref) nil 57 | 24: s (ref) -> 58 | 28: lock (ref) nil 59 | val maybe realizes if str'ed",shape="box"]; 60 | "18113332544" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="32 bytes 61 | 1 object, 32 bytes reachable 62 | this object in no reference cycles 63 | j.u.c.locks.ReentrantLock$NonfairSync 64 | 12: exclusiveOwnerThread (ref) nil 65 | 16: state (int) 0 66 | 20: head (ref) nil 67 | 24: tail (ref) nil 68 | val maybe realizes if str'ed",shape="box"]; 69 | "19212641920" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="32 bytes 70 | 41 objects, 1144 bytes reachable 71 | this object in no reference cycles 72 | c.l.ChunkedCons 73 | 12: _meta (ref) nil 74 | 16: _hash (int) 0 75 | 20: _hasheq (int) 0 76 | 24: chunk (ref) -> 77 | 28: _more (ref) -> 78 | val maybe realizes if str'ed",shape="box"]; 79 | "18113332416" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="16 bytes 80 | 2 objects, 48 bytes reachable 81 | this object in no reference cycles 82 | j.u.c.locks.ReentrantLock 83 | 12: sync (ref) -> 84 | val maybe realizes if str'ed",shape="box"]; 85 | "17758700160" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="16 bytes 86 | 1 object, 16 bytes reachable 87 | this object in no reference cycles 88 | j.u.c.atomic.AtomicReference 89 | 12: value (ref) .setAccessible failed 90 | val maybe realizes if str'ed",shape="box"]; 91 | "18113331648" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="24 bytes 92 | 3 objects, 96 bytes reachable 93 | this object in no reference cycles 94 | clojure.core$map$fn__5954 95 | 12: __methodImplCache (ref) nil 96 | 16: coll (ref) -> 97 | 20: f (ref) -> 98 | val maybe realizes if str'ed",shape="box"]; 99 | "18113329792" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="32 bytes 100 | 41 objects, 1144 bytes reachable 101 | this object in no reference cycles 102 | c.l.ChunkedCons 103 | 12: _meta (ref) nil 104 | 16: _hash (int) 0 105 | 20: _hasheq (int) 0 106 | 24: chunk (ref) -> 107 | 28: _more (ref) -> 108 | val maybe realizes if str'ed",shape="box"]; 109 | "17758700288" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="144 bytes 110 | 1 object, 144 bytes reachable 111 | this object in no reference cycles 112 | array of 32 j.l.Object 113 | 114 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 115 | "18113330048" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="24 bytes 116 | 34 objects, 936 bytes reachable 117 | this object in no reference cycles 118 | c.l.ArrayChunk 119 | 12: off (int) 0 120 | 16: end (int) 32 121 | 20: array (ref) -> 122 | val maybe realizes if str'ed",shape="box"]; 123 | "19212643200" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,style="filled","my-unique-total-size"=1480,"my-unique-num-reachable-nodes"=49,label="40 bytes 124 | 49 objects, 1480 bytes reachable 125 | this object in no reference cycles 126 | c.l.PersistentVector 127 | 12: _hash (int) 0 128 | 16: _hasheq (int) 0 129 | 20: cnt (int) 2 130 | 24: shift (int) 5 131 | 28: root (ref) -> 132 | 32: tail (ref) -> 133 | 36: _meta (ref) nil 134 | val maybe realizes if str'ed",shape="box"]; 135 | "19212642176" ["reachable-only-from"=19212643200,"scc-num-nodes"=1,label="24 bytes 136 | 34 objects, 936 bytes reachable 137 | this object in no reference cycles 138 | c.l.ArrayChunk 139 | 12: off (int) 20 140 | 16: end (int) 32 141 | 20: array (ref) -> 142 | val maybe realizes if str'ed",shape="box"]; 143 | "18113331392" -> "18113331648" ["field-name"="fn",label="fn"]; 144 | "18113331392" -> "18113332416" ["field-name"="lock",label="lock"]; 145 | "19212643008" -> "18113329536" ["field-name"="[0]",label="[0]"]; 146 | "19212643008" -> "19212641920" ["field-name"="[1]",label="[1]"]; 147 | "17758699968" -> "17758700160" ["field-name"="edit",label="edit"]; 148 | "17758699968" -> "17758700288" ["field-name"="array",label="array"]; 149 | "18113329536" -> "18113329792" ["field-name"="s",label="s"]; 150 | "19212641920" -> "19212642176" ["field-name"="chunk",label="chunk"]; 151 | "19212641920" -> "18113331392" ["field-name"="_more",label="_more"]; 152 | "18113332416" -> "18113332544" ["field-name"="sync",label="sync"]; 153 | "18113331648" -> "18113331840" ["field-name"="coll",label="coll"]; 154 | "18113331648" -> "18113332288" ["field-name"="f",label="f"]; 155 | "18113329792" -> "18113330048" ["field-name"="chunk",label="chunk"]; 156 | "18113329792" -> "18113331392" ["field-name"="_more",label="_more"]; 157 | "18113330048" -> "18113330240" ["field-name"="array",label="array"]; 158 | "19212643200" -> "17758699968" ["field-name"="root",label="root"]; 159 | "19212643200" -> "19212643008" ["field-name"="tail",label="tail"]; 160 | "19212642176" -> "18113330240" ["field-name"="array",label="array"]; 161 | } -------------------------------------------------------------------------------- /doc/images/lazy-fib-seq-vector-of-nthrest.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "18203120768" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 4 | 15 objects, 424 bytes reachable 5 | this object in no reference cycles 6 | array of 3 j.l.Object 7 | 8 | val maybe realizes if str'ed",shape="box"]; 9 | "34349647488" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | this object in no reference cycles 12 | j.l.Long 13 | 16: value (long) 0 14 | 0",shape="box"]; 15 | "18004806656" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 16 | 14 objects, 392 bytes reachable 17 | this object in no reference cycles 18 | c.l.LazySeq 19 | 12: _meta (ref) nil 20 | 16: fn (ref) nil 21 | 20: sv (ref) nil 22 | 24: s (ref) -> 23 | 28: lock (ref) nil 24 | val maybe realizes if str'ed",shape="box"]; 25 | "18004807168" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 26 | 11 objects, 304 bytes reachable 27 | this object in no reference cycles 28 | c.l.LazySeq 29 | 12: _meta (ref) nil 30 | 16: fn (ref) nil 31 | 20: sv (ref) nil 32 | 24: s (ref) -> 33 | 28: lock (ref) nil 34 | val maybe realizes if str'ed",shape="box"]; 35 | "17758699968" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 36 | 3 objects, 184 bytes reachable 37 | this object in no reference cycles 38 | c.l.PersistentVector$Node 39 | 12: edit (ref) -> 40 | 16: array (ref) -> 41 | val maybe realizes if str'ed",shape="box"]; 42 | "18004806912" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 43 | 13 objects, 360 bytes reachable 44 | this object in no reference cycles 45 | c.l.Cons 46 | 12: _meta (ref) nil 47 | 16: _hash (int) 0 48 | 20: _hasheq (int) 0 49 | 24: _first (ref) -> 50 | 28: _more (ref) -> 51 | val maybe realizes if str'ed",shape="box"]; 52 | "18004808640" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="16 bytes 53 | 2 objects, 48 bytes reachable 54 | this object in no reference cycles 55 | j.u.c.locks.ReentrantLock 56 | 12: sync (ref) -> 57 | val maybe realizes if str'ed",shape="box"]; 58 | "18004807680" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 59 | 9 objects, 240 bytes reachable 60 | this object in no reference cycles 61 | c.l.LazySeq 62 | 12: _meta (ref) nil 63 | 16: fn (ref) nil 64 | 20: sv (ref) nil 65 | 24: s (ref) -> 66 | 28: lock (ref) nil 67 | val maybe realizes if str'ed",shape="box"]; 68 | "18004808192" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 69 | 6 objects, 152 bytes reachable 70 | this object in no reference cycles 71 | c.l.LazySeq 72 | 12: _meta (ref) nil 73 | 16: fn (ref) -> 74 | 20: sv (ref) nil 75 | 24: s (ref) nil 76 | 28: lock (ref) -> 77 | val maybe realizes if str'ed",shape="box"]; 78 | "17758700160" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="16 bytes 79 | 1 object, 16 bytes reachable 80 | this object in no reference cycles 81 | j.u.c.atomic.AtomicReference 82 | 12: value (ref) .setAccessible failed 83 | val maybe realizes if str'ed",shape="box"]; 84 | "34349648064" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 85 | 1 object, 24 bytes reachable 86 | this object in no reference cycles 87 | j.l.Long 88 | 16: value (long) 3 89 | 3",shape="box"]; 90 | "18004808448" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 91 | 3 objects, 72 bytes reachable 92 | this object in no reference cycles 93 | user$fib_fn$fn__10167 94 | 12: __methodImplCache (ref) nil 95 | 16: a (ref) -> 96 | 20: b (ref) -> 97 | val maybe realizes if str'ed",shape="box"]; 98 | "18203121024" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,style="filled","my-unique-total-size"=648,"my-unique-num-reachable-nodes"=19,label="40 bytes 99 | 19 objects, 648 bytes reachable 100 | this object in no reference cycles 101 | c.l.PersistentVector 102 | 12: _hash (int) 0 103 | 16: _hasheq (int) 0 104 | 20: cnt (int) 3 105 | 24: shift (int) 5 106 | 28: root (ref) -> 107 | 32: tail (ref) -> 108 | 36: _meta (ref) nil 109 | val maybe realizes if str'ed",shape="box"]; 110 | "18004808768" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 111 | 1 object, 32 bytes reachable 112 | this object in no reference cycles 113 | j.u.c.locks.ReentrantLock$NonfairSync 114 | 12: exclusiveOwnerThread (ref) nil 115 | 16: state (int) 0 116 | 20: head (ref) nil 117 | 24: tail (ref) nil 118 | val maybe realizes if str'ed",shape="box"]; 119 | "17758700288" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="144 bytes 120 | 1 object, 144 bytes reachable 121 | this object in no reference cycles 122 | array of 32 j.l.Object 123 | 124 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 125 | "18004807936" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 126 | 8 objects, 208 bytes reachable 127 | this object in no reference cycles 128 | c.l.Cons 129 | 12: _meta (ref) nil 130 | 16: _hash (int) 0 131 | 20: _hasheq (int) 0 132 | 24: _first (ref) -> 133 | 28: _more (ref) -> 134 | val maybe realizes if str'ed",shape="box"]; 135 | "34349647872" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 136 | 1 object, 24 bytes reachable 137 | this object in no reference cycles 138 | j.l.Long 139 | 16: value (long) 2 140 | 2",shape="box"]; 141 | "18004807424" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="32 bytes 142 | 10 objects, 272 bytes reachable 143 | this object in no reference cycles 144 | c.l.Cons 145 | 12: _meta (ref) nil 146 | 16: _hash (int) 0 147 | 20: _hasheq (int) 0 148 | 24: _first (ref) -> 149 | 28: _more (ref) -> 150 | val maybe realizes if str'ed",shape="box"]; 151 | "34349647680" ["reachable-only-from"=18203121024,"scc-num-nodes"=1,label="24 bytes 152 | 1 object, 24 bytes reachable 153 | this object in no reference cycles 154 | j.l.Long 155 | 16: value (long) 1 156 | 1",shape="box"]; 157 | "18203120768" -> "18004806656" ["field-name"="[0]",label="[0]"]; 158 | "18203120768" -> "18004807424" ["field-name"="[1]",label="[1]"]; 159 | "18203120768" -> "18004807936" ["field-name"="[2]",label="[2]"]; 160 | "18004806656" -> "18004806912" ["field-name"="s",label="s"]; 161 | "18004807168" -> "18004807424" ["field-name"="s",label="s"]; 162 | "17758699968" -> "17758700160" ["field-name"="edit",label="edit"]; 163 | "17758699968" -> "17758700288" ["field-name"="array",label="array"]; 164 | "18004806912" -> "34349647488" ["field-name"="_first",label="_first"]; 165 | "18004806912" -> "18004807168" ["field-name"="_more",label="_more"]; 166 | "18004808640" -> "18004808768" ["field-name"="sync",label="sync"]; 167 | "18004807680" -> "18004807936" ["field-name"="s",label="s"]; 168 | "18004808192" -> "18004808448" ["field-name"="fn",label="fn"]; 169 | "18004808192" -> "18004808640" ["field-name"="lock",label="lock"]; 170 | "18004808448" -> "34349647872" ["field-name"="a",label="a"]; 171 | "18004808448" -> "34349648064" ["field-name"="b",label="b"]; 172 | "18203121024" -> "17758699968" ["field-name"="root",label="root"]; 173 | "18203121024" -> "18203120768" ["field-name"="tail",label="tail"]; 174 | "18004807936" -> "34349647680" ["field-name"="_first",label="_first"]; 175 | "18004807936" -> "18004808192" ["field-name"="_more",label="_more"]; 176 | "18004807424" -> "34349647680" ["field-name"="_first",label="_first"]; 177 | "18004807424" -> "18004807680" ["field-name"="_more",label="_more"]; 178 | } -------------------------------------------------------------------------------- /doc/images/chunked-seq-and-nthrest-40-realized33.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "19212555456" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 4 | 34 objects, 936 bytes reachable 5 | this object in no reference cycles 6 | c.l.ArrayChunk 7 | 12: off (int) 0 8 | 16: end (int) 32 9 | 20: array (ref) -> 10 | val maybe realizes if str'ed",shape="box"]; 11 | "18626146112" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="144 bytes 12 | 33 objects, 912 bytes reachable 13 | this object in no reference cycles 14 | array of 32 j.l.Object 15 | 16 | [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 ...",shape="box"]; 17 | "19212556096" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 18 | 6 objects, 176 bytes reachable 19 | this object in no reference cycles 20 | c.l.LazySeq 21 | 12: _meta (ref) nil 22 | 16: fn (ref) -> 23 | 20: sv (ref) nil 24 | 24: s (ref) nil 25 | 28: lock (ref) -> 26 | val maybe realizes if str'ed",shape="box"]; 27 | "19212556352" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 28 | 3 objects, 96 bytes reachable 29 | this object in no reference cycles 30 | clojure.core$map$fn__5954 31 | 12: __methodImplCache (ref) nil 32 | 16: coll (ref) -> 33 | 20: f (ref) -> 34 | val maybe realizes if str'ed",shape="box"]; 35 | "19212222528" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="144 bytes 36 | 33 objects, 912 bytes reachable 37 | this object in no reference cycles 38 | array of 32 j.l.Object 39 | 40 | [33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 4 ...",shape="box"]; 41 | "18626145664" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 42 | at least 51 objects, 1488 bytes reachable 43 | this object in no reference cycles 44 | c.l.ChunkedCons 45 | 12: _meta (ref) nil 46 | 16: _hash (int) 0 47 | 20: _hasheq (int) 0 48 | 24: chunk (ref) -> 49 | 28: _more (ref) -> 50 | val maybe realizes if str'ed",shape="box"]; 51 | "19212556672" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 52 | 1 object, 32 bytes reachable 53 | this object in no reference cycles 54 | j.u.c.locks.ReentrantLock$NonfairSync 55 | 12: exclusiveOwnerThread (ref) nil 56 | 16: state (int) 0 57 | 20: head (ref) nil 58 | 24: tail (ref) nil 59 | val maybe realizes if str'ed",shape="box"]; 60 | "17758699968" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 61 | 3 objects, 184 bytes reachable 62 | this object in no reference cycles 63 | c.l.PersistentVector$Node 64 | 12: edit (ref) -> 65 | 16: array (ref) -> 66 | val maybe realizes if str'ed",shape="box"]; 67 | "18626148160" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="16 bytes 68 | 1 object, 16 bytes reachable 69 | this object in no reference cycles 70 | user$fn__10278 71 | 12: __methodImplCache (ref) nil 72 | val maybe realizes if str'ed",shape="box"]; 73 | "18661195968" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 74 | at least 51 objects, 1496 bytes reachable 75 | this object in no reference cycles 76 | array of 2 j.l.Object 77 | 78 | val maybe realizes if str'ed",shape="box"]; 79 | "17758700160" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="16 bytes 80 | 1 object, 16 bytes reachable 81 | this object in no reference cycles 82 | j.u.c.atomic.AtomicReference 83 | 12: value (ref) .setAccessible failed 84 | val maybe realizes if str'ed",shape="box"]; 85 | "18661195136" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 86 | 34 objects, 936 bytes reachable 87 | this object in no reference cycles 88 | c.l.ArrayChunk 89 | 12: off (int) 8 90 | 16: end (int) 32 91 | 20: array (ref) -> 92 | val maybe realizes if str'ed",shape="box"]; 93 | "19212556544" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="16 bytes 94 | 2 objects, 48 bytes reachable 95 | this object in no reference cycles 96 | j.u.c.locks.ReentrantLock 97 | 12: sync (ref) -> 98 | val maybe realizes if str'ed",shape="box"]; 99 | "18626145408" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 100 | at least 51 objects, 1496 bytes reachable 101 | this object in no reference cycles 102 | c.l.LazySeq 103 | 12: _meta (ref) nil 104 | 16: fn (ref) nil 105 | 20: sv (ref) nil 106 | 24: s (ref) -> 107 | 28: lock (ref) nil 108 | val maybe realizes if str'ed",shape="box"]; 109 | "18626145920" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="24 bytes 110 | 34 objects, 936 bytes reachable 111 | this object in no reference cycles 112 | c.l.ArrayChunk 113 | 12: off (int) 0 114 | 16: end (int) 32 115 | 20: array (ref) -> 116 | val maybe realizes if str'ed",shape="box"]; 117 | "18626147264" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 118 | 42 objects, 1176 bytes reachable 119 | this object in no reference cycles 120 | c.l.LazySeq 121 | 12: _meta (ref) nil 122 | 16: fn (ref) nil 123 | 20: sv (ref) nil 124 | 24: s (ref) -> 125 | 28: lock (ref) nil 126 | val maybe realizes if str'ed",shape="box"]; 127 | "17758700288" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="144 bytes 128 | 1 object, 144 bytes reachable 129 | this object in no reference cycles 130 | array of 32 j.l.Object 131 | 132 | [nil nil nil nil nil nil nil nil nil nil nil nil n ...",shape="box"]; 133 | "18661194880" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 134 | 41 objects, 1144 bytes reachable 135 | this object in no reference cycles 136 | c.l.ChunkedCons 137 | 12: _meta (ref) nil 138 | 16: _hash (int) 0 139 | 20: _hasheq (int) 0 140 | 24: chunk (ref) -> 141 | 28: _more (ref) -> 142 | val maybe realizes if str'ed",shape="box"]; 143 | "19212555648" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="56 bytes 144 | 1 object, 56 bytes reachable 145 | this object in no reference cycles 146 | c.l.LongRange 147 | 12: _meta (ref) nil 148 | 16: _hash (int) 0 149 | 20: _hasheq (int) 0 150 | 24: start (long) 64 151 | 32: end (long) 1000 152 | 40: step (long) 1 153 | 48: count (int) 936 154 | val maybe realizes if str'ed",shape="box"]; 155 | "18661196160" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,style="filled","my-unique-total-size"=2480,"my-unique-num-reachable-nodes"=85,label="40 bytes 156 | 85 objects, 2480 bytes reachable 157 | this object in no reference cycles 158 | c.l.PersistentVector 159 | 12: _hash (int) 0 160 | 16: _hasheq (int) 0 161 | 20: cnt (int) 2 162 | 24: shift (int) 5 163 | 28: root (ref) -> 164 | 32: tail (ref) -> 165 | 36: _meta (ref) nil 166 | val maybe realizes if str'ed",shape="box"]; 167 | "19212556928" ["reachable-only-from"=18661196160,"scc-num-nodes"=1,label="32 bytes 168 | 41 objects, 1144 bytes reachable 169 | this object in no reference cycles 170 | c.l.ChunkedCons 171 | 12: _meta (ref) nil 172 | 16: _hash (int) 0 173 | 20: _hasheq (int) 0 174 | 24: chunk (ref) -> 175 | 28: _more (ref) -> 176 | val maybe realizes if str'ed",shape="box"]; 177 | "19212555456" -> "19212222528" ["field-name"="array",label="array"]; 178 | "19212556096" -> "19212556352" ["field-name"="fn",label="fn"]; 179 | "19212556096" -> "19212556544" ["field-name"="lock",label="lock"]; 180 | "19212556352" -> "19212555648" ["field-name"="coll",label="coll"]; 181 | "19212556352" -> "18626148160" ["field-name"="f",label="f"]; 182 | "18626145664" -> "18626145920" ["field-name"="chunk",label="chunk"]; 183 | "18626145664" -> "18626147264" ["field-name"="_more",label="_more"]; 184 | "17758699968" -> "17758700160" ["field-name"="edit",label="edit"]; 185 | "17758699968" -> "17758700288" ["field-name"="array",label="array"]; 186 | "18661195968" -> "18626145408" ["field-name"="[0]",label="[0]"]; 187 | "18661195968" -> "18661194880" ["field-name"="[1]",label="[1]"]; 188 | "18661195136" -> "19212222528" ["field-name"="array",label="array"]; 189 | "19212556544" -> "19212556672" ["field-name"="sync",label="sync"]; 190 | "18626145408" -> "18626145664" ["field-name"="s",label="s"]; 191 | "18626145920" -> "18626146112" ["field-name"="array",label="array"]; 192 | "18626147264" -> "19212556928" ["field-name"="s",label="s"]; 193 | "18661194880" -> "18661195136" ["field-name"="chunk",label="chunk"]; 194 | "18661194880" -> "19212556096" ["field-name"="_more",label="_more"]; 195 | "18661196160" -> "17758699968" ["field-name"="root",label="root"]; 196 | "18661196160" -> "18661195968" ["field-name"="tail",label="tail"]; 197 | "19212556928" -> "19212555456" ["field-name"="chunk",label="chunk"]; 198 | "19212556928" -> "19212556096" ["field-name"="_more",label="_more"]; 199 | } -------------------------------------------------------------------------------- /src/generate/gen/generate.clj: -------------------------------------------------------------------------------- 1 | (ns gen.generate 2 | (:gen-class) 3 | (:import (java.io File)) 4 | (:require [cljol.dig9 :as d] 5 | [cljol.version-info :as ver] 6 | [clojure.string :as str] 7 | [clojure.data.int-map :as im])) 8 | 9 | 10 | (defn fname [s opts] 11 | (str (:output-dir opts) File/separator 12 | s "-" (get @ver/version-data :stack-desc) ".dot")) 13 | 14 | 15 | ;; Copied from clojure.math.combinatorics namespace 16 | 17 | (defn unchunk 18 | "Given a sequence that may have chunks, return a sequence that is 1-at-a-time 19 | lazy with no chunks. Chunks are good for efficiency when the data items are 20 | small, but when being processed via map, for example, a reference is kept to 21 | every function result in the chunk until the entire chunk has been processed, 22 | which increases the amount of memory in use that cannot be garbage 23 | collected." 24 | [s] 25 | (lazy-seq 26 | (when (seq s) 27 | (cons (first s) (unchunk (rest s)))))) 28 | 29 | 30 | (defn gen [obj name opts] 31 | (println "Generating" name "...") 32 | (d/write-dot-file [obj] (fname name opts) opts)) 33 | 34 | 35 | (defn parse-args [args] 36 | (when (not= 1 (count args)) 37 | (binding [*out* *err*] 38 | (println (format "usage: %s " *file*)) 39 | (System/exit 1))) 40 | {:output-dir (nth args 0)}) 41 | 42 | 43 | (def opts-show-field-values 44 | {:node-label-functions 45 | [;;d/address-hex 46 | ;;d/address-decimal 47 | d/size-bytes 48 | d/total-size-bytes 49 | d/scc-size 50 | d/class-description 51 | d/field-values 52 | ;;d/path-to-object 53 | d/javaobj->str 54 | ;;d/non-realizing-javaobj->str 55 | ]}) 56 | 57 | ;; Avoid calling clojure.core/str or any similar function on a lazy 58 | ;; sequence if you do not want it to be realized. 59 | (def opts-dont-realize-values 60 | {:node-label-functions 61 | [;;d/address-hex 62 | ;;d/address-decimal 63 | d/size-bytes 64 | d/total-size-bytes 65 | d/scc-size 66 | d/class-description 67 | d/field-values 68 | ;;d/path-to-object 69 | ;;d/javaobj->str 70 | d/non-realizing-javaobj->str]}) 71 | 72 | (defn -main [& args] 73 | (let [cmdline-opts (parse-args args) 74 | opts-default cmdline-opts 75 | opts-show-field-values (merge cmdline-opts opts-show-field-values) 76 | opts-dont-realize-values (merge cmdline-opts opts-dont-realize-values) 77 | opts opts-show-field-values] 78 | 79 | (let [map1 (let [x :a y :b] {x y y x})] 80 | (gen map1 "map1" opts)) 81 | 82 | (gen {"a" 1 "foobar" 3.5} "map2" opts) 83 | 84 | (let [vec10 (vec (range 10)) 85 | unboxed-vec10 (apply vector-of :long (range 10)) 86 | arr10 (long-array (range 10))] 87 | (gen vec10 "vec10" opts) 88 | (gen unboxed-vec10 "unboxed-vec10" opts) 89 | (gen arr10 "arr10" opts)) 90 | 91 | ;; Interesting! Self-loop for optimal memory efficiency! 92 | (let [opts opts-dont-realize-values 93 | repeat-42 (repeat 42) 94 | repeat-10-a (repeat 10 "a")] 95 | (gen repeat-42 "unlimited-repeat-unrealized" opts) 96 | (println "(take 1 repeat-42)" (take 1 repeat-42)) 97 | (gen repeat-42 "unlimited-repeat-realized1" opts) 98 | (println "(take 50 repeat-42)" (take 50 repeat-42)) 99 | (gen repeat-42 "unlimited-repeat-realized50" opts) 100 | 101 | (gen repeat-10-a "repeat-10-unrealized" opts) 102 | (println "(take 1 repeat-10-a)" (take 1 repeat-10-a)) 103 | (gen repeat-10-a "repeat-10-realized1" opts) 104 | (println "(take 4 repeat-10-a)" (take 4 repeat-10-a)) 105 | (gen repeat-10-a "repeat-10-realized4" opts)) 106 | 107 | (let [opts opts-dont-realize-values 108 | unchunked-range (unchunk (range 50))] 109 | 110 | (gen unchunked-range "unchunked-range-unrealized" opts) 111 | (println "(take 1 unchunked-range)" (take 1 unchunked-range)) 112 | (gen unchunked-range "unchunked-range-realized1" opts) 113 | (println "(take 5 unchunked-range)" (take 5 unchunked-range)) 114 | (gen unchunked-range "unchunked-range-realized5" opts)) 115 | 116 | (let [opts opts-dont-realize-values 117 | fib-fn (fn fib-fn [a b] 118 | (lazy-seq (cons a (fib-fn b (+ a b))))) 119 | fib-seq (fib-fn 0 1)] 120 | (gen fib-seq "lazy-fibonacci-unrealized" opts) 121 | (println "(take 1 fib-seq)" (take 1 fib-seq)) 122 | (gen fib-seq "lazy-fibonacci-realized1" opts) 123 | (println "(take 2 fib-seq)" (take 2 fib-seq)) 124 | (gen fib-seq "lazy-fibonacci-realized2" opts) 125 | (println "(take 3 fib-seq)" (take 3 fib-seq)) 126 | (gen fib-seq "lazy-fibonacci-realized3" opts) 127 | 128 | (gen [fib-seq (nthrest fib-seq 1) (nthrest fib-seq 2) 129 | (nthrest fib-seq 3) (nthrest fib-seq 4)] 130 | "lazy-fibonacci-vector-of-nthrest" opts)) 131 | 132 | (let [opts opts-show-field-values] 133 | (gen ["food has only 8-bit characters" 134 | "f\u1234od has non-8-bit characters!"] 135 | "strings-8-bit-and-not" opts)) 136 | 137 | (let [opts opts-show-field-values] 138 | (gen (list (/ 3 2) 12345678901234567890N true) 139 | "various-types1" opts) 140 | (gen (object-array [true 12345678901234567890N "hammock time!"]) 141 | "various-types2" opts)) 142 | 143 | (let [opts opts-show-field-values] 144 | (gen ["food has only 8-bit characters" 145 | "f\u1234od has non-8-bit characters!"] 146 | "strings-8-bit-and-not" opts)) 147 | 148 | (let [opts (merge opts-show-field-values 149 | {:max-value-len 200})] 150 | (doseq [[name min-char-code max-char-code] 151 | [ 152 | ["s00-00" 0 1] 153 | ["s00" 0 32] 154 | ["s01" 32 64] 155 | ["s02" 64 96] 156 | ["s03" 96 128] 157 | ["s04" 128 160] 158 | ["s05" 160 192] 159 | ["s06" 192 224] 160 | ["s07" 224 256] 161 | ["s08" 256 271] 162 | ["s09" (- 65536 32) 65536] 163 | ["s09-00" (- 65536 32) (- 65536 31)] 164 | ["s09-01" (- 65536 31) (- 65536 30)] 165 | ["s09-02" (- 65536 30) (- 65536 29)] 166 | ["s09-03" (- 65536 29) (- 65536 28)] 167 | ["s09-04" (- 65536 28) (- 65536 27)] 168 | 169 | ["s09-29" (- 65536 3) (- 65536 2)] 170 | ["s09-30" (- 65536 2) (- 65536 1)] 171 | ["s09-31" (- 65536 1) (- 65536 0)] 172 | 173 | ["s10" (- 0xd800 16) (+ 0xd800 16)] 174 | ["s11" (- 0xd800 16) (+ 0xd800 16)] 175 | ["s12" (- 0xe000 16) (+ 0xe000 16)] 176 | ]] 177 | (let [s (apply str (map char (range min-char-code max-char-code)))] 178 | (gen s name opts) 179 | (gen (seq s) (str "c" name) opts)))) 180 | 181 | ;; Show effects of a lazy sequence being generated on demand, 182 | ;; without chunking. 183 | 184 | ;; Creators of sequences: repeat, range 185 | 186 | ;; Functions in core.clj that have special code to handle chunked 187 | ;; sequences in: 188 | ;; reduce1 189 | ;; reverse - from reduce1 190 | ;; A _lot_ of functions in core use reduce1 191 | 192 | ;; sequence 193 | ;; map map-indexed filter remove (inherited from filter) keep keep-indexed 194 | ;; random-sample - from filter 195 | ;; doseq 196 | ;; iterator-seq 197 | 198 | )) 199 | 200 | 201 | (comment 202 | 203 | (do 204 | 205 | (require 'gen.generate) 206 | (require '[cljol.dig9 :as d] 207 | '[clojure.data.int-map :as im] 208 | '[ubergraph.core :as uber]) 209 | (in-ns 'gen.generate) 210 | 211 | ) 212 | 213 | (defn pairs-2i-to-inc [n] 214 | (for [i (range 0 (* 2 n) 2)] 215 | [(* 2 i) (inc (* 2 i))])) 216 | (pairs-2i-to-inc 5) 217 | 218 | (def map12 (into {} (pairs-2i-to-inc 12))) 219 | (def intmap12 (into (im/int-map) (pairs-2i-to-inc 12))) 220 | (d/view [map12]) 221 | (d/view [intmap12]) 222 | 223 | (def map1e4 (into {} (pairs-2i-to-inc 1e4))) 224 | (def intmap1e4 (into (im/int-map) (pairs-2i-to-inc 1e4))) 225 | (println "map1e4") 226 | (def g (d/sum [map1e4])) 227 | (println "intmap1e4") 228 | (def g (d/sum [intmap1e4])) 229 | 230 | (def e1 *e) 231 | (use 'clojure.repl) 232 | (pst e1 100) 233 | 234 | (def set1e4 (set (range 1e4))) 235 | (def intset1e4 (set (into (im/int-set) (range 1e4)))) 236 | (def denseintset1e4 (set (into (im/dense-int-set) (range 1e4)))) 237 | (println "set1e4") 238 | (def g (d/sum [set1e4])) 239 | ;; 14,795 objects, 464,088 bytes 240 | (println "intset1e4") 241 | (def g (d/sum [intset1e4])) 242 | ;; 821 objects, 33,016 bytes 243 | (println "denseintset1e4") 244 | (def g (d/sum [denseintset1e4])) 245 | ;; 25 objects, 2,056 bytes 246 | (d/view-graph g) 247 | 248 | 249 | ;; Examine the data structures of ubergraph library using itself 250 | 251 | (def gr1 (uber/multidigraph [1 {:label "n1"}] 252 | [2 {:label "n2"}] 253 | [1 2 {:label "edge12"}])) 254 | (uber/pprint gr1) 255 | (d/view [gr1]) 256 | 257 | 258 | (def opts opts-dont-realize-values) 259 | (def repeat-42 (repeat 42)) 260 | (def o1 (d/consistent-reachable-objmaps [repeat-42])) 261 | (def u1 (d/object-graph->ubergraph o1 opts)) 262 | (def g1 (d/add-viz-attributes u1 opts)) 263 | (def r1 (d/graph-of-reachable-objects [repeat-42] opts)) 264 | (d/view [repeat-42] opts) 265 | (println (take 10 repeat-42)) 266 | 267 | (defn lazy-fib-sequence* [a b] 268 | (lazy-seq (let [sum (+ a b)] 269 | (cons sum (lazy-fib-sequence* b sum))))) 270 | 271 | (defn lazy-fib-sequence [a b] 272 | (cons a (cons b (lazy-seq (lazy-fib-sequence* a b))))) 273 | 274 | (def fib-seq (lazy-fib-sequence 1 1)) 275 | 276 | 277 | (defn fib-fn [a b] 278 | (lazy-seq (cons a (fib-fn b (+ a b))))) 279 | 280 | (def fib-seq (fib-fn 0 1)) 281 | 282 | (def opts opts-dont-realize-values) 283 | (d/view [fib-seq] opts) 284 | (println (take 2 fib-seq)) 285 | (println (take 3 fib-seq)) 286 | (println (take 4 fib-seq)) 287 | (println (take 7 fib-seq)) 288 | 289 | (d/view [fib-seq (nthrest fib-seq 1) (nthrest fib-seq 2) 290 | (nthrest fib-seq 3) (nthrest fib-seq 4)] opts) 291 | 292 | (def lazy4 (seq (vec (range 100)))) 293 | (d/view [lazy4] opts-dont-realize-values) 294 | (take 1 lazy4) 295 | (take 4 lazy4) 296 | 297 | 298 | ;; It might be nice to figure out how seq on a map is implemented some 299 | ;; day. Not extremely obvious to me yet from the figures here, but 300 | ;; not surprising as I have never looked at the implementation. 301 | 302 | (let [opts opts-dont-realize-values 303 | m {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20} 304 | s (seq m)] 305 | (gen s "seq-on-map1-unrealized" opts) 306 | (doall (take 1 s)) 307 | (gen s "seq-on-map1-realized1" opts) 308 | (doall (take 3 s)) 309 | (gen s "seq-on-map1-realized3" opts) 310 | (println "(vec s)" (vec s))) 311 | 312 | ) 313 | -------------------------------------------------------------------------------- /doc/images/chunked-seq-realized1.dot: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [layout="dot",rankdir="LR"]; 3 | "34349650944" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 4 | 1 object, 24 bytes reachable 5 | this object in no reference cycles 6 | j.l.Long 7 | 16: value (long) 18 8 | 18",shape="box"]; 9 | "34349651712" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 10 | 1 object, 24 bytes reachable 11 | this object in no reference cycles 12 | j.l.Long 13 | 16: value (long) 22 14 | 22",shape="box"]; 15 | "34349650368" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 16 | 1 object, 24 bytes reachable 17 | this object in no reference cycles 18 | j.l.Long 19 | 16: value (long) 15 20 | 15",shape="box"]; 21 | "18094456128" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="56 bytes 22 | 1 object, 56 bytes reachable 23 | this object in no reference cycles 24 | c.l.LongRange 25 | 12: _meta (ref) nil 26 | 16: _hash (int) 0 27 | 20: _hasheq (int) 0 28 | 24: start (long) 32 29 | 32: end (long) 1000 30 | 40: step (long) 1 31 | 48: count (int) 968 32 | val maybe realizes if str'ed",shape="box"]; 33 | "34349649600" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 34 | 1 object, 24 bytes reachable 35 | this object in no reference cycles 36 | j.l.Long 37 | 16: value (long) 11 38 | 11",shape="box"]; 39 | "34349649408" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 40 | 1 object, 24 bytes reachable 41 | this object in no reference cycles 42 | j.l.Long 43 | 16: value (long) 10 44 | 10",shape="box"]; 45 | "18094454080" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 46 | 41 objects, 1144 bytes reachable 47 | this object in no reference cycles 48 | c.l.ChunkedCons 49 | 12: _meta (ref) nil 50 | 16: _hash (int) 0 51 | 20: _hasheq (int) 0 52 | 24: chunk (ref) -> 53 | 28: _more (ref) -> 54 | val maybe realizes if str'ed",shape="box"]; 55 | "34349651904" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 56 | 1 object, 24 bytes reachable 57 | this object in no reference cycles 58 | j.l.Long 59 | 16: value (long) 23 60 | 23",shape="box"]; 61 | "34349649984" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 62 | 1 object, 24 bytes reachable 63 | this object in no reference cycles 64 | j.l.Long 65 | 16: value (long) 13 66 | 13",shape="box"]; 67 | "34349653248" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 68 | 1 object, 24 bytes reachable 69 | this object in no reference cycles 70 | j.l.Long 71 | 16: value (long) 30 72 | 30",shape="box"]; 73 | "18094455680" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 74 | 6 objects, 176 bytes reachable 75 | this object in no reference cycles 76 | c.l.LazySeq 77 | 12: _meta (ref) nil 78 | 16: fn (ref) -> 79 | 20: sv (ref) nil 80 | 24: s (ref) nil 81 | 28: lock (ref) -> 82 | val maybe realizes if str'ed",shape="box"]; 83 | "34349652480" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 84 | 1 object, 24 bytes reachable 85 | this object in no reference cycles 86 | j.l.Long 87 | 16: value (long) 26 88 | 26",shape="box"]; 89 | "18094454528" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="144 bytes 90 | 33 objects, 912 bytes reachable 91 | this object in no reference cycles 92 | array of 32 j.l.Object 93 | 94 | [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 ...",shape="box"]; 95 | "34349650752" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 96 | 1 object, 24 bytes reachable 97 | this object in no reference cycles 98 | j.l.Long 99 | 16: value (long) 17 100 | 17",shape="box"]; 101 | "18094456704" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="16 bytes 102 | 2 objects, 48 bytes reachable 103 | this object in no reference cycles 104 | j.u.c.locks.ReentrantLock 105 | 12: sync (ref) -> 106 | val maybe realizes if str'ed",shape="box"]; 107 | "34349653056" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 108 | 1 object, 24 bytes reachable 109 | this object in no reference cycles 110 | j.l.Long 111 | 16: value (long) 29 112 | 29",shape="box"]; 113 | "34349648640" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 114 | 1 object, 24 bytes reachable 115 | this object in no reference cycles 116 | j.l.Long 117 | 16: value (long) 6 118 | 6",shape="box"]; 119 | "34349653632" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 120 | 1 object, 24 bytes reachable 121 | this object in no reference cycles 122 | j.l.Long 123 | 16: value (long) 32 124 | 32",shape="box"]; 125 | "34349649024" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 126 | 1 object, 24 bytes reachable 127 | this object in no reference cycles 128 | j.l.Long 129 | 16: value (long) 8 130 | 8",shape="box"]; 131 | "34349648256" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 132 | 1 object, 24 bytes reachable 133 | this object in no reference cycles 134 | j.l.Long 135 | 16: value (long) 4 136 | 4",shape="box"]; 137 | "34349651328" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 138 | 1 object, 24 bytes reachable 139 | this object in no reference cycles 140 | j.l.Long 141 | 16: value (long) 20 142 | 20",shape="box"]; 143 | "34349649792" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 144 | 1 object, 24 bytes reachable 145 | this object in no reference cycles 146 | j.l.Long 147 | 16: value (long) 12 148 | 12",shape="box"]; 149 | "34349652096" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 150 | 1 object, 24 bytes reachable 151 | this object in no reference cycles 152 | j.l.Long 153 | 16: value (long) 24 154 | 24",shape="box"]; 155 | "34349653440" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 156 | 1 object, 24 bytes reachable 157 | this object in no reference cycles 158 | j.l.Long 159 | 16: value (long) 31 160 | 31",shape="box"]; 161 | "34349648064" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 162 | 1 object, 24 bytes reachable 163 | this object in no reference cycles 164 | j.l.Long 165 | 16: value (long) 3 166 | 3",shape="box"]; 167 | "34349652864" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 168 | 1 object, 24 bytes reachable 169 | this object in no reference cycles 170 | j.l.Long 171 | 16: value (long) 28 172 | 28",shape="box"]; 173 | "34349652288" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 174 | 1 object, 24 bytes reachable 175 | this object in no reference cycles 176 | j.l.Long 177 | 16: value (long) 25 178 | 25",shape="box"]; 179 | "34349648832" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 180 | 1 object, 24 bytes reachable 181 | this object in no reference cycles 182 | j.l.Long 183 | 16: value (long) 7 184 | 7",shape="box"]; 185 | "18094456832" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="32 bytes 186 | 1 object, 32 bytes reachable 187 | this object in no reference cycles 188 | j.u.c.locks.ReentrantLock$NonfairSync 189 | 12: exclusiveOwnerThread (ref) nil 190 | 16: state (int) 0 191 | 20: head (ref) nil 192 | 24: tail (ref) nil 193 | val maybe realizes if str'ed",shape="box"]; 194 | "18094453824" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,style="filled","my-unique-total-size"=1176,"my-unique-num-reachable-nodes"=42,label="32 bytes 195 | 42 objects, 1176 bytes reachable 196 | this object in no reference cycles 197 | c.l.LazySeq 198 | 12: _meta (ref) nil 199 | 16: fn (ref) nil 200 | 20: sv (ref) nil 201 | 24: s (ref) -> 202 | 28: lock (ref) nil 203 | val maybe realizes if str'ed",shape="box"]; 204 | "34349649216" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 205 | 1 object, 24 bytes reachable 206 | this object in no reference cycles 207 | j.l.Long 208 | 16: value (long) 9 209 | 9",shape="box"]; 210 | "34349652672" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 211 | 1 object, 24 bytes reachable 212 | this object in no reference cycles 213 | j.l.Long 214 | 16: value (long) 27 215 | 27",shape="box"]; 216 | "18094456576" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="16 bytes 217 | 1 object, 16 bytes reachable 218 | this object in no reference cycles 219 | user$fn__10278 220 | 12: __methodImplCache (ref) nil 221 | val maybe realizes if str'ed",shape="box"]; 222 | "34349651136" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 223 | 1 object, 24 bytes reachable 224 | this object in no reference cycles 225 | j.l.Long 226 | 16: value (long) 19 227 | 19",shape="box"]; 228 | "34349650176" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 229 | 1 object, 24 bytes reachable 230 | this object in no reference cycles 231 | j.l.Long 232 | 16: value (long) 14 233 | 14",shape="box"]; 234 | "34349648448" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 235 | 1 object, 24 bytes reachable 236 | this object in no reference cycles 237 | j.l.Long 238 | 16: value (long) 5 239 | 5",shape="box"]; 240 | "34349650560" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 241 | 1 object, 24 bytes reachable 242 | this object in no reference cycles 243 | j.l.Long 244 | 16: value (long) 16 245 | 16",shape="box"]; 246 | "34349647872" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 247 | 1 object, 24 bytes reachable 248 | this object in no reference cycles 249 | j.l.Long 250 | 16: value (long) 2 251 | 2",shape="box"]; 252 | "18094455936" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 253 | 3 objects, 96 bytes reachable 254 | this object in no reference cycles 255 | clojure.core$map$fn__5954 256 | 12: __methodImplCache (ref) nil 257 | 16: coll (ref) -> 258 | 20: f (ref) -> 259 | val maybe realizes if str'ed",shape="box"]; 260 | "34349651520" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 261 | 1 object, 24 bytes reachable 262 | this object in no reference cycles 263 | j.l.Long 264 | 16: value (long) 21 265 | 21",shape="box"]; 266 | "34349647680" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 267 | 1 object, 24 bytes reachable 268 | this object in no reference cycles 269 | j.l.Long 270 | 16: value (long) 1 271 | 1",shape="box"]; 272 | "18094454336" ["reachable-only-from"=18094453824,"scc-num-nodes"=1,label="24 bytes 273 | 34 objects, 936 bytes reachable 274 | this object in no reference cycles 275 | c.l.ArrayChunk 276 | 12: off (int) 0 277 | 16: end (int) 32 278 | 20: array (ref) -> 279 | val maybe realizes if str'ed",shape="box"]; 280 | "18094454080" -> "18094454336" ["field-name"="chunk",label="chunk"]; 281 | "18094454080" -> "18094455680" ["field-name"="_more",label="_more"]; 282 | "18094455680" -> "18094455936" ["field-name"="fn",label="fn"]; 283 | "18094455680" -> "18094456704" ["field-name"="lock",label="lock"]; 284 | "18094454528" -> "34349650944" ["field-name"="[17]",label="[17]"]; 285 | "18094454528" -> "34349651712" ["field-name"="[21]",label="[21]"]; 286 | "18094454528" -> "34349650368" ["field-name"="[14]",label="[14]"]; 287 | "18094454528" -> "34349649600" ["field-name"="[10]",label="[10]"]; 288 | "18094454528" -> "34349649408" ["field-name"="[9]",label="[9]"]; 289 | "18094454528" -> "34349651904" ["field-name"="[22]",label="[22]"]; 290 | "18094454528" -> "34349649984" ["field-name"="[12]",label="[12]"]; 291 | "18094454528" -> "34349653248" ["field-name"="[29]",label="[29]"]; 292 | "18094454528" -> "34349652480" ["field-name"="[25]",label="[25]"]; 293 | "18094454528" -> "34349650752" ["field-name"="[16]",label="[16]"]; 294 | "18094454528" -> "34349653056" ["field-name"="[28]",label="[28]"]; 295 | "18094454528" -> "34349648640" ["field-name"="[5]",label="[5]"]; 296 | "18094454528" -> "34349653632" ["field-name"="[31]",label="[31]"]; 297 | "18094454528" -> "34349649024" ["field-name"="[7]",label="[7]"]; 298 | "18094454528" -> "34349648256" ["field-name"="[3]",label="[3]"]; 299 | "18094454528" -> "34349651328" ["field-name"="[19]",label="[19]"]; 300 | "18094454528" -> "34349649792" ["field-name"="[11]",label="[11]"]; 301 | "18094454528" -> "34349652096" ["field-name"="[23]",label="[23]"]; 302 | "18094454528" -> "34349653440" ["field-name"="[30]",label="[30]"]; 303 | "18094454528" -> "34349648064" ["field-name"="[2]",label="[2]"]; 304 | "18094454528" -> "34349652864" ["field-name"="[27]",label="[27]"]; 305 | "18094454528" -> "34349652288" ["field-name"="[24]",label="[24]"]; 306 | "18094454528" -> "34349648832" ["field-name"="[6]",label="[6]"]; 307 | "18094454528" -> "34349649216" ["field-name"="[8]",label="[8]"]; 308 | "18094454528" -> "34349652672" ["field-name"="[26]",label="[26]"]; 309 | "18094454528" -> "34349651136" ["field-name"="[18]",label="[18]"]; 310 | "18094454528" -> "34349650176" ["field-name"="[13]",label="[13]"]; 311 | "18094454528" -> "34349648448" ["field-name"="[4]",label="[4]"]; 312 | "18094454528" -> "34349650560" ["field-name"="[15]",label="[15]"]; 313 | "18094454528" -> "34349647872" ["field-name"="[1]",label="[1]"]; 314 | "18094454528" -> "34349651520" ["field-name"="[20]",label="[20]"]; 315 | "18094454528" -> "34349647680" ["field-name"="[0]",label="[0]"]; 316 | "18094456704" -> "18094456832" ["field-name"="sync",label="sync"]; 317 | "18094453824" -> "18094454080" ["field-name"="s",label="s"]; 318 | "18094455936" -> "18094456128" ["field-name"="coll",label="coll"]; 319 | "18094455936" -> "18094456576" ["field-name"="f",label="f"]; 320 | "18094454336" -> "18094454528" ["field-name"="array",label="array"]; 321 | } -------------------------------------------------------------------------------- /doc/jol-dev-ideas/jol-0-9-plus-parseInstanceIds.diff: -------------------------------------------------------------------------------- 1 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java 2 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java Fri Sep 22 17:29:07 2017 +0200 3 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java Fri Jul 12 00:55:21 2019 -0700 4 | @@ -37,6 +37,8 @@ 5 | import java.io.PrintWriter; 6 | import java.io.StringWriter; 7 | import java.util.*; 8 | +import java.util.function.BiConsumer; 9 | +import java.util.function.Predicate; 10 | 11 | /** 12 | * Holds the object graph layout info. 13 | @@ -44,7 +46,19 @@ 14 | public class GraphLayout { 15 | 16 | /** 17 | - * Parse the object graph starting from the given instance. 18 | + * Parse the object graph starting from the given instances. 19 | + * 20 | + * During this method's execution, if objects parsed are being 21 | + * modified in the references that exist between them by other 22 | + * threads, there is not much that can be guaranteed about the set 23 | + * of parsed objects that are returned, except that it will 24 | + * include the objects in the roots parameter. 25 | + * 26 | + * Even if no objects being parsed are modified while parsing, it 27 | + * is possible for the set of objects returned to be incomplete. 28 | + * This can occur if multiple objects happen to have the same 29 | + * address at different times during the call, because of garbage 30 | + * collection moving them around in memory. 31 | * 32 | * @param roots root instances to start from 33 | * @return object graph 34 | @@ -65,6 +79,98 @@ 35 | return data; 36 | } 37 | 38 | + /** 39 | + * Parse the object graph starting from the given instances. 40 | + * 41 | + * See parseInstance() documentation for notes on the few 42 | + * guarantees that can be made by this method if other threads are 43 | + * modifying references between parsed objects. 44 | + * 45 | + * This method puts the collection of objects found as keys into 46 | + * an IdentityHashMap. The value in the map corresponding to each 47 | + * object is a GraphPathRecord describing the object. You can get 48 | + * that IdentityHashMap using the objectsFound() method. 49 | + * 50 | + * If no parsed objects are modified, at least not in the 51 | + * references that exist between them, during this method's 52 | + * execution, the set of objects that are the keys of the 53 | + * IdentityHashMap created should always be correct. 54 | + * 55 | + * The address map returned by the addresses() method should be 56 | + * correct if no garbage collection occurred during this method's 57 | + * execution, but there is nothing returned to indicate whether 58 | + * they are complete or not. This is similar to what can happen 59 | + * in the parseInstance() method, but for this method at least the 60 | + * IdentityHashMap should have the correct set of objects as keys, 61 | + * and you may try calling the createAddressSnapshot() method on this 62 | + * GraphLayout instance to have a better chance of getting a 63 | + * correct address map. 64 | + * 65 | + * The addressSnapshot() method allocates much less memory than 66 | + * this method does, and is thus less likely to trigger a garbage 67 | + * collection run. 68 | + * 69 | + * @param stop null gives the default behavior of walking all references of all objects. By supplying a Predicate, every time the walk reaches an object o, it will call the stop.test(o) predicate, and avoid following any references out of o to other objects. 70 | + * @param roots root instances to start from 71 | + * @return object graph 72 | + */ 73 | + public static GraphLayout parseInstanceIds(Predicate stop, 74 | + Object... roots) { 75 | + if (roots == null) { 76 | + throw new IllegalArgumentException("Roots are null"); 77 | + } 78 | + for (Object root : roots) { 79 | + if (root == null) { 80 | + throw new IllegalArgumentException("Some root is null"); 81 | + } 82 | + } 83 | + GraphLayout data = new GraphLayout(roots); 84 | + GraphWalker walker = new GraphWalker(roots); 85 | + walker.addVisitor(data.visitorById()); 86 | + walker.walk(stop); 87 | + if (data.createAddressSnapshot(1)) { 88 | + data.createAddressMap(); 89 | + } 90 | + return data; 91 | + } 92 | + 93 | + /** 94 | + * Try to create a consistent snapshot of object addresses. Very 95 | + * little memory -- a constant amount, regardless of the size of 96 | + * the object graph -- should be allocated by this method, to 97 | + * minimize its chances of triggering a garbage collection while 98 | + * it executes, and maximize the likelihood that the set of 99 | + * addresses returned represent a consistent set. No other 100 | + * attempt is made to prevent GC from occurring while this method 101 | + * executes, so even though it does do one pass over the objects 102 | + * to record all of their current addresses, and then another pass 103 | + * to see if any of them have moved, retrying that pair of steps 104 | + * up to maxAttempts times if anything has moved, it is still 105 | + * possible (though it seems unlikely) that this method could 106 | + * return true (indicating success) even though the addresses do 107 | + * not represent a consistent snapshot. 108 | + * 109 | + * @param maxAttempts Maximum number of times to attempt creating a snapshot before returning false. 110 | + * @return true for success, false for failure 111 | + */ 112 | + public boolean createAddressSnapshot(int maxAttempts) { 113 | + boolean attemptSucceeded = false; 114 | + int attempts = 0; 115 | + do { 116 | + ++attempts; 117 | + recordAddresses(); 118 | + Exception e = validateAddresses(); 119 | + if (e == null) { 120 | + attemptSucceeded = true; 121 | + break; 122 | + } 123 | + } while (attempts <= maxAttempts); 124 | + if (attemptSucceeded) { 125 | + createAddressMap(); 126 | + } 127 | + return attemptSucceeded; 128 | + } 129 | + 130 | private static final Comparator> CLASS_COMPARATOR = new Comparator>() { 131 | @Override 132 | public int compare(Class o1, Class o2) { 133 | @@ -75,6 +181,7 @@ 134 | private final Set> classes = new TreeSet>(CLASS_COMPARATOR); 135 | private final Multiset> classSizes = new Multiset>(); 136 | private final Multiset> classCounts = new Multiset>(); 137 | + private final IdentityHashMap objectsFound = new IdentityHashMap(); 138 | private final SortedMap addresses = new TreeMap(); 139 | 140 | private final String description; 141 | @@ -121,6 +228,89 @@ 142 | } 143 | } 144 | 145 | + private GraphVisitor visitorById() { 146 | + return new GraphVisitor() { 147 | + @Override 148 | + public void visit(GraphPathRecord gpr) { 149 | + addRecordById(gpr); 150 | + } 151 | + }; 152 | + } 153 | + 154 | + private void addRecordById(GraphPathRecord gpr) { 155 | + objectsFound.put(gpr.obj(), gpr); 156 | + 157 | + Class klass = gpr.klass(); 158 | + classes.add(klass); 159 | + classCounts.add(klass); 160 | + totalCount++; 161 | + try { 162 | + long size = gpr.size(); 163 | + totalSize += size; 164 | + classSizes.add(klass, size); 165 | + } catch (Exception e) { 166 | + classSizes.add(klass, 0); 167 | + } 168 | + } 169 | + 170 | + public void recordAddresses() { 171 | + /* Iterate through objectsFound, recording the address of each 172 | + * object once. This method should allocate only a small 173 | + * constant amount of memory, to maximize the chances that it 174 | + * does not trigger a GC while it executes, and thus the 175 | + * addresses recorded represent a consistent 'snapshot' across 176 | + * all objects at one point in time. */ 177 | + BiConsumer recordOneAddress = 178 | + new BiConsumer() { 179 | + @Override 180 | + public void accept(Object o, Object gprArg) { 181 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 182 | + gpr.recordAddress(); 183 | + } 184 | + }; 185 | + objectsFound.forEach(recordOneAddress); 186 | + } 187 | + 188 | + public Exception validateAddresses() { 189 | + /* Same comments apply here as for method 190 | + * recordAddresses(). */ 191 | + BiConsumer validateOneAddress = 192 | + new BiConsumer() { 193 | + @Override 194 | + public void accept(Object o, Object gprArg) { 195 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 196 | + if (o != gpr.obj()) { 197 | + String msg = "Found GraphPathRecord with key object " + o.getClass() + " different than the object " + gpr.obj().getClass() + " in the GraphPathRecord"; 198 | + throw new IllegalArgumentException(msg); 199 | + } 200 | + long curAddress = VM.current().addressOf(o); 201 | + if (curAddress != gpr.address()) { 202 | + String msg = "Found GraphPathRecord with current address " + curAddress + " different than recorded address " + gpr.address() + " object " + gpr.obj().getClass(); 203 | + throw new IllegalArgumentException(msg); 204 | + } 205 | + } 206 | + }; 207 | + try { 208 | + objectsFound.forEach(validateOneAddress); 209 | + } catch (Exception e) { 210 | + return e; 211 | + } 212 | + return null; 213 | + } 214 | + 215 | + public void createAddressMap() { 216 | + BiConsumer addOneAddress = 217 | + new BiConsumer() { 218 | + @Override 219 | + public void accept(Object o, Object gprArg) { 220 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 221 | + addresses.put(gpr.address(), gpr); 222 | + } 223 | + }; 224 | + addresses.clear(); 225 | + objectsFound.forEach(addOneAddress); 226 | + } 227 | + 228 | /** 229 | * Subtract another layout data from the current one. 230 | * This method does not change the current data object, but produces another one. 231 | @@ -238,6 +428,17 @@ 232 | } 233 | 234 | /** 235 | + * Answer the IdentityHashMap of object references to their 236 | + * associated GraphPathRecord. 237 | + * 238 | + * @return sorted set of addresses 239 | + * @see #record(long) 240 | + */ 241 | + public IdentityHashMap objectsFound() { 242 | + return objectsFound; 243 | + } 244 | + 245 | + /** 246 | * Answer the set of addresses for the discovered objects 247 | * 248 | * @return sorted set of addresses 249 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java 250 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java Fri Sep 22 17:29:07 2017 +0200 251 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java Fri Jul 12 00:55:21 2019 -0700 252 | @@ -38,7 +38,9 @@ 253 | private final int depth; 254 | private final Object obj; 255 | private final long size; 256 | - private final long address; 257 | + /* TBD: Does making field 'address' volatile be better for 258 | + * thread-safety of this class? */ 259 | + private volatile long address; 260 | private String toString; 261 | 262 | GraphPathRecord(GraphPathRecord parent, String path, int depth, Object obj) { 263 | @@ -74,6 +76,11 @@ 264 | return address; 265 | } 266 | 267 | + public long recordAddress() { 268 | + this.address = VM.current().addressOf(obj); 269 | + return address; 270 | + } 271 | + 272 | public String objToString() { 273 | String v = toString; 274 | if (v == null) { 275 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java 276 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java Fri Sep 22 17:29:07 2017 +0200 277 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java Fri Jul 12 00:55:21 2019 -0700 278 | @@ -34,6 +34,7 @@ 279 | import java.util.IdentityHashMap; 280 | import java.util.List; 281 | import java.util.Set; 282 | +import java.util.function.Predicate; 283 | 284 | /** 285 | * Basic class to walk object graphs. 286 | @@ -57,6 +58,10 @@ 287 | } 288 | 289 | public void walk() { 290 | + walk(null); 291 | + } 292 | + 293 | + public void walk(Predicate stop) { 294 | List curLayer = new ArrayList(); 295 | List newLayer = new ArrayList(); 296 | 297 | @@ -74,7 +79,7 @@ 298 | while (!curLayer.isEmpty()) { 299 | newLayer.clear(); 300 | for (GraphPathRecord next : curLayer) { 301 | - for (GraphPathRecord ref : peelReferences(next)) { 302 | + for (GraphPathRecord ref : peelReferences(next, stop)) { 303 | if (visited.add(ref.obj())) { 304 | visitObject(ref); 305 | newLayer.add(ref); 306 | @@ -92,7 +97,8 @@ 307 | } 308 | } 309 | 310 | - private List peelReferences(GraphPathRecord r) { 311 | + private List peelReferences(GraphPathRecord r, 312 | + Predicate stop) { 313 | Object o = r.obj(); 314 | 315 | if (o == null) { 316 | @@ -105,6 +111,12 @@ 317 | return Collections.emptyList(); 318 | } 319 | 320 | + if (stop != null && stop.test(o)) { 321 | + // The caller does not want any references from this 322 | + // object walked. 323 | + return Collections.emptyList(); 324 | + } 325 | + 326 | List result = new ArrayList(); 327 | 328 | if (o.getClass().isArray()) { 329 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Eclipse Public License - v 2.0 2 | 3 | THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE 4 | PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION 5 | OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. 6 | 7 | 1. DEFINITIONS 8 | 9 | "Contribution" means: 10 | 11 | a) in the case of the initial Contributor, the initial content 12 | Distributed under this Agreement, and 13 | 14 | b) in the case of each subsequent Contributor: 15 | i) changes to the Program, and 16 | ii) additions to the Program; 17 | where such changes and/or additions to the Program originate from 18 | and are Distributed by that particular Contributor. A Contribution 19 | "originates" from a Contributor if it was added to the Program by 20 | such Contributor itself or anyone acting on such Contributor's behalf. 21 | Contributions do not include changes or additions to the Program that 22 | are not Modified Works. 23 | 24 | "Contributor" means any person or entity that Distributes the Program. 25 | 26 | "Licensed Patents" mean patent claims licensable by a Contributor which 27 | are necessarily infringed by the use or sale of its Contribution alone 28 | or when combined with the Program. 29 | 30 | "Program" means the Contributions Distributed in accordance with this 31 | Agreement. 32 | 33 | "Recipient" means anyone who receives the Program under this Agreement 34 | or any Secondary License (as applicable), including Contributors. 35 | 36 | "Derivative Works" shall mean any work, whether in Source Code or other 37 | form, that is based on (or derived from) the Program and for which the 38 | editorial revisions, annotations, elaborations, or other modifications 39 | represent, as a whole, an original work of authorship. 40 | 41 | "Modified Works" shall mean any work in Source Code or other form that 42 | results from an addition to, deletion from, or modification of the 43 | contents of the Program, including, for purposes of clarity any new file 44 | in Source Code form that contains any contents of the Program. Modified 45 | Works shall not include works that contain only declarations, 46 | interfaces, types, classes, structures, or files of the Program solely 47 | in each case in order to link to, bind by name, or subclass the Program 48 | or Modified Works thereof. 49 | 50 | "Distribute" means the acts of a) distributing or b) making available 51 | in any manner that enables the transfer of a copy. 52 | 53 | "Source Code" means the form of a Program preferred for making 54 | modifications, including but not limited to software source code, 55 | documentation source, and configuration files. 56 | 57 | "Secondary License" means either the GNU General Public License, 58 | Version 2.0, or any later versions of that license, including any 59 | exceptions or additional permissions as identified by the initial 60 | Contributor. 61 | 62 | 2. GRANT OF RIGHTS 63 | 64 | a) Subject to the terms of this Agreement, each Contributor hereby 65 | grants Recipient a non-exclusive, worldwide, royalty-free copyright 66 | license to reproduce, prepare Derivative Works of, publicly display, 67 | publicly perform, Distribute and sublicense the Contribution of such 68 | Contributor, if any, and such Derivative Works. 69 | 70 | b) Subject to the terms of this Agreement, each Contributor hereby 71 | grants Recipient a non-exclusive, worldwide, royalty-free patent 72 | license under Licensed Patents to make, use, sell, offer to sell, 73 | import and otherwise transfer the Contribution of such Contributor, 74 | if any, in Source Code or other form. This patent license shall 75 | apply to the combination of the Contribution and the Program if, at 76 | the time the Contribution is added by the Contributor, such addition 77 | of the Contribution causes such combination to be covered by the 78 | Licensed Patents. The patent license shall not apply to any other 79 | combinations which include the Contribution. No hardware per se is 80 | licensed hereunder. 81 | 82 | c) Recipient understands that although each Contributor grants the 83 | licenses to its Contributions set forth herein, no assurances are 84 | provided by any Contributor that the Program does not infringe the 85 | patent or other intellectual property rights of any other entity. 86 | Each Contributor disclaims any liability to Recipient for claims 87 | brought by any other entity based on infringement of intellectual 88 | property rights or otherwise. As a condition to exercising the 89 | rights and licenses granted hereunder, each Recipient hereby 90 | assumes sole responsibility to secure any other intellectual 91 | property rights needed, if any. For example, if a third party 92 | patent license is required to allow Recipient to Distribute the 93 | Program, it is Recipient's responsibility to acquire that license 94 | before distributing the Program. 95 | 96 | d) Each Contributor represents that to its knowledge it has 97 | sufficient copyright rights in its Contribution, if any, to grant 98 | the copyright license set forth in this Agreement. 99 | 100 | e) Notwithstanding the terms of any Secondary License, no 101 | Contributor makes additional grants to any Recipient (other than 102 | those set forth in this Agreement) as a result of such Recipient's 103 | receipt of the Program under the terms of a Secondary License 104 | (if permitted under the terms of Section 3). 105 | 106 | 3. REQUIREMENTS 107 | 108 | 3.1 If a Contributor Distributes the Program in any form, then: 109 | 110 | a) the Program must also be made available as Source Code, in 111 | accordance with section 3.2, and the Contributor must accompany 112 | the Program with a statement that the Source Code for the Program 113 | is available under this Agreement, and informs Recipients how to 114 | obtain it in a reasonable manner on or through a medium customarily 115 | used for software exchange; and 116 | 117 | b) the Contributor may Distribute the Program under a license 118 | different than this Agreement, provided that such license: 119 | i) effectively disclaims on behalf of all other Contributors all 120 | warranties and conditions, express and implied, including 121 | warranties or conditions of title and non-infringement, and 122 | implied warranties or conditions of merchantability and fitness 123 | for a particular purpose; 124 | 125 | ii) effectively excludes on behalf of all other Contributors all 126 | liability for damages, including direct, indirect, special, 127 | incidental and consequential damages, such as lost profits; 128 | 129 | iii) does not attempt to limit or alter the recipients' rights 130 | in the Source Code under section 3.2; and 131 | 132 | iv) requires any subsequent distribution of the Program by any 133 | party to be under a license that satisfies the requirements 134 | of this section 3. 135 | 136 | 3.2 When the Program is Distributed as Source Code: 137 | 138 | a) it must be made available under this Agreement, or if the 139 | Program (i) is combined with other material in a separate file or 140 | files made available under a Secondary License, and (ii) the initial 141 | Contributor attached to the Source Code the notice described in 142 | Exhibit A of this Agreement, then the Program may be made available 143 | under the terms of such Secondary Licenses, and 144 | 145 | b) a copy of this Agreement must be included with each copy of 146 | the Program. 147 | 148 | 3.3 Contributors may not remove or alter any copyright, patent, 149 | trademark, attribution notices, disclaimers of warranty, or limitations 150 | of liability ("notices") contained within the Program from any copy of 151 | the Program which they Distribute, provided that Contributors may add 152 | their own appropriate notices. 153 | 154 | 4. COMMERCIAL DISTRIBUTION 155 | 156 | Commercial distributors of software may accept certain responsibilities 157 | with respect to end users, business partners and the like. While this 158 | license is intended to facilitate the commercial use of the Program, 159 | the Contributor who includes the Program in a commercial product 160 | offering should do so in a manner which does not create potential 161 | liability for other Contributors. Therefore, if a Contributor includes 162 | the Program in a commercial product offering, such Contributor 163 | ("Commercial Contributor") hereby agrees to defend and indemnify every 164 | other Contributor ("Indemnified Contributor") against any losses, 165 | damages and costs (collectively "Losses") arising from claims, lawsuits 166 | and other legal actions brought by a third party against the Indemnified 167 | Contributor to the extent caused by the acts or omissions of such 168 | Commercial Contributor in connection with its distribution of the Program 169 | in a commercial product offering. The obligations in this section do not 170 | apply to any claims or Losses relating to any actual or alleged 171 | intellectual property infringement. In order to qualify, an Indemnified 172 | Contributor must: a) promptly notify the Commercial Contributor in 173 | writing of such claim, and b) allow the Commercial Contributor to control, 174 | and cooperate with the Commercial Contributor in, the defense and any 175 | related settlement negotiations. The Indemnified Contributor may 176 | participate in any such claim at its own expense. 177 | 178 | For example, a Contributor might include the Program in a commercial 179 | product offering, Product X. That Contributor is then a Commercial 180 | Contributor. If that Commercial Contributor then makes performance 181 | claims, or offers warranties related to Product X, those performance 182 | claims and warranties are such Commercial Contributor's responsibility 183 | alone. Under this section, the Commercial Contributor would have to 184 | defend claims against the other Contributors related to those performance 185 | claims and warranties, and if a court requires any other Contributor to 186 | pay any damages as a result, the Commercial Contributor must pay 187 | those damages. 188 | 189 | 5. NO WARRANTY 190 | 191 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT 192 | PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS" 193 | BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR 194 | IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF 195 | TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR 196 | PURPOSE. Each Recipient is solely responsible for determining the 197 | appropriateness of using and distributing the Program and assumes all 198 | risks associated with its exercise of rights under this Agreement, 199 | including but not limited to the risks and costs of program errors, 200 | compliance with applicable laws, damage to or loss of data, programs 201 | or equipment, and unavailability or interruption of operations. 202 | 203 | 6. DISCLAIMER OF LIABILITY 204 | 205 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT 206 | PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS 207 | SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 208 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST 209 | PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 210 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 211 | ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE 212 | EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE 213 | POSSIBILITY OF SUCH DAMAGES. 214 | 215 | 7. GENERAL 216 | 217 | If any provision of this Agreement is invalid or unenforceable under 218 | applicable law, it shall not affect the validity or enforceability of 219 | the remainder of the terms of this Agreement, and without further 220 | action by the parties hereto, such provision shall be reformed to the 221 | minimum extent necessary to make such provision valid and enforceable. 222 | 223 | If Recipient institutes patent litigation against any entity 224 | (including a cross-claim or counterclaim in a lawsuit) alleging that the 225 | Program itself (excluding combinations of the Program with other software 226 | or hardware) infringes such Recipient's patent(s), then such Recipient's 227 | rights granted under Section 2(b) shall terminate as of the date such 228 | litigation is filed. 229 | 230 | All Recipient's rights under this Agreement shall terminate if it 231 | fails to comply with any of the material terms or conditions of this 232 | Agreement and does not cure such failure in a reasonable period of 233 | time after becoming aware of such noncompliance. If all Recipient's 234 | rights under this Agreement terminate, Recipient agrees to cease use 235 | and distribution of the Program as soon as reasonably practicable. 236 | However, Recipient's obligations under this Agreement and any licenses 237 | granted by Recipient relating to the Program shall continue and survive. 238 | 239 | Everyone is permitted to copy and distribute copies of this Agreement, 240 | but in order to avoid inconsistency the Agreement is copyrighted and 241 | may only be modified in the following manner. The Agreement Steward 242 | reserves the right to publish new versions (including revisions) of 243 | this Agreement from time to time. No one other than the Agreement 244 | Steward has the right to modify this Agreement. The Eclipse Foundation 245 | is the initial Agreement Steward. The Eclipse Foundation may assign the 246 | responsibility to serve as the Agreement Steward to a suitable separate 247 | entity. Each new version of the Agreement will be given a distinguishing 248 | version number. The Program (including Contributions) may always be 249 | Distributed subject to the version of the Agreement under which it was 250 | received. In addition, after a new version of the Agreement is published, 251 | Contributor may elect to Distribute the Program (including its 252 | Contributions) under the new version. 253 | 254 | Except as expressly stated in Sections 2(a) and 2(b) above, Recipient 255 | receives no rights or licenses to the intellectual property of any 256 | Contributor under this Agreement, whether expressly, by implication, 257 | estoppel or otherwise. All rights in the Program not expressly granted 258 | under this Agreement are reserved. Nothing in this Agreement is intended 259 | to be enforceable by any entity that is not a Contributor or Recipient. 260 | No third-party beneficiary rights are created under this Agreement. 261 | 262 | Exhibit A - Form of Secondary Licenses Notice 263 | 264 | "This Source Code may also be made available under the following 265 | Secondary Licenses when the conditions for such availability set forth 266 | in the Eclipse Public License, v. 2.0 are satisfied: {name license(s), 267 | version(s), and exceptions or additional permissions here}." 268 | 269 | Simply including a copy of this Agreement, including this Exhibit A 270 | is not sufficient to license the Source Code under Secondary Licenses. 271 | 272 | If it is not possible or desirable to put the notice in a particular 273 | file, then You may include the notice in a location (such as a LICENSE 274 | file in a relevant directory) where a recipient would be likely to 275 | look for such a notice. 276 | 277 | You may add additional accurate notices of copyright ownership. 278 | -------------------------------------------------------------------------------- /doc/jol-dev-ideas/jol-0-9-plus-parseInstanceIds-plus-andy-dev-changes.diff: -------------------------------------------------------------------------------- 1 | diff -r 9a5fff552b23 jol-core/pom.xml 2 | --- a/jol-core/pom.xml Fri Sep 22 17:29:07 2017 +0200 3 | +++ b/jol-core/pom.xml Fri Jul 12 00:55:21 2019 -0700 4 | @@ -30,7 +30,7 @@ 5 | 6 | org.openjdk.jol 7 | jol-parent 8 | - 0.9 9 | + 0.9.1 10 | 11 | 12 | jol-core 13 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java 14 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java Fri Sep 22 17:29:07 2017 +0200 15 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphLayout.java Fri Jul 12 00:55:21 2019 -0700 16 | @@ -37,6 +37,8 @@ 17 | import java.io.PrintWriter; 18 | import java.io.StringWriter; 19 | import java.util.*; 20 | +import java.util.function.BiConsumer; 21 | +import java.util.function.Predicate; 22 | 23 | /** 24 | * Holds the object graph layout info. 25 | @@ -44,7 +46,19 @@ 26 | public class GraphLayout { 27 | 28 | /** 29 | - * Parse the object graph starting from the given instance. 30 | + * Parse the object graph starting from the given instances. 31 | + * 32 | + * During this method's execution, if objects parsed are being 33 | + * modified in the references that exist between them by other 34 | + * threads, there is not much that can be guaranteed about the set 35 | + * of parsed objects that are returned, except that it will 36 | + * include the objects in the roots parameter. 37 | + * 38 | + * Even if no objects being parsed are modified while parsing, it 39 | + * is possible for the set of objects returned to be incomplete. 40 | + * This can occur if multiple objects happen to have the same 41 | + * address at different times during the call, because of garbage 42 | + * collection moving them around in memory. 43 | * 44 | * @param roots root instances to start from 45 | * @return object graph 46 | @@ -65,6 +79,98 @@ 47 | return data; 48 | } 49 | 50 | + /** 51 | + * Parse the object graph starting from the given instances. 52 | + * 53 | + * See parseInstance() documentation for notes on the few 54 | + * guarantees that can be made by this method if other threads are 55 | + * modifying references between parsed objects. 56 | + * 57 | + * This method puts the collection of objects found as keys into 58 | + * an IdentityHashMap. The value in the map corresponding to each 59 | + * object is a GraphPathRecord describing the object. You can get 60 | + * that IdentityHashMap using the objectsFound() method. 61 | + * 62 | + * If no parsed objects are modified, at least not in the 63 | + * references that exist between them, during this method's 64 | + * execution, the set of objects that are the keys of the 65 | + * IdentityHashMap created should always be correct. 66 | + * 67 | + * The address map returned by the addresses() method should be 68 | + * correct if no garbage collection occurred during this method's 69 | + * execution, but there is nothing returned to indicate whether 70 | + * they are complete or not. This is similar to what can happen 71 | + * in the parseInstance() method, but for this method at least the 72 | + * IdentityHashMap should have the correct set of objects as keys, 73 | + * and you may try calling the createAddressSnapshot() method on this 74 | + * GraphLayout instance to have a better chance of getting a 75 | + * correct address map. 76 | + * 77 | + * The addressSnapshot() method allocates much less memory than 78 | + * this method does, and is thus less likely to trigger a garbage 79 | + * collection run. 80 | + * 81 | + * @param stop null gives the default behavior of walking all references of all objects. By supplying a Predicate, every time the walk reaches an object o, it will call the stop.test(o) predicate, and avoid following any references out of o to other objects. 82 | + * @param roots root instances to start from 83 | + * @return object graph 84 | + */ 85 | + public static GraphLayout parseInstanceIds(Predicate stop, 86 | + Object... roots) { 87 | + if (roots == null) { 88 | + throw new IllegalArgumentException("Roots are null"); 89 | + } 90 | + for (Object root : roots) { 91 | + if (root == null) { 92 | + throw new IllegalArgumentException("Some root is null"); 93 | + } 94 | + } 95 | + GraphLayout data = new GraphLayout(roots); 96 | + GraphWalker walker = new GraphWalker(roots); 97 | + walker.addVisitor(data.visitorById()); 98 | + walker.walk(stop); 99 | + if (data.createAddressSnapshot(1)) { 100 | + data.createAddressMap(); 101 | + } 102 | + return data; 103 | + } 104 | + 105 | + /** 106 | + * Try to create a consistent snapshot of object addresses. Very 107 | + * little memory -- a constant amount, regardless of the size of 108 | + * the object graph -- should be allocated by this method, to 109 | + * minimize its chances of triggering a garbage collection while 110 | + * it executes, and maximize the likelihood that the set of 111 | + * addresses returned represent a consistent set. No other 112 | + * attempt is made to prevent GC from occurring while this method 113 | + * executes, so even though it does do one pass over the objects 114 | + * to record all of their current addresses, and then another pass 115 | + * to see if any of them have moved, retrying that pair of steps 116 | + * up to maxAttempts times if anything has moved, it is still 117 | + * possible (though it seems unlikely) that this method could 118 | + * return true (indicating success) even though the addresses do 119 | + * not represent a consistent snapshot. 120 | + * 121 | + * @param maxAttempts Maximum number of times to attempt creating a snapshot before returning false. 122 | + * @return true for success, false for failure 123 | + */ 124 | + public boolean createAddressSnapshot(int maxAttempts) { 125 | + boolean attemptSucceeded = false; 126 | + int attempts = 0; 127 | + do { 128 | + ++attempts; 129 | + recordAddresses(); 130 | + Exception e = validateAddresses(); 131 | + if (e == null) { 132 | + attemptSucceeded = true; 133 | + break; 134 | + } 135 | + } while (attempts <= maxAttempts); 136 | + if (attemptSucceeded) { 137 | + createAddressMap(); 138 | + } 139 | + return attemptSucceeded; 140 | + } 141 | + 142 | private static final Comparator> CLASS_COMPARATOR = new Comparator>() { 143 | @Override 144 | public int compare(Class o1, Class o2) { 145 | @@ -75,6 +181,7 @@ 146 | private final Set> classes = new TreeSet>(CLASS_COMPARATOR); 147 | private final Multiset> classSizes = new Multiset>(); 148 | private final Multiset> classCounts = new Multiset>(); 149 | + private final IdentityHashMap objectsFound = new IdentityHashMap(); 150 | private final SortedMap addresses = new TreeMap(); 151 | 152 | private final String description; 153 | @@ -121,6 +228,89 @@ 154 | } 155 | } 156 | 157 | + private GraphVisitor visitorById() { 158 | + return new GraphVisitor() { 159 | + @Override 160 | + public void visit(GraphPathRecord gpr) { 161 | + addRecordById(gpr); 162 | + } 163 | + }; 164 | + } 165 | + 166 | + private void addRecordById(GraphPathRecord gpr) { 167 | + objectsFound.put(gpr.obj(), gpr); 168 | + 169 | + Class klass = gpr.klass(); 170 | + classes.add(klass); 171 | + classCounts.add(klass); 172 | + totalCount++; 173 | + try { 174 | + long size = gpr.size(); 175 | + totalSize += size; 176 | + classSizes.add(klass, size); 177 | + } catch (Exception e) { 178 | + classSizes.add(klass, 0); 179 | + } 180 | + } 181 | + 182 | + public void recordAddresses() { 183 | + /* Iterate through objectsFound, recording the address of each 184 | + * object once. This method should allocate only a small 185 | + * constant amount of memory, to maximize the chances that it 186 | + * does not trigger a GC while it executes, and thus the 187 | + * addresses recorded represent a consistent 'snapshot' across 188 | + * all objects at one point in time. */ 189 | + BiConsumer recordOneAddress = 190 | + new BiConsumer() { 191 | + @Override 192 | + public void accept(Object o, Object gprArg) { 193 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 194 | + gpr.recordAddress(); 195 | + } 196 | + }; 197 | + objectsFound.forEach(recordOneAddress); 198 | + } 199 | + 200 | + public Exception validateAddresses() { 201 | + /* Same comments apply here as for method 202 | + * recordAddresses(). */ 203 | + BiConsumer validateOneAddress = 204 | + new BiConsumer() { 205 | + @Override 206 | + public void accept(Object o, Object gprArg) { 207 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 208 | + if (o != gpr.obj()) { 209 | + String msg = "Found GraphPathRecord with key object " + o.getClass() + " different than the object " + gpr.obj().getClass() + " in the GraphPathRecord"; 210 | + throw new IllegalArgumentException(msg); 211 | + } 212 | + long curAddress = VM.current().addressOf(o); 213 | + if (curAddress != gpr.address()) { 214 | + String msg = "Found GraphPathRecord with current address " + curAddress + " different than recorded address " + gpr.address() + " object " + gpr.obj().getClass(); 215 | + throw new IllegalArgumentException(msg); 216 | + } 217 | + } 218 | + }; 219 | + try { 220 | + objectsFound.forEach(validateOneAddress); 221 | + } catch (Exception e) { 222 | + return e; 223 | + } 224 | + return null; 225 | + } 226 | + 227 | + public void createAddressMap() { 228 | + BiConsumer addOneAddress = 229 | + new BiConsumer() { 230 | + @Override 231 | + public void accept(Object o, Object gprArg) { 232 | + GraphPathRecord gpr = (GraphPathRecord) gprArg; 233 | + addresses.put(gpr.address(), gpr); 234 | + } 235 | + }; 236 | + addresses.clear(); 237 | + objectsFound.forEach(addOneAddress); 238 | + } 239 | + 240 | /** 241 | * Subtract another layout data from the current one. 242 | * This method does not change the current data object, but produces another one. 243 | @@ -238,6 +428,17 @@ 244 | } 245 | 246 | /** 247 | + * Answer the IdentityHashMap of object references to their 248 | + * associated GraphPathRecord. 249 | + * 250 | + * @return sorted set of addresses 251 | + * @see #record(long) 252 | + */ 253 | + public IdentityHashMap objectsFound() { 254 | + return objectsFound; 255 | + } 256 | + 257 | + /** 258 | * Answer the set of addresses for the discovered objects 259 | * 260 | * @return sorted set of addresses 261 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java 262 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java Fri Sep 22 17:29:07 2017 +0200 263 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphPathRecord.java Fri Jul 12 00:55:21 2019 -0700 264 | @@ -38,7 +38,9 @@ 265 | private final int depth; 266 | private final Object obj; 267 | private final long size; 268 | - private final long address; 269 | + /* TBD: Does making field 'address' volatile be better for 270 | + * thread-safety of this class? */ 271 | + private volatile long address; 272 | private String toString; 273 | 274 | GraphPathRecord(GraphPathRecord parent, String path, int depth, Object obj) { 275 | @@ -74,6 +76,11 @@ 276 | return address; 277 | } 278 | 279 | + public long recordAddress() { 280 | + this.address = VM.current().addressOf(obj); 281 | + return address; 282 | + } 283 | + 284 | public String objToString() { 285 | String v = toString; 286 | if (v == null) { 287 | diff -r 9a5fff552b23 jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java 288 | --- a/jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java Fri Sep 22 17:29:07 2017 +0200 289 | +++ b/jol-core/src/main/java/org/openjdk/jol/info/GraphWalker.java Fri Jul 12 00:55:21 2019 -0700 290 | @@ -34,6 +34,7 @@ 291 | import java.util.IdentityHashMap; 292 | import java.util.List; 293 | import java.util.Set; 294 | +import java.util.function.Predicate; 295 | 296 | /** 297 | * Basic class to walk object graphs. 298 | @@ -57,6 +58,10 @@ 299 | } 300 | 301 | public void walk() { 302 | + walk(null); 303 | + } 304 | + 305 | + public void walk(Predicate stop) { 306 | List curLayer = new ArrayList(); 307 | List newLayer = new ArrayList(); 308 | 309 | @@ -74,7 +79,7 @@ 310 | while (!curLayer.isEmpty()) { 311 | newLayer.clear(); 312 | for (GraphPathRecord next : curLayer) { 313 | - for (GraphPathRecord ref : peelReferences(next)) { 314 | + for (GraphPathRecord ref : peelReferences(next, stop)) { 315 | if (visited.add(ref.obj())) { 316 | visitObject(ref); 317 | newLayer.add(ref); 318 | @@ -92,7 +97,8 @@ 319 | } 320 | } 321 | 322 | - private List peelReferences(GraphPathRecord r) { 323 | + private List peelReferences(GraphPathRecord r, 324 | + Predicate stop) { 325 | Object o = r.obj(); 326 | 327 | if (o == null) { 328 | @@ -105,6 +111,12 @@ 329 | return Collections.emptyList(); 330 | } 331 | 332 | + if (stop != null && stop.test(o)) { 333 | + // The caller does not want any references from this 334 | + // object walked. 335 | + return Collections.emptyList(); 336 | + } 337 | + 338 | List result = new ArrayList(); 339 | 340 | if (o.getClass().isArray()) { 341 | diff -r 9a5fff552b23 jol-core/src/test/java/org/openjdk/jol/layouters/LayouterInvariantsTest.java 342 | --- a/jol-core/src/test/java/org/openjdk/jol/layouters/LayouterInvariantsTest.java Fri Sep 22 17:29:07 2017 +0200 343 | +++ b/jol-core/src/test/java/org/openjdk/jol/layouters/LayouterInvariantsTest.java Fri Jul 12 00:55:21 2019 -0700 344 | @@ -19,7 +19,8 @@ 345 | new X86_64_COOPS_DataModel(), 346 | new X86_64_DataModel() 347 | }; 348 | - private static final int ITERATIONS = 20000; 349 | + //private static final int ITERATIONS = 20000; 350 | + private static final int ITERATIONS = 500; 351 | 352 | @Test 353 | public void testRaw() throws Exception { 354 | diff -r 9a5fff552b23 pom.xml 355 | --- a/pom.xml Fri Sep 22 17:29:07 2017 +0200 356 | +++ b/pom.xml Fri Jul 12 00:55:21 2019 -0700 357 | @@ -29,7 +29,7 @@ 358 | 359 | org.openjdk.jol 360 | jol-parent 361 | - 0.9 362 | + 0.9.1 363 | pom 364 | 365 | Java Object Layout: Parent 366 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | `cljol` is specific to Clojure on Java. It uses a JVM library that 4 | knows deep internal details of the JVM, and those parts would need to 5 | be replaced with something else in order to work on a non-JVM 6 | platform. 7 | 8 | `cljol` uses the [Java Object 9 | Layout](https://openjdk.java.net/projects/code-tools/jol) library to 10 | determine the precise size of a Java object, and all of the objects 11 | that it references, either directly, or by following a chain of 12 | references through multiple Java objects. 13 | 14 | It can create images of these graphs, either: 15 | 16 | + popping up a window using the `view` function. 17 | + writing to a GraphViz dot file using the `write-dot-file` function. 18 | + writing to any of several other image file formats using the 19 | `write-drawing-file` function. 20 | 21 | `cljol` has been tested most with Clojure 1.10.1 and 1.12.0 so far, 22 | but as far as I know should work fine with Clojure 1.7.0 or later (it 23 | cannot go earlier, since the Loom graph library requires that version 24 | of Clojure or later). 25 | 26 | See the [gallery](doc/README-gallery.md) for examples of figures 27 | created by this library that demonstrate aspects of the Java VM or 28 | Clojure's implementation that I find interesting. 29 | 30 | 31 | # Quick example 32 | 33 | You must install [GraphViz](http://www.graphviz.org) in order for the 34 | generation of figures to work -- see its home page for downloads and 35 | installation instructions if the following do not work: 36 | 37 | * Ubuntu Linux - `sudo apt-get install graphviz` 38 | * Mac OS X 39 | * If you use Homebrew: `brew install graphviz` 40 | * If you use MacPorts: `sudo port install graphviz` 41 | 42 | There are not yet any packaged releases of `cljol` on Clojars. You 43 | can clone the repository yourself and create a JAR if you like, or use 44 | the `clj` / `clojure` commands provided by the Clojure installer. 45 | 46 | No extra JVM options are necessary if you are running JDK 8 or 11: 47 | 48 | ```bash 49 | clj -Sdeps '{:deps {cljol/cljol {:git/url "https://github.com/jafingerhut/cljol" :sha "787343793edb6df3a6769d99ebb87e9c60839f4f"}}}' 50 | ``` 51 | 52 | If you are running JDK 16 or later, then the additional JVM command 53 | line options below are strongly recommended. Not using these options 54 | often leads to missing edges between objects in drawn graphs, and/or 55 | fields with the values omitted, showing the message `.setAccessible 56 | failed` in place of the field value. 57 | 58 | ```bash 59 | clj -Sdeps '{:deps {cljol/cljol {:git/url "https://github.com/jafingerhut/cljol" :sha "787343793edb6df3a6769d99ebb87e9c60839f4f"}}}' \ 60 | -J-Djdk.attach.allowAttachSelf \ 61 | -J-Djol.tryWithSudo=true \ 62 | -J-XX:+EnableDynamicAgentLoading \ 63 | -J--add-opens -Jjava.base/java.lang=ALL-UNNAMED 64 | ``` 65 | 66 | In the REPL: 67 | ``` 68 | (require '[cljol.dig9 :as d]) 69 | (def my-map {"a" 1 "foobar" 3.5}) 70 | 71 | ;; Open a new window containing the figure. Takes a collection of 72 | ;; objects. 73 | (d/view [my-map]) 74 | 75 | ;; Write the figure to a Graphviz dot file, or one of many other 76 | ;; formats. I believe you can get a complete list of formats 77 | ;; supported by Graphviz by looking at the "device" list in the output 78 | ;; of the command "dot -v < /dev/null". 79 | 80 | (d/write-dot-file [my-map] "my-map.dot") 81 | (d/write-drawing-file [my-map] "my-map.pdf" :pdf) 82 | ``` 83 | 84 | See the "Warning messages" section below for messages that you are 85 | likely to see when using this code. 86 | 87 | Graphviz dot files are a fairly simple text file format you can read 88 | in any text editor, and convert to many other graphic formats. You 89 | need not use the commands below to create these other formats, as the 90 | example of creating a PDF format file above shows, but below are some 91 | sample commands you can run in a shell to do this conversion: 92 | 93 | ```bash 94 | $ dot -Tpdf my-map.dot -o my-map.pdf 95 | $ dot -Tpng my-map.dot -o my-map.png 96 | ``` 97 | 98 | Below is the figure in the file `my-map.png` I get from the last 99 | command above. 100 | 101 | Each rectangle is a Java object. The object (or objects) that you 102 | specified to start from are filled with a gray shading. By default 103 | each rectangle shows: 104 | 105 | * the object's size 106 | * the number of objects reachable from that object, via following a 107 | path of references starting from that object, which includes that 108 | object itself. Also the total size in bytes of all of those 109 | reachable objects. 110 | * either the message "this object in no reference cycles" or 111 | "`` objects in same SCC with this one". The first message 112 | means that in the graph of objects reached from the ones you 113 | specified to start from, that object is not contained in any cycle 114 | of references. The second message means that the object does appear 115 | in at least one cycle of references, and there are `` 116 | objects such that they are all in the same SCC, or "strongly 117 | connected component", of the graph of references. Two nodes "a" and 118 | "b" are in the same SCC exactly when there is a path from "a" to 119 | "b", and also a path from "b" to "a". 120 | * its type, usually a class name, with common prefixes like 121 | "clojure.lang." replaced with "c.l." and "java.lang." replaced with 122 | "j.l.". Java arrays are shown as "array of N `class-name`". 123 | * A sequence of lines, one per field stored in the object. This is 124 | all per-instance fields (i.e. not declared static in Java) defined 125 | for the object's class, and all of its superclasses. Each is listed 126 | with: 127 | * its byte offset from the beginning of the object in memory where 128 | the field is stored 129 | * the name of the field 130 | * its type, in parentheses 131 | * the value of the field 132 | * All references to other objects only show "ref" as the type. The 133 | value of a "ref" field is shown as "nil" if it is a Clojure nil 134 | value (i.e. Java null), or `->` if the reference is to another 135 | object -- you may find the actual class of the referenced object by 136 | following the edge labeled with the field name that leaves the node. 137 | * a string representation of the object's value, or the message "val 138 | maybe realizes if str'ed". If you see that message, it means that 139 | `cljol` has determined that if it tried to convert the value to a 140 | string, it might cause lazy sequences to be realized more than they 141 | already have been before, and it is avoiding this possibility. 142 | 143 | The string representation is by default limited to 50 characters, with 144 | " ..." appearing at the end if it was truncated. 145 | 146 | The arrows out of an array object are labeled with "[i]", where "i" is 147 | a number that is the array index. Other labels on edges are the name 148 | of the field in the Java object that the edge comes from. 149 | 150 | Immediately below is the `cljol` drawing of the objects representing 151 | the Clojure map `{"a" 1 "foobar" 3.5}`. In a 152 | `clojure.lang.PersistentArrayMap`, map keys are in even array indices, 153 | and their associated values in the index 1 larger. 154 | 155 | ![my-map.png](doc/images/my-map-macos-10.13.6-oraclejdk-1.8.0_192-clj-1.10.1.png) 156 | 157 | 158 | 159 | # More examples 160 | 161 | It does not take much code to create data structures with very large 162 | graphs. For example, this graph likely has more nodes than you want 163 | to look at: 164 | 165 | ``` 166 | (def v1 (vec (range 1000))) 167 | (d/view [v1]) 168 | ``` 169 | 170 | `cljol` includes some code to give you summary statistics about a 171 | graph, and some functions that can produce a subset of a graph, which 172 | you can then display. 173 | 174 | The `sum` function takes a collection of objects, creates and returns 175 | a graph representing its objects without drawing it, and prints some 176 | statistics about this graph. The example below shows that v1's graph 177 | has 1067 objects. The info near the end shows that 1001 of those have 178 | out-degree 0, where out-degree is the number of edges that leave a 179 | node directed towards another node. Those 1001 are 'leaf nodes' of 180 | the graph. 181 | 182 | ``` 183 | user=> (def g (d/sum [v1] {:summary-options #{:all}})) 184 | 1067 objects 185 | 1097 references between them 186 | 29480 bytes total in all objects 187 | no cycles 188 | 1 weakly connected components found in: 5.6 msec, 0 gc-count, 0 gc-time-msec 189 | number of nodes in all weakly connected components, 190 | from most to fewest nodes: 191 | (1067) 192 | The scc-graph has 1067 nodes and 1097 edges, took: 5.8 msec, 0 gc-count, 0 gc-time-msec 193 | The largest size strongly connected components, at most 10: 194 | (1 1 1 1 1 1 1 1 1 1) 195 | number of objects of each size in bytes: 196 | ({:size-bytes 16, :num-objects 1, :total-size 16} 197 | {:size-bytes 24, :num-objects 1032, :total-size 24768} 198 | {:size-bytes 40, :num-objects 1, :total-size 40} 199 | {:size-bytes 48, :num-objects 1, :total-size 48} 200 | {:size-bytes 144, :num-objects 32, :total-size 4608}) 201 | number and size of objects of each class: 202 | ({:total-size 16, 203 | :num-objects 1, 204 | :class "j.u.c.atomic.AtomicReference"} 205 | {:total-size 40, :num-objects 1, :class "c.l.PersistentVector"} 206 | {:total-size 768, :num-objects 32, :class "c.l.PersistentVector$Node"} 207 | {:total-size 4656, :num-objects 33, :class "j.l.Object/1"} 208 | {:total-size 24000, :num-objects 1000, :class "j.l.Long"}) 209 | 210 | 1001 leaf objects (no references to other objects) 211 | 1 root nodes (no reference to them from other objects _in this graph_) 212 | number of objects of each in-degree (# of references to it): 213 | ({:in-degree 0, :num-objects 1} 214 | {:in-degree 1, :num-objects 1065} 215 | {:in-degree 32, :num-objects 1}) 216 | number of objects of each out-degree (# of references from it): 217 | ({:out-degree 0, :num-objects 1001} 218 | {:out-degree 2, :num-objects 33} 219 | {:out-degree 8, :num-objects 1} 220 | {:out-degree 31, :num-objects 1} 221 | {:out-degree 32, :num-objects 31}) 222 | Number and total size of objects at each distance from a starting object: 223 | ({:distance 0, :num-objects 1, :total-size 40} 224 | {:distance 1, :num-objects 2, :total-size 72} 225 | {:distance 2, :num-objects 10, :total-size 352} 226 | {:distance 3, :num-objects 31, :total-size 744} 227 | {:distance 4, :num-objects 31, :total-size 4464} 228 | {:distance 5, :num-objects 992, :total-size 23808}) 229 | #'user/g 230 | ``` 231 | 232 | Here is a way to create another graph `g2` from `g` with all of `g`'s 233 | leaves removed, and then draw `g2`: 234 | 235 | ``` 236 | (require '[ubergraph.core :as uber] 237 | '[cljol.graph :as gr] 238 | '[cljol.ubergraph-extras :as gre]) 239 | 240 | (def g2 (uber/remove-nodes* g (gr/leaf-nodes g))) 241 | 242 | (d/view-graph g2) 243 | ``` 244 | 245 | Below we demonstrate keeping only those nodes that are within at most 246 | distance 3 of the starting objects given when creating the graph. 247 | That is, the Java object is reachable from one of the starting objects 248 | in 3 or fewer 'hops'. 249 | 250 | ``` 251 | (def g3 (gre/induced-subgraph g (filter #(<= (uber/attr g % :distance) 3) 252 | (uber/nodes g)))) 253 | 254 | (d/view-graph g3) 255 | ``` 256 | 257 | These graphs `g`, `g2`, and `g3` are all created using the 258 | [Ubergraph](https://github.com/Engelberg/ubergraph) library. All of 259 | its features are available for manipulating these graphs. The drawing 260 | functions use keys in the node and edge attribute maps to affect some 261 | aspects of the drawings, e.g. the `:label` key is used to generate the 262 | labels. 263 | 264 | Another thing that can be interesting to see is the fraction of 265 | objects shared between a persistent collection, and the persistent 266 | collection created by making a small change to the first collection. 267 | 268 | ``` 269 | ;; Create a graph of objects reachable from one or more 270 | ;; root objects. Use a function add-attributes-by-reachability 271 | ;; to add color attributes to the graph nodes so that if 272 | ;; a node is reachable from one root object, but not any of 273 | ;; the others, it will be given a specified color. 274 | ;; If a node is reachable from more than one root node, 275 | ;; assign it the color 'multi-color'. In this function, 276 | ;; all nodes of the graph should be reachable from one of 277 | ;; the given root objects, but in case there is some kind 278 | ;; of situation where a node cannot be reached from any 279 | ;; root object, assign it the color "gray". 280 | 281 | (defn colored-graph [obj-color-pairs multi-color] 282 | (let [objs (map first obj-color-pairs) 283 | attrs (mapv (fn [[obj color]] 284 | {:only-from obj :attrs {:color color}}) 285 | obj-color-pairs) 286 | attrs (conj attrs 287 | {:from-multiple true :attrs {:color multi-color}} 288 | {:from-none true :attrs {:color "gray"}}) 289 | g (d/sum objs)] 290 | (d/add-attributes-by-reachability g attrs))) 291 | 292 | ;; Two similar Clojure vectors, with lots of sharing of objects. 293 | (def v1 (vec (range 5))) 294 | (def v2 (conj v1 5)) 295 | (def g (colored-graph [[v1 "red"] [v2 "green"]] "blue")) 296 | (d/view-graph g) 297 | (d/view-graph g {:save {:filename "g.pdf" :format :pdf}}) 298 | 299 | ;; Two similar Clojure hash maps, with lots of sharing of objects. 300 | (def m1 (hash-map 1 -1 2 -2 3 -3 5 -5 9 -9)) 301 | (def m2 (assoc m1 4 -4)) 302 | (def g (colored-graph [[m1 "red"] [m2 "green"]] "blue")) 303 | (d/view-graph g) 304 | (d/view-graph g {:save {:filename "g.pdf" :format :pdf}}) 305 | ``` 306 | 307 | 308 | # Error and Warning messages 309 | 310 | When running with JDK 16 or later and the extra command line options 311 | recommended above, you may see error messages like these: 312 | 313 | ``` 314 | ERROR: Add these JVM command line options to avoid errors determining field values of objects: --add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED 315 | ERROR: Add these JVM command line options to avoid errors determining field values of objects: --add-opens java.base/java.lang=ALL-UNNAMED 316 | ``` 317 | 318 | These occur if `cljol` is attempting to read the values of fields of 319 | JVM objects, but an `InaccessibleObjectException` was thrown. This 320 | causes `cljol` to produce incorrect graphs of objects, and to show the 321 | message `.setAccessible failed` instead of the true values of fields. 322 | 323 | I typically see a warning like the one below the 324 | first time I call the function `view`, `write-dot-file`, or 325 | `write-drawing-file`: 326 | 327 | ```bash 328 | # WARNING: Unable to attach Serviceability Agent. sun.jvm.hotspot.memory.Universe.getNarrowOopBase() 329 | ``` 330 | 331 | The presence of this warning does not seem to harm the functionality 332 | of `cljol` in any way. 333 | 334 | Tested with: 335 | 336 | * Ubuntu 24.04, Adoptium OpenJDK 8, 11, 16-24, Clojure 1.12.0 337 | * Ubuntu 18.04.2, OpenJDK 11, Clojure 1.10.1 338 | * Ubuntu 18.04.2, Oracle JDK 8, Clojure 1.10.1 339 | * Mac OS X 10.13 High Sierra, Oracle JDK 8, Clojure 1.10.1 340 | 341 | It should work with older versions of Clojure, too, but I do not know 342 | how far back it can go. Probably as far back as Clojure 1.7, which is 343 | required by the version of the Loom library that cljol depends upon. 344 | 345 | 346 | # Possible future work 347 | 348 | Perhaps some day this library might be enhanced to create nice figures 349 | and/or summary statistics showing how many of these objects are shared 350 | between two Clojure collections. There is some code in the 351 | `cljol.dig` namespace written with that in mind, but it is at best not 352 | well tested and thus probably contains many errors, if it even runs at 353 | all. 354 | 355 | It would be nice if there was a way to cause the edges out of Java 356 | array objects to at least usually be in increasing order of array 357 | index. Right now they are fairly arbitrary. 358 | 359 | 360 | ## License 361 | 362 | Copyright © 2016-2019 Andy Fingerhut 363 | 364 | This program and the accompanying materials are made available under the 365 | terms of the Eclipse Public License 2.0 which is available at 366 | http://www.eclipse.org/org/documents/epl-v10.html 367 | --------------------------------------------------------------------------------