├── .github └── workflows │ └── clojure.yml ├── .gitignore ├── LICENSE ├── README.md ├── build.clj ├── deps.edn ├── doc ├── indices.md └── performance.md ├── src └── xitdb │ ├── array_list.clj │ ├── common.clj │ ├── db.clj │ ├── hash_map.clj │ ├── hash_set.clj │ ├── linked_list.clj │ ├── snapshot.clj │ ├── util │ ├── conversion.clj │ ├── operations.clj │ └── validation.clj │ └── xitdb_types.clj ├── test-resources ├── map-1.edn └── map-2.edn └── test └── xitdb ├── cursor_test.clj ├── data_types_test.clj ├── database_test.clj ├── deep_merge_test.clj ├── gen_map.clj ├── generated_data_test.clj ├── kv_reduce_test.clj ├── map_test.clj ├── multi_threaded_test.clj ├── set_test.clj ├── test_utils.clj └── transient_test.clj /.github/workflows/clojure.yml: -------------------------------------------------------------------------------- 1 | name: Clojure CI 2 | 3 | on: 4 | push: 5 | branches: [ "master" ] 6 | pull_request: 7 | branches: [ "master" ] 8 | 9 | jobs: 10 | build: 11 | 12 | runs-on: ubuntu-latest 13 | 14 | steps: 15 | - name: Set up Java 21 16 | uses: actions/setup-java@v3 17 | with: 18 | java-version: '21' 19 | distribution: 'temurin' 20 | 21 | - name: Install Clojure CLI 22 | run: | 23 | sudo apt update 24 | sudo apt install -y curl gnupg 25 | curl -L -O https://github.com/clojure/brew-install/releases/latest/download/linux-install.sh 26 | chmod +x linux-install.sh 27 | sudo ./linux-install.sh 28 | 29 | - uses: actions/checkout@v4 30 | 31 | - name: Install dependencies 32 | run: clojure -P 33 | 34 | - name: Run tests 35 | run: clojure -M:test 36 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea/ 2 | .cpcache/ 3 | .nrepl-port 4 | *.iml 5 | *.db 6 | .DS_Store 7 | *.xdb 8 | target/ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (Expat) 2 | 3 | Copyright (c) contributors 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | > **⚠️ Alpha Software - Work in Progress** 2 | > 3 | > This project is in early development and rapidly evolving. 4 | > Expect breaking changes, rough edges, and incomplete documentation. 5 | > 6 | > **Help Wanted!** If you find this useful, please consider contributing: 7 | > - Report bugs and issues you encounter 8 | > - Suggest improvements or new features 9 | > - Submit pull requests for fixes or enhancements 10 | > - Share your configuration patterns and workflows 11 | > - Help improve documentation and examples 12 | > 13 | > Your feedback and contributions will help make this tool better for the entire Clojure community! 14 | 15 | ## Overview 16 | 17 | `xitdb-clj` is a embedded database for efficiently storing and retrieving immutable, persistent data structures. 18 | The library provides atom-like semantics for working with the database from Clojure. 19 | 20 | It is a Clojure interface for [xitdb-java](https://github.com/radarroark/xitdb-java), itself a port of [xitdb](https://github.com/radarroark/xitdb), written in Zig. 21 | 22 | 23 | [![Clojars Project](https://img.shields.io/clojars/v/io.github.codeboost/xitdb-clj.svg)](https://clojars.org/io.github.codeboost/xitdb-clj) 24 | 25 | ## Main characteristics 26 | 27 | - Embeddable, tiny library. 28 | - Supports writing to a file as well as purely in-memory use. 29 | - Each transaction (done via `swap!`) efficiently creates a new "copy" of the database, and past copies can still be read from. 30 | - Reading/Writing to the database is efficient, only the necessary nodes are read or written. 31 | - Thread safe. Multiple readers, one writer. 32 | - Append-only. The data you are writing is invisible to any reader until the very last step, when the top-level history header is updated. 33 | - All heavy lifting done by the bare-to-the-jvm java library. 34 | - Database files can be used from other languages, via [xitdb Java library](https://github.com/radarroark/xitdb-java) or the [xitdb Zig library](https://github.com/radarroark/xitdb) 35 | 36 | ## Quickstart 37 | 38 | Add the dependency to your project, start a REPL. 39 | 40 | ### You already know how to use it! 41 | 42 | For the programmer, a `xitdb` database is like a Clojure atom. 43 | `reset!` or `swap!` to reset or update, `deref` or `@` to read. 44 | 45 | ```clojure 46 | (require '[xitdb.db :as xdb]) 47 | (def db (xdb/xit-db "my-app.db")) 48 | ;; Use it like an atom 49 | (reset! db {:users {"alice" {:name "Alice" :age 30} 50 | "bob" {:name "Bob" :age 25}}}) 51 | ;; Read the entire database 52 | (xdb/materialize @db) 53 | ;; => {:users {"alice" {:name "Alice", :age 30}, "bob" {:name "Bob", :age 25}}} 54 | 55 | (get-in @db [:users "alice" :age]) 56 | ;; => 30 57 | (swap! db assoc-in [:users "alice" :age] 31) 58 | 59 | (get-in @db [:users "alice" :age]) 60 | ;; => 31 61 | ``` 62 | One important distinction from the Clojure atom is that inside a transaction (eg. a `swap!`), 63 | 'change' operations on the received `db` argument are mutating the underlying data structure. 64 | 65 | ```clojure 66 | (with-db [db (xdb/xit-db :memory)] 67 | (reset! db {}) 68 | (swap! db (fn [db] 69 | (let [db1 (assoc db :foo :bar)] 70 | (println "db1:" db1) 71 | (println "db:" db))))) 72 | ``` 73 | prints 74 | ``` 75 | db1: {:foo :bar} 76 | db: {:foo :bar} 77 | ``` 78 | As you can see, `(assoc db :foo :bar)` changed the value of `db`, in contrast 79 | to how it works with a Clojure persistent map. This is because, inside `swap!`, 80 | `db` is referencing a WriteCursor, which writes the value to the underlying 81 | ArrayList or HashMap objects inside `xit-db-java`. 82 | The value will actually be commited to the database when the `swap!` function returns. 83 | 84 | ## Data structures are read lazily from the database 85 | 86 | Reading from the database returns wrappers around cursors in the database file: 87 | 88 | ```clojure 89 | (type @db) ;; => xitdb.hash_map.XITDBHashMap 90 | ``` 91 | 92 | The returned value is a `XITDBHashMap` which is a wrapper around the xitdb-java's `ReadHashMap`, 93 | which itself has a cursor to the tree node in the database file. 94 | These wrappers implement the protocols for Clojure collections - vectors, lists, maps and sets, 95 | so they behave exactly like the Clojure native data structures. 96 | Any read operation on these types is going to return new `XITDB` types: 97 | 98 | ```clojure 99 | (type (get-in @db [:users "alice"])) ;; => xitdb.hash_map.XITDBHashMap 100 | ``` 101 | 102 | So it will not read the entire nested structure into memory, but return a 'cursor' type, which you can operate upon 103 | using Clojure functions. 104 | 105 | Use `materialize` to convert a nested `XITDB` data structure to a native Clojure data structure: 106 | 107 | ```clojure 108 | (materialize (get-in @db [:users "alice"])) ;; => {:name "Alice" :age 31} 109 | ``` 110 | 111 | ## No query language 112 | 113 | Use `filter`, `group-by`, `reduce`, etc. 114 | If you want a query engine, `datascript` works out of the box, you can store the datoms as a vector in the db. 115 | 116 | Here's a taste of how your queries could look like: 117 | ```clojure 118 | (defn titles-of-songs-for-artist 119 | [db artist] 120 | (->> (get-in db [:songs-indices :artist artist]) 121 | (map #(get-in db [:songs % :title])))) 122 | 123 | (defn what-is-the-most-viewed-song? [db tag] 124 | (let [views (->> (get-in db [:songs-indices :tag tag]) 125 | (map (:songs db)) 126 | (map (juxt :id :views)) 127 | (sort-by #(parse-long (second %))))] 128 | (get-in db [:songs (first (last views))]))) 129 | 130 | ``` 131 | 132 | ## History 133 | Since the database is immutable, all previous values are accessing by reading 134 | from the respective `history index`. 135 | The root data structure of a xitdb database is a ArrayList, called 'history'. 136 | Each transaction adds a new entry into this array, which points to the latest value 137 | of the database (usually a map). 138 | It is also possible to create a transaction which returns the previous and current 139 | values of the database, by setting the `*return-history?*` binding to `true`. 140 | 141 | ```clojure 142 | ;; Work with history tracking 143 | (binding [xdb/*return-history?* true] 144 | (let [[history-index old-value new-value] (swap! db assoc :new-key "value")] 145 | (println "old value:" old-value) 146 | (println "new value:" new-value))) 147 | ``` 148 | 149 | ### Architecture 150 | `xitdb-clj` builds on [xitdb-java](https://github.com/radarroark/xitdb-java) which implements: 151 | 152 | - **Hash Array Mapped Trie (HAMT)** - For efficient map and set operations 153 | - **RRB Trees** - For vector operations with good concatenation performance 154 | - **Structural Sharing** - Minimizes memory usage across versions 155 | - **Copy-on-Write** - Ensures immutability while maintaining performance 156 | 157 | The Clojure wrapper adds: 158 | - Idiomatic Clojure interfaces (`IAtom`, `IDeref`) 159 | - Automatic type conversion between Clojure and Java types 160 | - Thread-local read connections for scalability 161 | - Integration with Clojure's sequence abstractions 162 | 163 | ### Supported Data Types 164 | 165 | - **Maps** - Hash maps with efficient key-value access 166 | - **Vectors** - Array lists with indexed access 167 | - **Sets** - Hash sets with unique element storage 168 | - **Lists** - Linked lists and RRB tree-based linked array lists 169 | - **Primitives** - Numbers, strings, keywords, booleans, dates. 170 | 171 | ## Performance Characteristics 172 | 173 | - **Read Operations**: O(log16 n) for maps and vectors due to trie structure 174 | - **Write Operations**: O(log16 n) with structural sharing for efficiency 175 | - **Memory Usage**: Minimal overhead with automatic deduplication of identical subtrees 176 | - **Concurrency**: Thread-safe with optimized read-write locks 177 | 178 | 179 | ## License 180 | 181 | MIT 182 | -------------------------------------------------------------------------------- /build.clj: -------------------------------------------------------------------------------- 1 | (ns build 2 | (:require [clojure.tools.build.api :as b])) 3 | 4 | (def lib 'io.github.codeboost/xitdb-clj) 5 | (def version "0.1.2") 6 | (def class-dir "target/classes") 7 | (def basis (b/create-basis {:project "deps.edn"})) 8 | (def jar-file (format "target/%s-%s.jar" (name lib) version)) 9 | 10 | (defn clean [_] 11 | (b/delete {:path "target"})) 12 | 13 | (defn jar [_] 14 | (b/write-pom {:class-dir class-dir 15 | :lib lib 16 | :version version 17 | :basis basis 18 | :src-dirs ["src"] 19 | :scm {:url "https://github.com/codeboost/xitdb-clj"} 20 | :pom-data [[:description "Embeddable, immutable database."] 21 | [:url "https://github.com/codeboost/xitdb-clj"] 22 | [:licenses 23 | [:license 24 | [:name "MIT License"] 25 | [:url "https://opensource.org/licenses/MIT"]]]]}) 26 | (b/copy-dir {:src-dirs ["src" "resources"] 27 | :target-dir class-dir}) 28 | (b/jar {:class-dir class-dir 29 | :jar-file jar-file})) 30 | 31 | (defn deploy [_] 32 | (jar nil) 33 | ((requiring-resolve 'deps-deploy.deps-deploy/deploy) 34 | {:installer :remote 35 | :artifact jar-file 36 | :pom-file (b/pom-path {:lib lib :class-dir class-dir})})) -------------------------------------------------------------------------------- /deps.edn: -------------------------------------------------------------------------------- 1 | {:paths ["src" "test"] 2 | :deps {org.clojure/clojure {:mvn/version "1.12.0"} 3 | io.github.radarroark/xitdb {:mvn/version "0.20.0"}} 4 | 5 | :aliases 6 | {:test {:extra-deps {io.github.cognitect-labs/test-runner 7 | {:git/url "https://github.com/cognitect-labs/test-runner.git" 8 | :git/tag "v0.5.1" 9 | :git/sha "dfb30dd"}} 10 | 11 | :main-opts ["-m" "cognitect.test-runner"]} 12 | :build {:deps {io.github.clojure/tools.build {:mvn/version "0.9.6"} 13 | slipset/deps-deploy {:mvn/version "0.2.2"}} 14 | :ns-default build}}} 15 | 16 | -------------------------------------------------------------------------------- /doc/indices.md: -------------------------------------------------------------------------------- 1 | # Using indices with xitdb 2 | 3 | There's no built-in support for indices, because... 4 | Indices are yet another data structure which you can store in the database. 5 | 6 | ```clojure 7 | 8 | (defn records->index-set 9 | "Returns a map keyed by `k` and valued by a `set` of ids." 10 | [records k & {:keys [id] :or {id :id}}] 11 | (->> records 12 | (group-by k) 13 | (map (fn [[k songs]] 14 | [k (->> songs (map id) set)])) 15 | (into {}))) 16 | 17 | (defn update-index 18 | "Updates an index in `m`. 19 | All indices are stored in the 'index' map (eg. songs-index). 20 | Eg: 21 | `{:songs-index {:artist {'foo' #{8 9}}}` 22 | The songs-index contains an index which groups the songs by artist name -> set of song ids. 23 | k represents the `key` on which the `records` are going to be grouped and 24 | then merged into the `k` index." 25 | [m index k records] 26 | (let [k->song-id-set (records->index-set records k)] 27 | (update-in m [index k] #(merge-with into % k->song-id-set)))) 28 | 29 | (defn insert-songs! [db songs] 30 | (let [id->song (into {} (map (juxt :id identity)) songs)] 31 | (swap! db (fn [m] 32 | (-> m 33 | (update :songs merge id->song) 34 | (update-index :songs-indices :artist songs) 35 | (update-index :songs-indices :tag songs) 36 | (update-index :songs-indices :year songs)))))) 37 | ``` 38 | 39 | 40 | # External indices 41 | Indices can be stored in a separate database file. 42 | In fact, you might not need all indices in your 'live' database, but you might 43 | need some indices for analytics or reporting. -------------------------------------------------------------------------------- /doc/performance.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/codeboost/xitdb-clj/87ccef7b164716c78fb352cbd66bc7a37b294ff9/doc/performance.md -------------------------------------------------------------------------------- /src/xitdb/array_list.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.array-list 2 | (:require 3 | [xitdb.common :as common] 4 | [xitdb.util.operations :as operations]) 5 | (:import 6 | (io.github.radarroark.xitdb ReadArrayList ReadCursor WriteArrayList WriteCursor))) 7 | 8 | (defn array-seq 9 | [^ReadArrayList ral] 10 | "The cursors used must implement the IReadFromCursor protocol." 11 | (operations/array-seq ral #(common/-read-from-cursor %))) 12 | 13 | (deftype XITDBArrayList [^ReadArrayList ral] 14 | clojure.lang.IPersistentCollection 15 | (seq [_] 16 | (array-seq ral)) 17 | 18 | (count [_] 19 | (.count ral)) 20 | 21 | (cons [_ o] 22 | (throw (UnsupportedOperationException. "XITDBArrayList is read-only"))) 23 | 24 | (empty [_] 25 | (throw (UnsupportedOperationException. "XITDBArrayList is read-only"))) 26 | 27 | (equiv [this other] 28 | (and (sequential? other) 29 | (= (count this) (count other)) 30 | (every? identity (map = this other)))) 31 | 32 | clojure.lang.Sequential ;; Add this to mark as sequential 33 | 34 | clojure.lang.IPersistentVector 35 | (assocN [this i val] 36 | (throw (UnsupportedOperationException. "XITDBArrayList is read-only"))) 37 | 38 | (length [this] 39 | (.count ral)) 40 | 41 | clojure.lang.Indexed 42 | (nth [_ i] 43 | (let [cursor (.getCursor ral (long i))] 44 | (common/-read-from-cursor cursor))) 45 | 46 | (nth [_ i not-found] 47 | (let [cursor (.getCursor ral (long i))] 48 | (if cursor 49 | (common/-read-from-cursor cursor) 50 | not-found))) 51 | 52 | clojure.lang.ILookup 53 | (valAt [this k] 54 | (if (number? k) 55 | (.nth this (long k)) 56 | (throw (IllegalArgumentException. "Key must be a number")))) 57 | 58 | (valAt [this k not-found] 59 | (if (number? k) 60 | (.nth this (long k) not-found) 61 | not-found)) 62 | 63 | clojure.lang.IFn 64 | (invoke [this k] 65 | (.valAt this k)) 66 | 67 | (invoke [this k not-found] 68 | (.valAt this k not-found)) 69 | 70 | (applyTo [this args] 71 | (case (count args) 72 | 1 (.invoke this (first args)) 73 | 2 (.invoke this (first args) (second args)) 74 | (throw (IllegalArgumentException. "Wrong number of args passed to XITDBArrayList")))) 75 | 76 | clojure.lang.IReduce 77 | (reduce [this f] 78 | (let [s (seq this)] 79 | (if s 80 | (reduce f (first s) (rest s)) 81 | (f)))) 82 | 83 | clojure.lang.IReduceInit 84 | (reduce [this f init] 85 | (reduce f init (array-seq ral))) 86 | 87 | clojure.core.protocols/IKVReduce 88 | (kv-reduce [this f init] 89 | (operations/array-kv-reduce ral #(common/-read-from-cursor %) f init)) 90 | 91 | java.util.Collection 92 | (^objects toArray [this] 93 | (to-array (into [] this))) 94 | 95 | (^objects toArray [this ^objects array] 96 | (let [len (count this) 97 | ^objects result (if (or (nil? array) (< (alength array) len)) 98 | (make-array Object len) 99 | array)] 100 | (dotimes [i len] 101 | (aset result i (nth this i))) 102 | (when (> (alength result) len) 103 | (aset result len nil)) 104 | result)) 105 | 106 | common/IUnwrap 107 | (-unwrap [this] 108 | ral) 109 | 110 | Object 111 | (toString [this] 112 | (pr-str (into [] this)))) 113 | 114 | (defmethod print-method XITDBArrayList [o ^java.io.Writer w] 115 | (.write w "#XITDBArrayList") 116 | (print-method (into [] o) w)) 117 | 118 | (extend-protocol common/IMaterialize 119 | XITDBArrayList 120 | (-materialize [this] 121 | (reduce (fn [a v] 122 | (conj a (common/materialize v))) [] (seq this)))) 123 | 124 | ;;----------------------------------------------- 125 | 126 | (deftype XITDBWriteArrayList [^WriteArrayList wal] 127 | clojure.lang.IPersistentCollection 128 | (count [this] 129 | (.count wal)) 130 | 131 | (cons [this o] 132 | (operations/array-list-append-value! wal (common/unwrap o)) 133 | this) 134 | 135 | (empty [this] 136 | (operations/array-list-empty! wal) 137 | this) 138 | 139 | (equiv [this other] 140 | (if (instance? XITDBWriteArrayList other) 141 | (and (= (count this) (count other)) 142 | (every? (fn [i] (= (get this i) (get other i))) 143 | (range (count this)))) 144 | false)) 145 | 146 | clojure.lang.Indexed 147 | (nth [this i] 148 | (.nth this i nil)) 149 | 150 | (nth [this i not-found] 151 | (if (and (>= i 0) (< i (.count wal))) 152 | (common/-read-from-cursor (.putCursor wal i)) 153 | not-found)) 154 | 155 | clojure.lang.IPersistentVector 156 | (assocN [this i val] 157 | (operations/array-list-assoc-value! wal i (common/unwrap val)) 158 | this) 159 | 160 | (length [this] 161 | (.count wal)) 162 | 163 | clojure.lang.Associative 164 | (assoc [this k v] 165 | (when-not (integer? k) 166 | (throw (IllegalArgumentException. "Key must be integer"))) 167 | (operations/array-list-assoc-value! wal k (common/unwrap v)) 168 | this) 169 | 170 | (containsKey [this k] 171 | (and (integer? k) (>= k 0) (< k (.count wal)))) 172 | 173 | (entryAt [this k] 174 | (when (.containsKey this k) 175 | (clojure.lang.MapEntry. k (.valAt this k)))) 176 | 177 | clojure.lang.ILookup 178 | (valAt [this k] 179 | (.valAt this k nil)) 180 | 181 | (valAt [this k not-found] 182 | (.nth this k not-found)) 183 | 184 | clojure.lang.Seqable 185 | (seq [this] 186 | (array-seq wal)) 187 | 188 | clojure.core.protocols/IKVReduce 189 | (kv-reduce [this f init] 190 | (operations/array-kv-reduce wal #(common/-read-from-cursor %) f init)) 191 | 192 | clojure.lang.IObj 193 | (withMeta [this _] 194 | this) 195 | 196 | clojure.lang.IMeta 197 | (meta [this] 198 | nil) 199 | 200 | clojure.lang.IEditableCollection 201 | (asTransient [this] 202 | this) 203 | 204 | clojure.lang.ITransientCollection 205 | (conj [this val] 206 | (operations/array-list-append-value! wal (common/unwrap val)) 207 | this) 208 | 209 | (persistent [this] 210 | this) 211 | 212 | clojure.lang.ITransientVector ;; assoc already implemented 213 | 214 | (pop [this] 215 | (let [value (common/-read-from-cursor (-> wal .-cursor))] 216 | (operations/array-list-pop! wal) 217 | value)) 218 | 219 | common/ISlot 220 | (-slot [this] 221 | (-> wal .cursor .slot)) 222 | 223 | common/IUnwrap 224 | (-unwrap [this] 225 | wal) 226 | 227 | Object 228 | (toString [this] 229 | (str "XITDBWriteArrayList"))) 230 | 231 | ;; Constructors 232 | 233 | (defn xwrite-array-list [^WriteCursor write-cursor] 234 | (->XITDBWriteArrayList (WriteArrayList. write-cursor))) 235 | 236 | (defn xarray-list [^ReadCursor cursor] 237 | (->XITDBArrayList (ReadArrayList. cursor))) -------------------------------------------------------------------------------- /src/xitdb/common.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.common) 2 | 3 | (defprotocol ISlot 4 | (-slot [this])) 5 | 6 | (defprotocol IReadFromCursor 7 | (-read-from-cursor [this])) 8 | 9 | (defprotocol IMaterialize 10 | (-materialize [this])) 11 | 12 | (defprotocol IUnwrap 13 | (-unwrap [this])) 14 | 15 | 16 | (defn materialize [v] 17 | (cond 18 | (satisfies? IMaterialize v) (-materialize v) 19 | (vector? v) (mapv materialize v) 20 | (map? v) (reduce-kv (fn [m k v] (assoc m (materialize k) (materialize v))) {} v) 21 | (set? v) (into #{} (map materialize v)) 22 | (seq? v) (doall (map materialize v)) 23 | :else v)) 24 | 25 | (defn unwrap 26 | "For a value that wraps another value, returns the wrapped value." 27 | [v] 28 | (if (satisfies? IUnwrap v) 29 | (-unwrap v) 30 | v)) -------------------------------------------------------------------------------- /src/xitdb/db.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.db 2 | (:require 3 | [xitdb.common :as common] 4 | [xitdb.util.conversion :as conversion] 5 | [xitdb.xitdb-types :as xtypes]) 6 | (:import 7 | [io.github.radarroark.xitdb 8 | CoreBufferedFile CoreFile CoreMemory Database Database$ContextFunction Hasher 9 | RandomAccessBufferedFile RandomAccessMemory ReadArrayList WriteArrayList WriteCursor] 10 | [java.io File RandomAccessFile] 11 | [java.security MessageDigest] 12 | [java.util.concurrent.locks ReentrantLock])) 13 | 14 | ;; When set to true, 15 | ;; swap! will return [current-history-index old-dbval new-dbval] 16 | (defonce ^:dynamic *return-history?* false) 17 | 18 | ;; Avoid extra require in your ns 19 | (def materialize common/materialize) 20 | 21 | (defn open-database 22 | "Opens database `filename`. 23 | If `filename` is `:memory`, returns a memory based db. 24 | open-mode can be `r` or `rw`." 25 | [filename ^String open-mode] 26 | (let [core (if (= filename :memory) 27 | (CoreMemory. (RandomAccessMemory.)) 28 | (CoreBufferedFile. (RandomAccessBufferedFile. (File. ^String filename) open-mode))) 29 | hasher (Hasher. (MessageDigest/getInstance "SHA-1"))] 30 | (Database. core hasher))) 31 | 32 | 33 | (defn ^WriteArrayList db-history [^Database db] 34 | (WriteArrayList. (.rootCursor db))) 35 | 36 | (defn append-context! 37 | "Appends a new history context and calls `fn` with a write cursor. 38 | Returns the new history index." 39 | [^WriteArrayList history slot fn] 40 | (.appendContext 41 | history 42 | slot 43 | (reify Database$ContextFunction 44 | (^void run [_ ^WriteCursor cursor] 45 | (fn cursor) 46 | nil))) 47 | (.count history)) 48 | 49 | (defn xitdb-reset! 50 | "Sets the value of the database to `new-value`. 51 | Returns new history index." 52 | [^WriteArrayList history new-value] 53 | (append-context! history nil (fn [^WriteCursor cursor] 54 | (conversion/v->slot! cursor new-value)))) 55 | 56 | (defn v->slot! 57 | "Converts a value to a slot which can be written to a cursor. 58 | For XITDB* types (which support ISlot), will return `-slot`, 59 | for all other types `conversion/v->slot!`" 60 | [^WriteCursor cursor v] 61 | (if (satisfies? common/ISlot v) 62 | (common/-slot v) 63 | (conversion/v->slot! cursor v))) 64 | 65 | (defn xitdb-swap! 66 | "Starts a new transaction and calls `f` with the value at `base-keypath`. 67 | If `base-keypath` is nil, will use the root cursor. 68 | `f` will receive a XITDBWrite* type with the value at `base-keypath` and `args`. 69 | Actions on the XITDBWrite* type (like `assoc`) will mutate it. 70 | Return value of `f` is written at `base-keypath` (or root) cursor. 71 | Returns the transaction history index." 72 | [db base-keypath f & args] 73 | (let [history (db-history db) 74 | slot (.getSlot history -1)] 75 | (append-context! 76 | history 77 | slot 78 | (fn [^WriteCursor cursor] 79 | (let [cursor (conversion/keypath-cursor cursor base-keypath) 80 | obj (xtypes/read-from-cursor cursor true)] 81 | (let [retval (apply f (into [obj] args))] 82 | (.write cursor (v->slot! cursor retval)))))))) 83 | 84 | (defn xitdb-swap-with-lock! 85 | "Performs the 'swap!' operation while locking `db.lock`. 86 | Returns the new value of the database. 87 | If the binding `*return-history?*` is true, returns 88 | `[current-history-index db-before db-after]`. 89 | If `keypath` is not empty, the result of `f` will be written to the db at `keypath` rather 90 | than db root. 91 | Similarly, if `keypath` is not empty, the returned value will be the value at `keypath`." 92 | [xitdb base-keypath f & args] 93 | (let [^ReentrantLock lock (.-lock xitdb)] 94 | (when (.isHeldByCurrentThread lock) 95 | (throw (IllegalStateException. "swap! should not be called from swap! or reset!"))) 96 | (try 97 | (.lock lock) 98 | (let [old-value (when *return-history?* (deref xitdb)) 99 | index (apply xitdb-swap! (into [(-> xitdb .rwdb) base-keypath f] args)) 100 | new-value (deref xitdb)] 101 | (if *return-history?* 102 | [index old-value new-value] 103 | new-value)) 104 | (finally 105 | (.unlock lock))))) 106 | 107 | (defn- close-db-internal! 108 | "Closes the db file. Does nothing if it's a memory db" 109 | [^Database db] 110 | (let [core (-> db .-core)] 111 | (when (instance? CoreFile core) 112 | (.close ^RandomAccessFile (-> db .-core .file))))) 113 | 114 | 115 | (defn ^ReadArrayList read-history 116 | "Returns the read only transaction history array." 117 | [^Database db] 118 | (ReadArrayList. (-> db .rootCursor))) 119 | 120 | (defn history-index 121 | "Returns the current size of the transaction history array." 122 | [xdb] 123 | (.count (read-history (-> xdb .tldbro .get)))) 124 | 125 | (deftype XITDBDatabase [tldbro rwdb lock] 126 | 127 | java.io.Closeable 128 | (close [this] 129 | (close-db-internal! (.get tldbro)) 130 | (close-db-internal! rwdb)) 131 | 132 | clojure.lang.IDeref 133 | (deref [this] 134 | (let [history (read-history (.get tldbro)) 135 | cursor (.getCursor history -1)] 136 | (xtypes/read-from-cursor cursor false))) 137 | 138 | clojure.lang.IAtom 139 | 140 | (reset [this new-value] 141 | 142 | (when (.isHeldByCurrentThread lock) 143 | (throw (IllegalStateException. "reset! should not be called from swap! or reset!"))) 144 | 145 | (try 146 | (.lock lock) 147 | (let [history (db-history rwdb)] 148 | (xitdb-reset! history new-value) 149 | (deref this)) 150 | (finally 151 | (.unlock lock)))) 152 | 153 | (swap [this f] 154 | (xitdb-swap-with-lock! this nil f)) 155 | 156 | (swap [this f a] 157 | (xitdb-swap-with-lock! this nil f a)) 158 | 159 | (swap [this f a1 a2] 160 | (xitdb-swap-with-lock! this nil f a1 a2)) 161 | 162 | (swap [this f x y args] 163 | (apply xitdb-swap-with-lock! (concat [this nil f x y] args)))) 164 | 165 | (defn xit-db 166 | "Returns a new XITDBDatabase which can be used to query and transact data. 167 | `filename` can be `:memory` or the name of a file on the filesystem. 168 | If the file does not exist, it will be created. 169 | The returned database handle can be used from multiple threads. 170 | Reads can run in parallel, transactions (eg. `swap!`) will only allow one writer at a time." 171 | [filename] 172 | (if (= :memory filename) 173 | (let [memdb (open-database :memory "rw") 174 | tdbmem (proxy [ThreadLocal] [] 175 | (initialValue [] 176 | memdb))] 177 | (->XITDBDatabase tdbmem memdb (ReentrantLock.))) 178 | 179 | (let [tldb (proxy [ThreadLocal] [] 180 | (initialValue [] 181 | (open-database filename "r"))) 182 | rwdb (open-database filename "rw")] 183 | (->XITDBDatabase tldb rwdb (ReentrantLock.))))) 184 | 185 | 186 | (deftype XITDBCursor [xdb keypath] 187 | 188 | java.io.Closeable 189 | (close [this]) 190 | 191 | clojure.lang.IDeref 192 | (deref [this] 193 | (let [v (deref xdb)] 194 | (get-in v keypath))) 195 | 196 | clojure.lang.IAtom 197 | 198 | (reset [this new-value] 199 | (xitdb-swap-with-lock! xdb keypath (constantly new-value))) 200 | 201 | (swap [this f] 202 | (xitdb-swap-with-lock! xdb keypath f)) 203 | 204 | (swap [this f a] 205 | (xitdb-swap-with-lock! xdb keypath f a)) 206 | 207 | (swap [this f a1 a2] 208 | (xitdb-swap-with-lock! xdb keypath f a1 a2)) 209 | 210 | (swap [this f x y args] 211 | (apply xitdb-swap-with-lock! (concat [xdb keypath f x y] args)))) 212 | 213 | (defn xdb-cursor [xdb keypath] 214 | (cond 215 | (instance? XITDBCursor xdb) 216 | (XITDBCursor. (.-xdb xdb) (vec (concat (.-keypath xdb) keypath))) 217 | 218 | (instance? XITDBDatabase xdb) 219 | (XITDBCursor. xdb keypath) 220 | 221 | :else 222 | (throw (IllegalArgumentException. (str "xdb must be an instance of XITDBCursor or XITDBDatabase, got: " (type xdb)))))) 223 | 224 | 225 | -------------------------------------------------------------------------------- /src/xitdb/hash_map.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.hash-map 2 | (:require 3 | [xitdb.common :as common] 4 | [xitdb.util.conversion :as conversion] 5 | [xitdb.util.operations :as operations]) 6 | (:import 7 | [io.github.radarroark.xitdb 8 | ReadCountedHashMap ReadCursor ReadHashMap 9 | WriteCountedHashMap WriteCursor WriteHashMap])) 10 | 11 | (defn map-seq 12 | [rhm] 13 | "The cursors used must implement the IReadFromCursor protocol." 14 | (operations/map-seq rhm #(common/-read-from-cursor %))) 15 | 16 | (deftype XITDBHashMap [^ReadHashMap rhm] 17 | 18 | clojure.lang.ILookup 19 | (valAt [this key] 20 | (.valAt this key nil)) 21 | 22 | (valAt [this key not-found] 23 | (let [cursor (operations/map-read-cursor rhm key)] 24 | (if (nil? cursor) 25 | not-found 26 | (common/-read-from-cursor cursor)))) 27 | 28 | clojure.lang.Associative 29 | (containsKey [this key] 30 | (operations/map-contains-key? rhm key)) 31 | 32 | (entryAt [this key] 33 | (when (.containsKey this key) 34 | (let [v (.valAt this key nil)] 35 | (clojure.lang.MapEntry. key v)))) 36 | 37 | (assoc [this k v] 38 | (throw (UnsupportedOperationException. "XITDBHashMap is read-only"))) 39 | 40 | clojure.lang.IPersistentMap 41 | (without [_ _] 42 | (throw (UnsupportedOperationException. "XITDBHashMap is read-only"))) 43 | 44 | (count [this] 45 | (operations/map-item-count rhm)) 46 | 47 | clojure.lang.IPersistentCollection 48 | (cons [_ _] 49 | (throw (UnsupportedOperationException. "XITDBHashMap is read-only"))) 50 | 51 | (empty [_] 52 | (throw (UnsupportedOperationException. "XITDBHashMap is read-only"))) 53 | 54 | (equiv [this other] 55 | (and (instance? clojure.lang.IPersistentMap other) 56 | (= (into {} this) (into {} other)))) 57 | 58 | clojure.lang.Seqable 59 | (seq [this] 60 | (map-seq rhm)) 61 | 62 | clojure.lang.IFn 63 | (invoke [this k] 64 | (.valAt this k)) 65 | 66 | (invoke [this k not-found] 67 | (.valAt this k not-found)) 68 | 69 | java.lang.Iterable 70 | (iterator [this] 71 | (let [iter (clojure.lang.SeqIterator. (seq this))] 72 | (reify java.util.Iterator 73 | (hasNext [_] 74 | (.hasNext iter)) 75 | (next [_] 76 | (.next iter)) 77 | (remove [_] 78 | (throw (UnsupportedOperationException. "XITDBHashMap iterator is read-only")))))) 79 | 80 | clojure.core.protocols/IKVReduce 81 | (kv-reduce [this f init] 82 | (operations/map-kv-reduce rhm #(common/-read-from-cursor %) f init)) 83 | 84 | common/IUnwrap 85 | (-unwrap [this] 86 | rhm) 87 | 88 | Object 89 | (toString [this] 90 | (str (into {} this)))) 91 | 92 | (defmethod print-method XITDBHashMap [o ^java.io.Writer w] 93 | (.write w "#XITDBHashMap") 94 | (print-method (into {} o) w)) 95 | 96 | (extend-protocol common/IMaterialize 97 | XITDBHashMap 98 | (-materialize [this] 99 | (reduce (fn [m [k v]] 100 | (assoc m k (common/materialize v))) {} (seq this)))) 101 | 102 | ;--------------------------------------------------- 103 | 104 | 105 | (deftype XITDBWriteHashMap [^WriteHashMap whm] 106 | clojure.lang.IPersistentCollection 107 | (cons [this o] 108 | 109 | (cond 110 | (instance? clojure.lang.MapEntry o) 111 | (.assoc this (key o) (val o)) 112 | 113 | (map? o) 114 | (doseq [[k v] (seq o)] 115 | (.assoc this k v)) 116 | 117 | (and (sequential? o) (= 2 (count o))) 118 | (do 119 | (.assoc this (first o) (second o))) 120 | 121 | :else 122 | (throw (IllegalArgumentException. "Can only cons MapEntries or key-value pairs onto maps"))) 123 | this) 124 | 125 | (empty [this] 126 | (operations/map-empty! whm) 127 | this) 128 | 129 | (equiv [this other] 130 | (and (= (count this) (count other)) 131 | (every? (fn [[k v]] (= v (get other k ::not-found))) 132 | (seq this)))) 133 | clojure.lang.Associative 134 | (assoc [this k v] 135 | (operations/map-assoc-value! whm k (common/unwrap v)) 136 | this) 137 | 138 | (containsKey [this key] 139 | (operations/map-contains-key? whm key)) 140 | 141 | (entryAt [this key] 142 | (when (.containsKey this key) 143 | (clojure.lang.MapEntry. key (.valAt this key)))) 144 | 145 | clojure.lang.IPersistentMap 146 | (without [this key] 147 | (operations/map-dissoc-key! whm key) 148 | this) 149 | 150 | (count [this] 151 | (operations/map-item-count whm)) 152 | 153 | clojure.lang.ILookup 154 | (valAt [this key] 155 | (.valAt this key nil)) 156 | 157 | (valAt [this key not-found] 158 | (let [cursor (operations/map-read-cursor whm key)] 159 | (if (nil? cursor) 160 | not-found 161 | (common/-read-from-cursor (conversion/map-write-cursor whm key))))) 162 | 163 | clojure.lang.Seqable 164 | (seq [this] 165 | (map-seq whm)) 166 | 167 | clojure.core.protocols/IKVReduce 168 | (kv-reduce [this f init] 169 | (operations/map-kv-reduce whm #(common/-read-from-cursor %) f init)) 170 | 171 | common/ISlot 172 | (-slot [this] 173 | (-> whm .cursor .slot)) 174 | 175 | common/IUnwrap 176 | (-unwrap [this] 177 | whm) 178 | 179 | Object 180 | (toString [this] 181 | (str "XITDBWriteHashMap"))) 182 | 183 | 184 | (defn xwrite-hash-map [^WriteCursor write-cursor] 185 | (->XITDBWriteHashMap (WriteHashMap. write-cursor))) 186 | 187 | (defn xhash-map [^ReadCursor read-cursor] 188 | (->XITDBHashMap (ReadHashMap. read-cursor))) 189 | 190 | (defn xwrite-hash-map-counted [^WriteCursor write-cursor] 191 | (->XITDBWriteHashMap (WriteCountedHashMap. write-cursor))) 192 | 193 | (defn xhash-map-counted [^ReadCursor read-cursor] 194 | (->XITDBHashMap (ReadCountedHashMap. read-cursor))) 195 | 196 | 197 | 198 | -------------------------------------------------------------------------------- /src/xitdb/hash_set.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.hash-set 2 | (:require 3 | [xitdb.common :as common] 4 | [xitdb.util.conversion :as conversion] 5 | [xitdb.util.operations :as operations]) 6 | (:import 7 | [io.github.radarroark.xitdb 8 | ReadCountedHashSet ReadCursor ReadHashSet 9 | WriteCountedHashSet WriteCursor WriteHashSet])) 10 | 11 | (defn set-seq 12 | [rhm] 13 | "The cursors used must implement the IReadFromCursor protocol." 14 | (operations/set-seq rhm common/-read-from-cursor)) 15 | 16 | (deftype XITDBHashSet [^ReadHashSet rhs] 17 | clojure.lang.IPersistentSet 18 | (disjoin [_ k] 19 | (throw (UnsupportedOperationException. "XITDBHashSet is read-only"))) 20 | 21 | (contains [this k] 22 | (operations/set-contains? rhs k)) 23 | 24 | (get [this k] 25 | (when (.contains this k) 26 | k)) 27 | 28 | clojure.lang.IPersistentCollection 29 | (cons [_ o] 30 | (throw (UnsupportedOperationException. "XITDBHashSet is read-only"))) 31 | 32 | (empty [_] 33 | (throw (UnsupportedOperationException. "XITDBHashSet is read-only"))) 34 | 35 | (equiv [this other] 36 | (and (instance? clojure.lang.IPersistentSet other) 37 | (= (count this) (count other)) 38 | (every? #(.contains this %) other))) 39 | 40 | (count [_] 41 | (operations/set-item-count rhs)) 42 | 43 | clojure.lang.Seqable 44 | (seq [_] 45 | (set-seq rhs)) 46 | 47 | clojure.lang.ILookup 48 | (valAt [this k] 49 | (.valAt this k nil)) 50 | 51 | (valAt [this k not-found] 52 | (if (.contains this k) 53 | k 54 | not-found)) 55 | 56 | clojure.lang.IFn 57 | (invoke [this k] 58 | (.valAt this k)) 59 | 60 | java.lang.Iterable 61 | (iterator [this] 62 | (let [iter (clojure.lang.SeqIterator. (seq this))] 63 | (reify java.util.Iterator 64 | (hasNext [_] 65 | (.hasNext iter)) 66 | (next [_] 67 | (.next iter)) 68 | (remove [_] 69 | (throw (UnsupportedOperationException. "XITDBHashSet iterator is read-only")))))) 70 | 71 | common/IUnwrap 72 | (-unwrap [_] 73 | rhs) 74 | 75 | Object 76 | (toString [this] 77 | (str (into #{} this)))) 78 | 79 | (defmethod print-method XITDBHashSet [o ^java.io.Writer w] 80 | (.write w "#XITDBHashSet") 81 | (print-method (into #{} o) w)) 82 | 83 | (extend-protocol common/IMaterialize 84 | XITDBHashSet 85 | (-materialize [this] 86 | (into #{} (map common/materialize (seq this))))) 87 | 88 | ;; Writable version of the set 89 | (deftype XITDBWriteHashSet [^WriteHashSet whs] 90 | clojure.lang.IPersistentSet 91 | (disjoin [this v] 92 | (operations/set-disj-value! whs (common/unwrap v)) 93 | this) 94 | 95 | (contains [this v] 96 | (operations/set-contains? whs (common/unwrap v))) 97 | 98 | (get [this k] 99 | (when (.contains this (common/unwrap k)) 100 | k)) 101 | 102 | clojure.lang.IPersistentCollection 103 | (cons [this o] 104 | (operations/set-assoc-value! whs (common/unwrap o)) 105 | this) 106 | 107 | (empty [this] 108 | (operations/set-empty! whs) 109 | this) 110 | 111 | (equiv [this other] 112 | (and (instance? clojure.lang.IPersistentSet other) 113 | (= (count this) (count other)) 114 | (every? #(.contains this %) other))) 115 | 116 | (count [_] 117 | (operations/set-item-count whs)) 118 | 119 | clojure.lang.Seqable 120 | (seq [_] 121 | (set-seq whs)) 122 | 123 | clojure.lang.ILookup 124 | (valAt [this k] 125 | (.valAt this k nil)) 126 | 127 | (valAt [this k not-found] 128 | (if (.contains this k) 129 | k 130 | not-found)) 131 | 132 | common/ISlot 133 | (-slot [_] 134 | (-> whs .cursor .slot)) 135 | 136 | common/IUnwrap 137 | (-unwrap [_] 138 | whs) 139 | 140 | Object 141 | (toString [_] 142 | (str "XITDBWriteHashSet"))) 143 | 144 | ;; Constructor functions 145 | (defn xwrite-hash-set [^WriteCursor write-cursor] 146 | (->XITDBWriteHashSet (WriteHashSet. write-cursor))) 147 | 148 | (defn xhash-set [^ReadCursor read-cursor] 149 | (->XITDBHashSet (ReadHashSet. read-cursor))) 150 | 151 | (defn xwrite-hash-set-counted [^WriteCursor write-cursor] 152 | (->XITDBWriteHashSet (WriteCountedHashSet. write-cursor))) 153 | 154 | (defn xhash-set-counted [^ReadCursor cursor] 155 | (->XITDBHashSet (ReadCountedHashSet. cursor))) -------------------------------------------------------------------------------- /src/xitdb/linked_list.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.linked-list 2 | (:require 3 | [xitdb.common :as common] 4 | [xitdb.util.conversion :as conversion] 5 | [xitdb.util.operations :as operations]) 6 | (:import 7 | [io.github.radarroark.xitdb ReadCursor ReadLinkedArrayList WriteCursor WriteLinkedArrayList])) 8 | 9 | (defn array-seq 10 | [^ReadLinkedArrayList rlal] 11 | "The cursors used must implement the IReadFromCursor protocol." 12 | (operations/linked-array-seq rlal #(common/-read-from-cursor %))) 13 | 14 | (deftype XITDBLinkedArrayList [^ReadLinkedArrayList rlal] 15 | clojure.lang.IPersistentCollection 16 | (seq [_] 17 | (array-seq rlal)) 18 | 19 | (count [_] 20 | (.count rlal)) 21 | 22 | (cons [_ o] 23 | (throw (UnsupportedOperationException. "XITDBLinkedArrayList is read-only"))) 24 | 25 | (empty [_] 26 | (throw (UnsupportedOperationException. "XITDBLinkedArrayList is read-only"))) 27 | 28 | (equiv [this other] 29 | (and (sequential? other) 30 | (= (count this) (count other)) 31 | (every? identity (map = this other)))) 32 | 33 | clojure.lang.Sequential ;; Mark as sequential 34 | 35 | clojure.lang.Indexed 36 | (nth [_ i] 37 | (let [cursor (.getCursor rlal (long i))] 38 | (common/-read-from-cursor cursor))) 39 | 40 | (nth [_ i not-found] 41 | (let [cursor (.getCursor rlal (long i))] 42 | (if cursor 43 | (common/-read-from-cursor cursor) 44 | not-found))) 45 | 46 | clojure.lang.ILookup 47 | (valAt [this k] 48 | (if (number? k) 49 | (.nth this (long k)) 50 | (throw (IllegalArgumentException. "Key must be a number")))) 51 | 52 | (valAt [this k not-found] 53 | (if (number? k) 54 | (.nth this (long k) not-found) 55 | not-found)) 56 | 57 | clojure.lang.IFn 58 | (invoke [this k] 59 | (.valAt this k)) 60 | 61 | (invoke [this k not-found] 62 | (.valAt this k not-found)) 63 | 64 | (applyTo [this args] 65 | (case (count args) 66 | 1 (.invoke this (first args)) 67 | 2 (.invoke this (first args) (second args)) 68 | (throw (IllegalArgumentException. "Wrong number of args passed to XITDBLinkedArrayList")))) 69 | 70 | clojure.lang.IReduceInit 71 | (reduce [this f init] 72 | (reduce f init (array-seq rlal))) 73 | 74 | java.util.Collection 75 | (^objects toArray [this] 76 | (to-array (into [] this))) 77 | 78 | (^objects toArray [this ^objects array] 79 | (let [len (count this) 80 | ^objects result (if (or (nil? array) (< (alength array) len)) 81 | (make-array Object len) 82 | array)] 83 | (dotimes [i len] 84 | (aset result i (nth this i))) 85 | (when (> (alength result) len) 86 | (aset result len nil)) 87 | result)) 88 | 89 | Object 90 | (toString [this] 91 | (pr-str (into [] this)))) 92 | 93 | (defmethod print-method XITDBLinkedArrayList [o ^java.io.Writer w] 94 | (.write w "#XITDBLinkedArrayList") 95 | (print-method (into [] o) w)) 96 | 97 | (extend-protocol common/IMaterialize 98 | XITDBLinkedArrayList 99 | (-materialize [this] 100 | (reduce (fn [a v] 101 | (conj a (common/materialize v))) [] (seq this)))) 102 | 103 | ;; ----------------------------------------------------------------- 104 | 105 | (deftype XITDBWriteLinkedArrayList [^WriteLinkedArrayList wlal] 106 | clojure.lang.IPersistentCollection 107 | (count [this] 108 | (.count wlal)) 109 | 110 | (cons [this o] 111 | ;; TODO: This should insert at position 0 112 | (operations/linked-array-list-insert-value! wlal 0 (common/unwrap o)) 113 | this) 114 | 115 | (empty [this] 116 | ;; Assuming similar empty behavior as arrays 117 | (let [^WriteCursor cursor (-> wlal .cursor)] 118 | (.write cursor (conversion/v->slot! cursor (list)))) 119 | this) 120 | 121 | (equiv [this other] 122 | (if (instance? XITDBWriteLinkedArrayList other) 123 | (and (= (count this) (count other)) 124 | (every? (fn [i] (= (get this i) (get other i))) 125 | (range (count this)))) 126 | false)) 127 | 128 | clojure.lang.Indexed 129 | (nth [this i] 130 | (.nth this i nil)) 131 | 132 | (nth [this i not-found] 133 | (if (and (>= i 0) (< i (.count wlal))) 134 | (common/-read-from-cursor (.putCursor wlal i)) 135 | not-found)) 136 | 137 | clojure.lang.ILookup 138 | (valAt [this k] 139 | (.valAt this k nil)) 140 | 141 | (valAt [this k not-found] 142 | (.nth this k not-found)) 143 | 144 | clojure.lang.Seqable 145 | (seq [this] 146 | (array-seq wlal)) 147 | 148 | clojure.lang.IObj 149 | (withMeta [this _] 150 | this) 151 | 152 | clojure.lang.IMeta 153 | (meta [this] 154 | nil) 155 | 156 | clojure.lang.IEditableCollection 157 | (asTransient [this] 158 | this) 159 | 160 | clojure.lang.ITransientCollection 161 | (conj [this val] 162 | (operations/linked-array-list-append-value! wlal (common/unwrap val)) 163 | this) 164 | 165 | (persistent [this] 166 | this) 167 | 168 | clojure.lang.IPersistentStack 169 | (peek [this] 170 | (if (pos? (.count wlal)) 171 | (common/-read-from-cursor (.getCursor wlal 0)) 172 | nil)) 173 | 174 | (pop [this] 175 | (if (pos? (.count wlal)) 176 | (operations/linked-array-list-pop! wlal) 177 | (throw (IllegalStateException. "Can't pop empty list"))) 178 | this) 179 | 180 | clojure.lang.IPersistentList 181 | ;; No additional methods needed, IPersistentList just extends IPersistentStack 182 | 183 | common/ISlot 184 | (-slot [this] 185 | (-> wlal .cursor .slot)) 186 | 187 | common/IUnwrap 188 | (-unwrap [this] 189 | wlal) 190 | 191 | Object 192 | (toString [this] 193 | (str "XITDBWriteLinkedArrayList"))) 194 | 195 | (extend-protocol common/IMaterialize 196 | XITDBLinkedArrayList 197 | (-materialize [this] 198 | (apply list 199 | (reduce (fn [a v] 200 | (conj a (common/materialize v))) [] (seq this))))) 201 | 202 | ;; Constructors 203 | 204 | (defn xlinked-list [^ReadCursor cursor] 205 | (->XITDBLinkedArrayList (ReadLinkedArrayList. cursor))) 206 | 207 | (defn xwrite-linked-list [^WriteCursor write-cursor] 208 | (->XITDBWriteLinkedArrayList (WriteLinkedArrayList. write-cursor))) -------------------------------------------------------------------------------- /src/xitdb/snapshot.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.snapshot 2 | (:require [xitdb.db :as xdb]) 3 | (:import [java.io File])) 4 | 5 | (defn xit-db-existing [filename] 6 | (when-not (or (= filename :memory) (.exists (File. ^String filename))) 7 | (throw (IllegalArgumentException. "Database file does not exist"))) 8 | (xdb/xit-db filename)) 9 | 10 | (defn snapshot-memory-db 11 | "Returns a memory database with the value of `keypath` in the database at `filename` 12 | When keypath is [], returns a memdb with all the data in the db `filename`. 13 | Useful for REPL-based investigation and testing." 14 | [filename keypath] 15 | (with-open [db (xit-db-existing filename)] 16 | (let [memdb (xdb/xit-db :memory)] 17 | (reset! memdb (get-in @db keypath)) 18 | memdb))) -------------------------------------------------------------------------------- /src/xitdb/util/conversion.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.util.conversion 2 | (:require 3 | [xitdb.util.validation :as validation]) 4 | (:import 5 | [io.github.radarroark.xitdb 6 | Database Database$Bytes Database$Float Database$Int 7 | ReadArrayList ReadCountedHashMap ReadCountedHashSet ReadCursor ReadHashMap 8 | ReadHashSet Slot Tag WriteArrayList WriteCountedHashMap WriteCountedHashSet WriteCursor 9 | WriteHashMap WriteHashSet WriteLinkedArrayList] 10 | [java.io OutputStream OutputStreamWriter] 11 | [java.security DigestOutputStream])) 12 | 13 | (defn xit-tag->keyword 14 | "Converts a XitDB Tag enum to a corresponding Clojure keyword." 15 | [tag] 16 | (cond 17 | (= tag Tag/NONE) :none 18 | (= tag Tag/INDEX) :index 19 | (= tag Tag/ARRAY_LIST) :array-list 20 | (= tag Tag/LINKED_ARRAY_LIST) :linked-array-list 21 | (= tag Tag/HASH_MAP) :hash-map 22 | (= tag Tag/KV_PAIR) :kv-pair 23 | (= tag Tag/BYTES) :bytes 24 | (= tag Tag/SHORT_BYTES) :short-bytes 25 | (= tag Tag/UINT) :uint 26 | (= tag Tag/INT) :int 27 | (= tag Tag/FLOAT) :float 28 | :else :unknown)) 29 | 30 | (defn fmt-tag-keyword [v] 31 | (cond 32 | (keyword? v) :keyword 33 | (boolean? v) :boolean 34 | (integer? v) :key-integer 35 | (instance? java.time.Instant v) :inst 36 | (instance? java.util.Date v) :date 37 | (coll? v) :coll 38 | (string? v) :string)) 39 | 40 | ;; map of logical tag -> string used as formatTag in the Bytes record. 41 | (def fmt-tag-value 42 | {:keyword "kw" 43 | :boolean "bl" 44 | :key-integer "ki" 45 | :nil "nl" ;; TODO: Could use Tag/NONE instead 46 | :inst "in" 47 | :date "da" 48 | :coll "co" 49 | :string "st"}) 50 | 51 | (def true-str "#t") 52 | (def false-str "#f") 53 | 54 | (defn ^Database$Bytes database-bytes 55 | ([^String s] 56 | (Database$Bytes. s)) 57 | ([^String s ^String tag] 58 | (Database$Bytes. s tag))) 59 | 60 | (defn ^String keyname [key] 61 | (if (keyword? key) 62 | (if (namespace key) 63 | (str (namespace key) "/" (name key)) 64 | (name key)) 65 | key)) 66 | 67 | (defn db-key-hash 68 | "Returns a byte array representing the stable hash digest of (Clojure) value `v`. 69 | Uses the MessageDigest from the database." 70 | ^bytes [^Database jdb v] 71 | (if (nil? v) 72 | (byte-array (-> jdb .md .getDigestLength)) 73 | (let [digest (.md jdb) 74 | fmt-tag (or (some-> v fmt-tag-keyword fmt-tag-value) 75 | (throw (IllegalArgumentException. (str "Unsupported key type: " (type v)))))] 76 | ;; add format tag 77 | (.update digest (.getBytes fmt-tag "UTF-8")) 78 | ;; add the value 79 | (cond 80 | (validation/lazy-seq? v) 81 | (throw (IllegalArgumentException. "Lazy sequences can be infinite and not allowed!")) 82 | 83 | (bytes? v) 84 | (.update digest v) 85 | 86 | (instance? Database$Bytes v) 87 | (.update digest (.value v)) 88 | 89 | (coll? v) 90 | (with-open [os (DigestOutputStream. (OutputStream/nullOutputStream) digest)] 91 | (with-open [writer (OutputStreamWriter. os)] 92 | (binding [*out* writer] 93 | (pr v)))) 94 | 95 | :else 96 | (.update digest (.getBytes (str v) "UTF-8"))) 97 | ;; finish hash 98 | (.digest digest)))) 99 | 100 | (defn ^Slot primitive-for 101 | "Converts a Clojure primitive value to its corresponding XitDB representation. 102 | Handles strings, keywords, integers, booleans, and floats. 103 | Throws an IllegalArgumentException for unsupported types." 104 | [v] 105 | (cond 106 | 107 | (validation/lazy-seq? v) 108 | (throw (IllegalArgumentException. "Lazy sequences can be infinite and not allowed!")) 109 | 110 | (string? v) 111 | (database-bytes v) 112 | 113 | (keyword? v) 114 | (database-bytes (keyname v) (fmt-tag-value :keyword)) 115 | 116 | (integer? v) 117 | (Database$Int. v) 118 | 119 | (boolean? v) 120 | (database-bytes (if v true-str false-str) (fmt-tag-value :boolean)) 121 | 122 | (double? v) 123 | (Database$Float. v) 124 | 125 | (nil? v) 126 | (database-bytes "" (fmt-tag-value :nil)) 127 | 128 | (instance? java.time.Instant v) 129 | (database-bytes (str v) (fmt-tag-value :inst)) 130 | 131 | (instance? java.util.Date v) 132 | (database-bytes (str (.toInstant ^java.util.Date v)) (fmt-tag-value :date)) 133 | 134 | :else 135 | (throw (IllegalArgumentException. (str "Unsupported type: " (type v) v))))) 136 | 137 | (declare ^WriteCursor map->WriteHashMapCursor!) 138 | (declare ^WriteCursor coll->ArrayListCursor!) 139 | (declare ^WriteCursor list->LinkedArrayListCursor!) 140 | (declare ^WriteCursor set->WriteCursor!) 141 | 142 | (defn ^Slot v->slot! 143 | "Converts a value to a XitDB slot. 144 | Handles WriteArrayList and WriteHashMap instances directly. 145 | Recursively processes Clojure maps and collections. 146 | Falls back to primitive conversion for other types." 147 | [^WriteCursor cursor v] 148 | (cond 149 | 150 | (instance? WriteArrayList v) 151 | (-> ^WriteArrayList v .cursor .slot) 152 | 153 | (instance? WriteLinkedArrayList v) 154 | (-> ^WriteLinkedArrayList v .cursor .slot) 155 | 156 | (instance? WriteHashMap v) 157 | (-> ^WriteHashMap v .cursor .slot) 158 | 159 | (instance? ReadHashMap v) 160 | (-> ^ReadHashMap v .cursor .slot) 161 | 162 | (instance? ReadCountedHashMap v) 163 | (-> ^ReadCountedHashMap v .cursor .slot) 164 | 165 | (instance? WriteCountedHashMap v) 166 | (-> ^WriteCountedHashMap v .cursor .slot) 167 | 168 | (instance? ReadArrayList v) 169 | (-> ^ReadArrayList v .cursor .slot) 170 | 171 | (instance? ReadHashSet v) 172 | (-> ^ReadHashSet v .cursor .slot) 173 | 174 | (instance? ReadCountedHashSet v) 175 | (-> ^ReadCountedHashSet v .cursor .slot) 176 | 177 | (instance? WriteHashSet v) 178 | (-> ^WriteHashSet v .cursor .slot) 179 | 180 | (instance? WriteCountedHashSet v) 181 | (-> ^WriteCountedHashSet v .cursor .slot) 182 | 183 | (map? v) 184 | (do 185 | (.write cursor nil) 186 | (.slot (map->WriteHashMapCursor! cursor v))) 187 | 188 | (validation/list-or-cons? v) 189 | (do 190 | (.write cursor nil) 191 | (.slot (list->LinkedArrayListCursor! cursor v))) 192 | 193 | (validation/vector-or-chunked? v) 194 | (do 195 | (.write cursor nil) 196 | (.slot (coll->ArrayListCursor! cursor v))) 197 | 198 | (set? v) 199 | (do 200 | (.write cursor nil) 201 | (.slot (set->WriteCursor! cursor v))) 202 | 203 | :else 204 | (primitive-for v))) 205 | 206 | (def ^:dynamic *debug?* false) 207 | 208 | (defn ^WriteCursor coll->ArrayListCursor! 209 | "Converts a Clojure collection to a XitDB ArrayList cursor. 210 | Handles nested maps and collections recursively. 211 | Returns the cursor of the created WriteArrayList." 212 | [^WriteCursor cursor coll] 213 | (when *debug?* (println "Write array" (type coll))) 214 | (let [write-array (WriteArrayList. cursor)] 215 | (doseq [v coll] 216 | (cond 217 | (map? v) 218 | (let [v-cursor (.appendCursor write-array)] 219 | (map->WriteHashMapCursor! v-cursor v)) 220 | 221 | (validation/list-or-cons? v) 222 | (let [v-cursor (.appendCursor write-array)] 223 | (list->LinkedArrayListCursor! v-cursor v)) 224 | 225 | (validation/vector-or-chunked? v) 226 | (let [v-cursor (.appendCursor write-array)] 227 | (coll->ArrayListCursor! v-cursor v)) 228 | 229 | :else 230 | (.append write-array (primitive-for v)))) 231 | (.-cursor write-array))) 232 | 233 | (defn ^WriteCursor list->LinkedArrayListCursor! 234 | "Converts a Clojure list or seq-like collection to a XitDB LinkedArrayList cursor. 235 | Optimized for sequential access collections rather than random access ones." 236 | [^WriteCursor cursor coll] 237 | (when *debug?* (println "Write list" (type coll))) 238 | (let [write-list (WriteLinkedArrayList. cursor)] 239 | (doseq [v coll] 240 | (when *debug?* (println "v=" v)) 241 | (cond 242 | (map? v) 243 | (let [v-cursor (.appendCursor write-list)] 244 | (map->WriteHashMapCursor! v-cursor v)) 245 | 246 | (validation/lazy-seq? v) 247 | (throw (IllegalArgumentException. "Lazy sequences can be infinite and not allowed !")) 248 | 249 | (validation/list-or-cons? v) 250 | (let [v-cursor (.appendCursor write-list)] 251 | (list->LinkedArrayListCursor! v-cursor v)) 252 | 253 | (validation/vector-or-chunked? v) 254 | (let [v-cursor (.appendCursor write-list)] 255 | (coll->ArrayListCursor! v-cursor v)) 256 | 257 | :else 258 | (.append write-list (primitive-for v)))) 259 | (.-cursor write-list))) 260 | 261 | (defn ^WriteCursor map->WriteHashMapCursor! 262 | "Writes a Clojure map to a XitDB WriteHashMap. 263 | Returns the cursor of the created WriteHashMap." 264 | [^WriteCursor cursor m] 265 | (let [whm (WriteCountedHashMap. cursor)] 266 | (doseq [[k v] m] 267 | (let [hash-value (db-key-hash (-> cursor .db) k) 268 | key-cursor (.putKeyCursor whm hash-value) 269 | cursor (.putCursor whm hash-value)] 270 | (.writeIfEmpty key-cursor (v->slot! key-cursor k)) 271 | (.write cursor (v->slot! cursor v)))) 272 | (.-cursor whm))) 273 | 274 | (defn ^WriteCursor set->WriteCursor! 275 | "Writes a Clojure set `s` to a XitDB WriteHashSet. 276 | Returns the cursor of the created WriteHashSet." 277 | [^WriteCursor cursor s] 278 | (let [whm (WriteCountedHashSet. cursor) 279 | db (-> cursor .db)] 280 | (doseq [v s] 281 | (let [hash-code (db-key-hash db v) 282 | cursor (.putCursor whm hash-code)] 283 | (.writeIfEmpty cursor (v->slot! cursor v)))) 284 | (.-cursor whm))) 285 | 286 | (defn read-bytes-with-format-tag 287 | "Reads a `BYTES` value (as a string) at `cursor` and converts it to a Clojure type. 288 | Checks the `formatTag` at cursor to determine the type encoded in the bytes object and 289 | converts the value to the respective Clojure type (eg. keyword, boolean, instant, date). 290 | Supported types are in the global constant `fmt-tag-value`. 291 | If there is no format tag (or it is unknown), returns the value as a string." 292 | [^ReadCursor cursor] 293 | (let [bytes-obj (.readBytesObject cursor nil) 294 | str (String. (.value bytes-obj)) 295 | fmt-tag (some-> bytes-obj .formatTag String.)] 296 | (cond 297 | 298 | (= fmt-tag (fmt-tag-value :keyword)) 299 | (keyword str) 300 | 301 | (= fmt-tag (fmt-tag-value :boolean)) 302 | (= str true-str) 303 | 304 | (= fmt-tag (fmt-tag-value :key-integer)) 305 | (Integer/parseInt str) 306 | 307 | (= fmt-tag (fmt-tag-value :inst)) 308 | (java.time.Instant/parse str) 309 | 310 | (= fmt-tag (fmt-tag-value :date)) 311 | (java.util.Date/from 312 | (java.time.Instant/parse str)) 313 | 314 | (= fmt-tag (fmt-tag-value :nil)) 315 | nil 316 | 317 | :else 318 | str))) 319 | 320 | (defn set-write-cursor 321 | [^WriteHashSet whs key] 322 | (let [hash-code (db-key-hash (-> whs .-cursor .-db) key)] 323 | (.putCursor whs hash-code))) 324 | 325 | (defn map-write-cursor 326 | "Gets a write cursor for the specified key in a WriteHashMap. 327 | Creates the key if it doesn't exist." 328 | [^WriteHashMap whm key] 329 | (let [key-hash (db-key-hash (-> whm .cursor .db) key)] 330 | (.putCursor whm key-hash))) 331 | 332 | (defn array-list-write-cursor 333 | "Returns a cursor to slot i in the array list. 334 | Throws if index is out of bounds." 335 | [^WriteArrayList wal i] 336 | (validation/validate-index-bounds i (.count wal) "Array list write cursor") 337 | (.putCursor wal i)) 338 | 339 | (defn linked-array-list-write-cursor 340 | [^WriteLinkedArrayList wlal i] 341 | (validation/validate-index-bounds i (.count wlal) "Linked array list write cursor") 342 | (.putCursor wlal i)) 343 | 344 | (defn write-cursor-for-key [cursor current-key] 345 | (let [value-tag (some-> cursor .slot .tag)] 346 | (cond 347 | (= value-tag Tag/HASH_MAP) 348 | (map-write-cursor (WriteHashMap. cursor) current-key) 349 | 350 | (= value-tag Tag/COUNTED_HASH_MAP) 351 | (map-write-cursor (WriteCountedHashMap. cursor) current-key) 352 | 353 | (= value-tag Tag/HASH_SET) 354 | (set-write-cursor (WriteHashSet. cursor) current-key) 355 | 356 | (= value-tag Tag/COUNTED_HASH_SET) 357 | (set-write-cursor (WriteCountedHashSet. cursor) current-key) 358 | 359 | (= value-tag Tag/ARRAY_LIST) 360 | (array-list-write-cursor (WriteArrayList. cursor) current-key) 361 | 362 | (= value-tag Tag/LINKED_ARRAY_LIST) 363 | (linked-array-list-write-cursor (WriteLinkedArrayList. cursor) current-key) 364 | 365 | :else 366 | (throw (IllegalArgumentException. 367 | (format "Cannot get cursor to key '%s' for value with tag '%s'" current-key (xit-tag->keyword value-tag))))))) 368 | 369 | (defn keypath-cursor 370 | "Recursively goes to keypath and returns the write cursor" 371 | [^WriteCursor cursor keypath] 372 | (if (empty? keypath) 373 | cursor 374 | (loop [cursor cursor 375 | [current-key & remaining-keys] keypath] 376 | (let [new-cursor (write-cursor-for-key cursor current-key)] 377 | (if (empty? remaining-keys) 378 | new-cursor 379 | (recur new-cursor remaining-keys)))))) -------------------------------------------------------------------------------- /src/xitdb/util/operations.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.util.operations 2 | (:require 3 | [xitdb.util.conversion :as conversion] 4 | [xitdb.util.validation :as validation]) 5 | (:import 6 | [io.github.radarroark.xitdb ReadArrayList ReadCountedHashMap ReadCountedHashSet ReadHashMap ReadHashSet ReadLinkedArrayList Tag WriteArrayList WriteCountedHashMap WriteCountedHashSet WriteCursor WriteHashMap WriteHashSet WriteLinkedArrayList])) 7 | 8 | ;; ============================================================================ 9 | ;; Array List Operations 10 | ;; ============================================================================ 11 | 12 | (defn ^WriteArrayList array-list-append-value! 13 | "Appends a value to a WriteArrayList. 14 | Converts the value to an appropriate XitDB representation using v->slot!." 15 | [^WriteArrayList wal v] 16 | (let [cursor (.appendCursor wal)] 17 | (.write cursor (conversion/v->slot! cursor v)) 18 | wal)) 19 | 20 | (defn ^WriteArrayList array-list-assoc-value! 21 | "Associates a value at index i in a WriteArrayList. 22 | Appends the value if the index equals the current count. 23 | Replaces the value at the specified index otherwise. 24 | Throws an IllegalArgumentException if the index is out of bounds." 25 | [^WriteArrayList wal i v] 26 | 27 | (assert (= Tag/ARRAY_LIST (-> wal .cursor .slot .tag))) 28 | (assert (number? i)) 29 | 30 | (validation/validate-index-bounds i (.count wal) "Array list assoc") 31 | 32 | (let [cursor (if (= i (.count wal)) 33 | (.appendCursor wal) 34 | (.putCursor wal i))] 35 | (.write cursor (conversion/v->slot! cursor v))) 36 | wal) 37 | 38 | (defn array-list-pop! 39 | "Removes the last element from a WriteArrayList. 40 | Throws IllegalStateException if the array is empty. 41 | Returns the array list with the last element removed." 42 | [^WriteArrayList wal] 43 | (validation/validate-non-empty (.count wal) "pop") 44 | (.slice wal (dec (.count wal)))) 45 | 46 | (defn array-list-empty! 47 | "Empties a WriteArrayList by replacing its contents with an empty array. 48 | Returns the modified WriteArrayList." 49 | [^WriteArrayList wal] 50 | (let [^WriteCursor cursor (-> wal .cursor)] 51 | (.write cursor (conversion/v->slot! cursor [])))) 52 | 53 | ;; ============================================================================ 54 | ;; Linked Array List Operations 55 | ;; ============================================================================ 56 | 57 | (defn linked-array-list-append-value! 58 | "Appends a value to a WriteLinkedArrayList. 59 | Converts the value to an appropriate XitDB representation using v->slot!." 60 | [^WriteLinkedArrayList wlal v] 61 | (let [cursor (.appendCursor wlal)] 62 | (.write cursor (conversion/v->slot! cursor v)) 63 | wlal)) 64 | 65 | (defn linked-array-list-insert-value! 66 | "Inserts a value at position pos in a WriteLinkedArrayList. 67 | Converts the value to an appropriate XitDB representation using v->slot!." 68 | [^WriteLinkedArrayList wlal pos v] 69 | (let [cursor (-> wlal .cursor)] 70 | (.insert wlal pos (conversion/v->slot! cursor v))) 71 | wlal) 72 | 73 | (defn linked-array-list-pop! 74 | "Removes the first element from a WriteLinkedArrayList. 75 | This is a stack-like operation (LIFO) for linked lists. 76 | Returns the modified WriteLinkedArrayList." 77 | [^WriteLinkedArrayList wlal] 78 | (.remove wlal 0) 79 | wlal) 80 | 81 | ;; ============================================================================ 82 | ;; Map Operations 83 | ;; ============================================================================ 84 | 85 | (defn map-assoc-value! 86 | "Associates a key-value pair in a WriteHashMap. 87 | 88 | Args: 89 | whm - The WriteHashMap to modify 90 | k - The key to associate (converted to XitDB representation) 91 | v - The value to associate (converted to XitDB representation) 92 | 93 | Returns the modified WriteHashMap. 94 | 95 | Throws IllegalArgumentException if attempting to associate an internal key. 96 | Updates the internal count if fast counting is enabled." 97 | [^WriteHashMap whm k v] 98 | (let [key-hash (conversion/db-key-hash (-> whm .cursor .db) k) 99 | key-cursor (.putKeyCursor whm key-hash) 100 | cursor (.putCursor whm key-hash)] 101 | (.writeIfEmpty key-cursor (conversion/v->slot! key-cursor k)) 102 | (.write cursor (conversion/v->slot! cursor v)) 103 | whm)) 104 | 105 | (defn map-dissoc-key! 106 | "Removes a key-value pair from a WriteHashMap. 107 | Throws IllegalArgumentException if attempting to remove an internal key. 108 | Updates the internal count if fast counting is enabled." 109 | [^WriteHashMap whm k] 110 | (let [key-hash (conversion/db-key-hash (-> whm .cursor .db) k)] 111 | (.remove whm key-hash)) 112 | whm) 113 | 114 | (defn ^WriteHashMap map-empty! 115 | "Empties a WriteHashMap by replacing its contents with an empty map. 116 | Returns the modified WriteHashMap." 117 | [^WriteHashMap whm] 118 | (let [^WriteCursor cursor (-> whm .cursor)] 119 | (.write cursor (conversion/v->slot! cursor {})) 120 | whm)) 121 | 122 | (defn map-contains-key? 123 | "Checks if a WriteHashMap contains the specified key. 124 | Returns true if the key exists, false otherwise." 125 | [^ReadHashMap whm key] 126 | (let [key-hash (conversion/db-key-hash (-> whm .cursor .db) key)] 127 | (not (nil? (.getKeyCursor whm key-hash))))) 128 | 129 | (defn map-item-count-iterated 130 | "Returns the number of keys in the map by iterating. 131 | The count includes internal keys if any." 132 | [^Iterable rhm] 133 | (let [it (.iterator rhm)] 134 | (loop [cnt 0] 135 | (if (.hasNext it) 136 | (do 137 | (.next it) 138 | (recur (inc cnt))) 139 | cnt)))) 140 | 141 | (defn map-item-count 142 | "Returns the number of key/vals in the map." 143 | [^ReadHashMap rhm] 144 | (if (instance? ReadCountedHashMap rhm) 145 | (.count ^ReadCountedHashMap rhm) 146 | (map-item-count-iterated rhm))) 147 | 148 | (defn map-read-cursor 149 | "Gets a read cursor for the specified key in a ReadHashMap. 150 | Returns the cursor if the key exists, nil otherwise." 151 | [^ReadHashMap rhm key] 152 | (let [key-hash (conversion/db-key-hash (-> rhm .cursor .db) key)] 153 | (.getCursor rhm key-hash))) 154 | 155 | ;; ============================================================================ 156 | ;; Set Operations 157 | ;; ============================================================================ 158 | 159 | (defn set-item-count 160 | "Returns the number of values in the set." 161 | [^ReadHashSet rhs] 162 | (if (instance? ReadCountedHashSet rhs) 163 | (.count ^ReadCountedHashSet rhs) 164 | (map-item-count-iterated rhs))) 165 | 166 | (defn set-assoc-value! 167 | "Adds a value to a set." 168 | [^WriteHashSet whs v] 169 | (let [hash-code (conversion/db-key-hash (-> whs .cursor .db) v) 170 | cursor (.putCursor whs hash-code)] 171 | (.writeIfEmpty cursor (conversion/v->slot! cursor v)) 172 | whs)) 173 | 174 | (defn set-disj-value! 175 | "Removes a value from a set" 176 | [^WriteHashSet whs v] 177 | (let [hash-code (conversion/db-key-hash (-> whs .cursor .db) v)] 178 | (.remove whs hash-code) 179 | whs)) 180 | 181 | (defn set-contains? 182 | "Returns true if `v` is in the set." 183 | [rhs v] 184 | (let [hash-code (conversion/db-key-hash (-> rhs .-cursor .-db) v) 185 | cursor (.getCursor rhs hash-code)] 186 | (some? cursor))) 187 | 188 | (defn ^WriteHashMap set-empty! 189 | "Replaces the whs value with an empty set." 190 | [^WriteHashSet whs] 191 | (let [empty-set (conversion/v->slot! (.cursor whs) #{})] 192 | (.write ^WriteCursor (.cursor whs) empty-set)) 193 | whs) 194 | 195 | ;; ============================================================================ 196 | ;; Sequence Operations 197 | ;; ============================================================================ 198 | 199 | (defn map-seq 200 | "Return a lazy seq of key-value MapEntry pairs." 201 | [^ReadHashMap rhm read-from-cursor] 202 | (let [it (.iterator rhm)] 203 | (letfn [(step [] 204 | (lazy-seq 205 | (when (.hasNext it) 206 | (let [cursor (.next it) 207 | kv (.readKeyValuePair cursor) 208 | k (read-from-cursor (.-keyCursor kv))] 209 | (let [v (read-from-cursor (.-valueCursor kv))] 210 | (cons (clojure.lang.MapEntry. k v) (step)))))))] 211 | (step)))) 212 | 213 | (defn set-seq 214 | "Return a lazy seq values from the set." 215 | [rhm read-from-cursor] 216 | (let [it (.iterator rhm)] 217 | (letfn [(step [] 218 | (lazy-seq 219 | (when (.hasNext it) 220 | (let [cursor (.next it) 221 | kv (.readKeyValuePair cursor) 222 | v (read-from-cursor (.-keyCursor kv))] 223 | (cons v (step))))))] 224 | (step)))) 225 | 226 | (defn array-seq 227 | "Creates a lazy sequence from a ReadArrayList. 228 | Uses the provided read-from-cursor function to convert cursors to values. 229 | Returns a lazy sequence of the array elements." 230 | [^ReadArrayList ral read-from-cursor] 231 | (let [iter (.iterator ral) 232 | lazy-iter (fn lazy-iter [] 233 | (when (.hasNext iter) 234 | (let [cursor (.next iter) 235 | value (read-from-cursor cursor)] 236 | (lazy-seq (cons value (lazy-iter))))))] 237 | (lazy-iter))) 238 | 239 | (defn linked-array-seq 240 | "Creates a lazy sequence from a ReadLinkedArrayList. 241 | Uses the provided read-from-cursor function to convert cursors to values. 242 | Returns a lazy sequence of the linked array elements." 243 | [^ReadLinkedArrayList rlal read-from-cursor] 244 | (let [iter (.iterator rlal) 245 | lazy-iter (fn lazy-iter [] 246 | (when (.hasNext iter) 247 | (let [cursor (.next iter) 248 | value (read-from-cursor cursor)] 249 | (lazy-seq (cons value (lazy-iter))))))] 250 | (lazy-iter))) 251 | 252 | (defn map-kv-reduce 253 | "Efficiently reduces over key-value pairs in a ReadHashMap, skipping hidden keys." 254 | [^ReadHashMap rhm read-from-cursor f init] 255 | (let [it (.iterator rhm)] 256 | (loop [result init] 257 | (if (.hasNext it) 258 | (let [cursor (.next it) 259 | kv (.readKeyValuePair cursor) 260 | k (read-from-cursor (.-keyCursor kv)) 261 | v (read-from-cursor (.-valueCursor kv)) 262 | new-result (f result k v)] 263 | (if (reduced? new-result) 264 | @new-result 265 | (recur new-result))) 266 | result)))) 267 | 268 | (defn array-kv-reduce 269 | "Efficiently reduces over index-value pairs in a ReadArrayList." 270 | [^ReadArrayList ral read-from-cursor f init] 271 | (let [count (.count ral)] 272 | (loop [i 0 273 | result init] 274 | (if (< i count) 275 | (let [cursor (.getCursor ral i) 276 | v (read-from-cursor cursor) 277 | new-result (f result i v)] 278 | (if (reduced? new-result) 279 | @new-result 280 | (recur (inc i) new-result))) 281 | result)))) 282 | -------------------------------------------------------------------------------- /src/xitdb/util/validation.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.util.validation) 2 | 3 | (defn lazy-seq? [v] 4 | (instance? clojure.lang.LazySeq v)) 5 | 6 | (defn vector-or-chunked? [v] 7 | (or (vector? v) (chunked-seq? v))) 8 | 9 | (defn list-or-cons? [v] 10 | (or (list? v) (instance? clojure.lang.Cons v))) 11 | 12 | (defn validate-index-bounds 13 | "Validates that index i is within bounds for a collection of given count. 14 | Throws IllegalArgumentException if out of bounds." 15 | [i count operation-name] 16 | (when (or (< i 0) (> i count)) 17 | (throw (IllegalArgumentException. 18 | (str operation-name " index " i " out of bounds for collection of size " count))))) 19 | 20 | (defn validate-non-empty 21 | "Validates that a collection is not empty. 22 | Throws IllegalStateException if empty." 23 | [count operation-name] 24 | (when (zero? count) 25 | (throw (IllegalStateException. 26 | (str "Cannot " operation-name " on empty collection"))))) 27 | 28 | (defn validate-not-lazy-seq 29 | "Validates that a value is not a lazy sequence. 30 | Throws IllegalArgumentException if it is a lazy sequence." 31 | [v] 32 | (when (lazy-seq? v) 33 | (throw (IllegalArgumentException. "Lazy sequences can be infinite and not allowed!")))) 34 | 35 | (defn validate-supported-type 36 | "Validates that a type is supported for conversion. 37 | Throws IllegalArgumentException for unsupported types." 38 | [v] 39 | (let [supported-types #{java.lang.String 40 | clojure.lang.Keyword 41 | java.lang.Long 42 | java.lang.Integer 43 | java.lang.Boolean 44 | java.lang.Double 45 | java.lang.Float 46 | java.time.Instant 47 | java.util.Date}] 48 | (when-not (or (nil? v) 49 | (some #(instance? % v) supported-types) 50 | (map? v) 51 | (vector? v) 52 | (list? v) 53 | (set? v)) 54 | (throw (IllegalArgumentException. (str "Unsupported type: " (type v) " for value: " v)))))) -------------------------------------------------------------------------------- /src/xitdb/xitdb_types.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.xitdb-types 2 | (:require 3 | [xitdb.array-list :as xarray-list] 4 | [xitdb.common :as common] 5 | [xitdb.hash-map :as xhash-map] 6 | [xitdb.hash-set :as xhash-set] 7 | [xitdb.linked-list :as xlinked-list] 8 | [xitdb.util.conversion :as conversion]) 9 | (:import 10 | [io.github.radarroark.xitdb ReadCursor Slot Tag WriteCursor])) 11 | 12 | (defn read-from-cursor 13 | "Reads the value at cursor and converts it to a Clojure type. 14 | Values tagged Hash map, Array List, Linked Array List and Set will be converted to 15 | the respective XITDB* type. 16 | If `for-writing?` is true, the function will return XITDBWrite* types, so `cursor` must be a WriteCursor. 17 | `BYTES` values will be pre-processed (see `conversion/read-bytes-with-format-tag`)." 18 | [^ReadCursor cursor for-writing?] 19 | (let [value-tag (some-> cursor .slot .tag)] 20 | #_(println "read-from-cursor" value-tag "for-writing?" for-writing?) 21 | (cond 22 | (contains? #{Tag/SHORT_BYTES Tag/BYTES} value-tag) 23 | (conversion/read-bytes-with-format-tag cursor) 24 | 25 | (= value-tag Tag/UINT) 26 | (.readUint cursor) 27 | 28 | (= value-tag Tag/INT) 29 | (.readInt cursor) 30 | 31 | (= value-tag Tag/FLOAT) 32 | (.readFloat cursor) 33 | 34 | (= value-tag Tag/HASH_MAP) 35 | (if for-writing? 36 | (xhash-map/xwrite-hash-map cursor) 37 | (xhash-map/xhash-map cursor)) 38 | 39 | (= value-tag Tag/COUNTED_HASH_MAP) 40 | (if for-writing? 41 | (xhash-map/xwrite-hash-map-counted cursor) 42 | (xhash-map/xhash-map-counted cursor)) 43 | 44 | (= value-tag Tag/HASH_SET) 45 | (if for-writing? 46 | (xhash-set/xwrite-hash-set cursor) 47 | (xhash-set/xhash-set cursor)) 48 | 49 | (= value-tag Tag/COUNTED_HASH_SET) 50 | (if for-writing? 51 | (xhash-set/xwrite-hash-set-counted cursor) 52 | (xhash-set/xhash-set-counted cursor)) 53 | 54 | (= value-tag Tag/ARRAY_LIST) 55 | (if for-writing? 56 | (xarray-list/xwrite-array-list cursor) 57 | (xarray-list/xarray-list cursor)) 58 | 59 | (= value-tag Tag/LINKED_ARRAY_LIST) 60 | (if for-writing? 61 | (xlinked-list/xwrite-linked-list cursor) 62 | (xlinked-list/xlinked-list cursor)) 63 | 64 | :else 65 | nil))) 66 | 67 | (extend-protocol common/IReadFromCursor 68 | ReadCursor 69 | (-read-from-cursor [this] 70 | (read-from-cursor this false)) 71 | 72 | WriteCursor 73 | (-read-from-cursor [this] 74 | (read-from-cursor this true))) 75 | 76 | (defn ^Slot slot-for-value! [^WriteCursor cursor v] 77 | (cond 78 | (satisfies? common/ISlot v) 79 | (common/-slot v) 80 | :else 81 | (conversion/v->slot! cursor v))) 82 | 83 | (defn materialize 84 | "Converts a xitdb data structure `v` to a clojure data structure. 85 | This has the effect of reading the whole data structure into memory." 86 | [v] 87 | (common/materialize v)) -------------------------------------------------------------------------------- /test-resources/map-1.edn: -------------------------------------------------------------------------------- 1 | {:key-660 {true {true {502 917, "string-72" 0.7057926526880189, nil "value-447", "string-56" 0.7095322760658269, :key-571 "value-122"}, 572 [nil "value-243" "value-780" 546 true], :key-695 721, 397 {888 0.939472161691777, nil nil, :key-435 0.5631314701491865, true 0.07007301751428419}, "string-533" {:key-494 0.3866498079986229, :key-914 0.15760011843793587, "string-1" false, nil 419, 199 430}}, false ["value-221"], :key-540 {false {:key-147 "value-619", "string-18" 589, false "value-446", nil nil, 166 false}, 128 [false true nil "value-927" "value-189"], :key-591 [0.5231761627710027], "string-346" {280 0.5526880064878685, "string-266" true, 511 198, 285 "value-723", nil nil}, :key-30 [nil 40 0.06733040721433614 "value-40"]}, "string-114" [0.5646821995695601 {"string-168" true, nil false, true false, 898 nil}], "string-35" nil}, false [{232 {:key-251 "value-796", :key-554 true, :key-830 "value-920", false nil, nil 0.4445492689071502}, 320 [nil "value-168" false 19 56], 585 [false nil "value-6" 0.30112034587875836], nil 0.4107281355818283, :key-487 nil}], :key-322 [{221 {:key-832 nil, :key-325 606, nil nil, :key-278 838, :key-782 nil}, nil {true "value-46", "string-997" nil, :key-348 "value-979", 254 nil, "string-436" false}, 496 [nil 0.9681387297368761 nil 33 true], true [true nil "value-446" "value-117"]} [["value-251" nil 0.6504368575121552] 5 522] 0.2626262683057642 0.964859650460192 [false 632]], "string-714" true} -------------------------------------------------------------------------------- /test-resources/map-2.edn: -------------------------------------------------------------------------------- 1 | {667 {false {:key-224 {nil {:key-462 {false {:key-99 "value-718", :key-394 0.564735231475375, 341 67, 679 "value-666", "string-43" nil}, nil [nil 510 "value-48" false], "string-892" {765 true, :key-408 "value-638", nil 0.05871806917898026, :key-685 "value-263", :key-927 0.3890570453520875}, "string-323" [nil], true [nil nil "value-631"]}, :key-782 [[true]], "string-545" {702 {:key-308 false, "string-53" "value-617", :key-744 503, "string-286" 0.518853552571306, true false}, nil [0.1246854207077478], 944 [false "value-344" "value-257" 29], "string-900" [771 nil 0.9220717936368023], 251 false}, "string-497" {320 {:key-163 656, 259 nil, "string-575" 489, "string-421" true, "string-39" nil}, 458 [false "value-753" 0.019742654016675742 nil nil], 925 {"string-223" 0.6927895970245272, 90 0.05718198878853642, 740 nil, 267 "value-634", 793 164}, :key-435 nil, false {"string-544" 0.01914377999759742, :key-136 0.5187654793891213, true 974, 45 704, "string-446" nil}}, true [["value-863" 162 "value-700" 914]]}, :key-690 [{213 {"string-338" "value-202", true true, false false, "string-496" false}, 94 [0.3145261408331218], "string-951" nil, 536 {true 0.2606585598321187, "string-774" 0.2243632247791265, nil "value-55", "string-132" nil}, 742 "value-282"}], true 0.4503371659499593, 689 nil, 385 nil}, "string-704" [[637] [{nil ["value-889" true], :key-750 [0.4379345158757044 "value-692"], 903 [false false 404], :key-315 148} "value-552" {:key-602 {"string-391" nil, nil true, 457 982, 867 "value-921"}, true [nil "value-291" "value-877" 0.7240727225726507 0.055949491293951326], :key-681 0.3841572981185959, :key-776 "value-233", "string-557" [0.23697090617895078 0.6450868652762549 0.5587422516488119 "value-547"]} {"string-795" {false true, :key-927 nil, "string-835" 764, nil nil, :key-404 true}, 385 [0.22006877828623772 "value-158" nil], "string-591" false, 322 {nil "value-754", true 0.019620946792087923, 651 nil, :key-954 0.6959520589155018, "string-79" false}, :key-824 {"string-400" "value-537", "string-537" 0.08372084784527734, "string-343" 162, 346 true, true false}}] 860], :key-201 {493 {"string-954" {"string-969" {nil 324, :key-701 nil, "string-685" 564, "string-3" nil}, "string-812" [0.4491235027898698 239 "value-670"], 677 {307 0.007617150739717138, "string-165" nil, true "value-454", false "value-9", 949 true}, nil false, 443 [0.1405117711471714 true]}, 724 [{true "value-56", nil 0.38681554110618155, :key-705 538, "string-165" 0.6335202310023818}], true "value-922", :key-246 793, 908 0.6822384824437364}, "string-749" ["value-226" 623 [[nil nil "value-641" 0.6974805406764036] [nil 0.16929273084781993 false nil 0.9796250317434333] 165]], nil 398, false nil}, :key-210 {"string-217" {nil 208, "string-61" [{"string-148" "value-349", nil true, "string-580" nil} {"string-351" false, false "value-157", :key-458 true, nil "value-453", :key-778 true} 7 889], 867 "value-831", 387 ["value-402" {560 true, "string-697" nil, 914 nil, :key-865 765, "string-268" "value-390"}]}, :key-433 [[[0.6412722408499496 0.8634304402188748 0.8660076396621781 0.09293186281935817] {:key-352 true, true 0.4932427382287212, 755 938, 301 0.41701160569528284, "string-418" 945} "value-538" [nil 590 0.4831459837071834 "value-220" 674]]], "string-893" [[[0.5288108690639687 "value-633" nil 0.9379058506117366] [nil 0.8410782619235389 527] ["value-633" 0.9152731959220466] true] [false] {:key-969 {nil 130, :key-723 520, true 699, :key-703 nil}, false [0.7404290849169385 0.25333954715651297], "string-621" false, true "value-753"}], 388 [{:key-412 {nil 968, true 614, :key-926 0.5792674445075398, :key-771 386}, false ["value-784" 724 0.04379653060463773 true], :key-130 [false 0.9090120965131231], 828 [true], :key-470 nil} {nil [false "value-453" 0.7187447892481023], :key-145 0.3073952412675087, true nil} 50], "string-701" 0.3143454933938551}, :key-495 "value-524"}, :key-658 [[[703 {136 {596 nil, nil false, :key-93 "value-469", 80 920, 217 177}, nil [nil], "string-663" [0.6476186464277819 nil 378], "string-818" false, "string-162" false} [0.838827058066444] false [[529] {:key-215 0.3807691755859268, :key-700 "value-818", 456 nil, 524 nil, 946 971} {true true, nil false, 266 0.0290065576630818} [782 "value-127" true 0.5546534999213076 0.8974967125132675]]] [[[0.8949830202350245 "value-554"] {nil 351, "string-343" nil, true false} {"string-18" 444, false true, "string-612" 0.18853370995330132, "string-130" nil, "string-588" 928}] [["value-998" nil true 0.6410994120060701] {false nil, "string-606" 205, "string-302" false} 214] "value-734"]] [[[true 94 [741 false 0.07397320240420002] {724 true, 103 "value-849", "string-611" 861, :key-838 false, 828 0.6463672083219033} nil] [[663 7.671965524236324E-5 nil 0.10552839026087457 0.7022033845544666] {645 true, nil nil, true 0.9482702505428078, :key-874 "value-235", :key-954 nil} [0.4710366948438849] {false nil, nil 0.40338346257322477, 896 "value-411", 581 false}]] "value-729" [{:key-38 {false 0.3552116373602703, nil nil, "string-923" 974, "string-387" "value-484", true false}, false ["value-519" 0.4159559260460747 false false "value-7"], true nil, :key-215 [nil 52 446 false 0.7058276046525154], "string-467" "value-785"} {:key-79 {true false, 631 0.6508517147508089, "string-617" false, :key-316 true, nil "value-573"}, :key-143 [true], false nil, "string-891" [90 0.4377564265664231 true "value-298"], "string-797" [0.9003885653716229 639 "value-818" "value-714"]} false nil]] 43], nil false}, false [["value-133"]], :key-551 {:key-123 {"string-141" {"string-156" {true {:key-415 {197 nil, :key-943 nil, false 78, nil 60}, "string-344" [nil 122 0.8049654212390277 0.15584055685655307 "value-444"], 160 ["value-379" true "value-455"], nil [false 0.24532218836148267], :key-341 [nil "value-326"]}, nil "value-351", false [156 [582 "value-117" 214] {nil "value-850", 868 "value-578", true false, false nil}], "string-377" {nil {:key-15 false, :key-295 "value-535", "string-663" 0.9469919122096419, false 850, nil 0.7554187400977536}, "string-706" [0.17441322342886822 false 0.8565870276177447 "value-849" 868], :key-865 true, "string-270" 0.432185196037314, "string-605" nil}}, "string-542" [[nil true nil nil] [[725] "value-998" {172 623, 34 false, 445 "value-175", :key-154 681, "string-220" false}] {586 {275 "value-773", "string-294" true, nil 0.899653112545776, "string-202" nil}, nil [752 nil 843 false true], true ["value-477" 0.46941514728967937 912], 374 true, "string-789" 881}], 419 [92], "string-762" [[{403 588, 123 true, 583 "value-201", nil "value-451", "string-828" nil} [false nil] {nil nil, "string-727" 870, :key-328 207, 22 false} {nil nil, false false, 464 "value-557", true nil, 89 "value-931"} [nil]] [[nil] nil [947 true 0.9481942023931113] {nil "value-40", 335 793}] [[true "value-534"]]], false nil}, false [["value-902" {"string-72" {nil false, :key-795 nil, false 0.541880806633766, "string-406" 464}, "string-43" [211 nil "value-609" 0.1915826911767471], false {true nil, nil 563, :key-104 false, :key-928 false}, "string-537" [true], 629 {"string-539" 0.5276602082138891, nil "value-730", :key-672 "value-52", "string-741" "value-252"}} [false] [[true false 0.6208904661325021 "value-962" 417]]] {"string-544" {true {true false, nil 526, "string-98" 354, "string-762" 0.9316698237375777}, false [753], 737 [true nil true "value-188"], "string-233" [false 0.3520101335404434 0.019459351708782324]}, true 0.8967876268079473, :key-867 [nil [0.32832470633800115 false true nil true] 140 [nil nil "value-179"]]}], true [{:key-440 {:key-432 {false nil, :key-540 false, 598 "value-571"}, :key-473 [0.5043175142756783], nil [45 558 0.31077415592210245], 878 "value-487"}, :key-200 [{:key-943 0.21369696947990535, "string-33" "value-102", nil "value-312", "string-398" 0.6886936519024495, false "value-295"} {"string-818" "value-846", nil "value-510", true 620} false 570], 607 [true {"string-97" 0.2666331343242322, false 314, nil nil, 719 0.6509564862501654} "value-255" 856], "string-631" {"string-431" {nil 300, false "value-67", "string-725" 683, true false}, nil false, "string-695" {false 0.5617942207830752, :key-547 "value-650", true false}, 837 [nil 586]}, 294 670} {"string-73" {true {nil false, 386 260, 121 nil, "string-310" nil, false 656}, nil [true 0.5806622906327807 0.9033894313861575 "value-244"], "string-51" {540 "value-525", false 693, "string-622" "value-916", 209 0.17166493799389038, :key-508 "value-684"}, 4 [nil 884 571 true]}, "string-182" [false "value-391"], "string-603" false, 274 {564 {nil nil, :key-626 false, 701 "value-796", 977 0.9114451086774163}, 243 [nil nil], 680 418, :key-1 [true true "value-121"], "string-154" {true nil, :key-281 true, :key-349 241, nil "value-216"}}, true {"string-803" {false "value-626", "string-889" "value-443", nil nil, "string-371" 0.5640968606522117}, "string-152" [0.7433503799550683 nil 0.0622434274084811 743 nil], :key-90 [true], "string-820" {nil 0.15784126463473325, :key-736 false, false 891, true nil}, 657 "value-776"}} nil], nil [false "value-85"], 548 "value-504"}, true [{48 {false {nil true, false [598 604 242 nil], "string-896" {14 0.3372053524523829, 604 0.4476697390051716, nil 0.11118834110497855, 376 0.04889781198051657, "string-310" nil}, true 307}, nil [377], :key-125 [nil {:key-74 true, "string-469" false, :key-205 785, 887 306, nil 0.11524951400784111}], "string-67" false}, 337 [[{true 885, "string-27" 0.28878451270704697, :key-443 423, "string-218" 5}] {783 {"string-210" 868, nil true, 121 375, "string-156" nil, true "value-915"}, false [0.1758019889960014 true "value-137" 913 false], 189 {nil true, true "value-361", :key-354 true}, :key-923 {"string-60" 453, "string-186" nil, 652 false, :key-583 141, 584 nil}, :key-516 {false true, 188 0.7691677510818615, "string-248" false, "string-683" "value-180"}} ["value-998" ["value-939" 523 977] ["value-942" "value-664"]]], :key-233 {"string-789" {nil {true true, false nil, 409 410, "string-823" "value-896", :key-390 227}, "string-998" [0.6398085197037408 0.638612020619713], "string-248" ["value-189" 0.33139276162584996]}, :key-986 [[false false nil nil] 0.47545127241297225], false [[nil] [nil 0.6320747434467932]], nil [{"string-133" "value-490", 51 0.607413841059913, false true, "string-464" nil, 723 "value-668"} {"string-593" true, "string-261" 0.5414976737310419, 128 157, nil nil, 335 0.7016577631346734} 0.08603615051217406 {true "value-581", :key-144 0.17910282051151738, :key-106 true, 655 nil, nil "value-173"} nil]}, :key-121 {:key-922 {true {nil 73, false 81, 314 0.3759978732571053}, nil [true true "value-93" 0.2213058857007223], 878 {nil "value-568", false nil, true true, :key-445 328}, "string-462" {false nil, 270 nil, 341 431, 647 nil}, false 0.5397105191583458}, true 0.9377859204542286, :key-72 true, "string-378" [{true true, :key-452 666, nil nil} nil true]}, "string-812" 349} nil {false {"string-107" {:key-951 {:key-512 49, false 102, "string-800" true, nil 865, :key-680 0.9233600012331409}, :key-910 [0.3463648497412246 "value-296" nil], "string-354" ["value-600" 825 0.8263309530678762 117 0.13385292303586172], true nil, 747 [true 527 true]}, "string-389" ["value-278" [897 "value-245"] {:key-624 "value-634", "string-619" 0.3683625945235097, true nil, false nil, nil nil}], 54 {861 {"string-934" nil, nil false, false nil}, "string-53" [0.13112806424231171], 553 "value-601", 309 "value-118", :key-910 false}, :key-627 178, 736 "value-460"}, 74 [{nil {false 635, nil "value-501", "string-468" "value-585", "string-659" true, "string-245" 0.7807439209285632}, true ["value-344" nil], :key-727 true} {:key-645 {448 nil, 583 808, false false, :key-65 0.64714535788093}, "string-256" [581 0.5743124883273985 nil "value-657"], 536 [false nil 0.3540345647314398 true 0.6975993344616749], :key-516 0.06883243863033306, nil {"string-433" "value-558", :key-989 "value-625", "string-116" "value-270", nil true}} true 43 660], 962 [[[0.8096916561660203 nil] 0.37079458317535174 [true nil false 813]]], 995 "value-983", true {nil nil, :key-448 [{false 0.036184802719489095, :key-38 0.17752326199296753, 330 0.47680754478105825, :key-660 true, 854 nil} "value-440" {799 0.9192167147080479, true false, 19 0.9412069956031606, :key-89 997, nil 0.3262655850487822} {861 false, 630 true, :key-750 false, :key-764 nil, false true} nil], true [[461] [false nil nil]]}} {"string-142" {:key-355 {531 {896 458, "string-404" 628, nil 673, :key-642 nil}, false [nil nil "value-912"], 418 [nil nil nil nil], :key-549 {"string-135" 988, "string-381" 0.9005321409334027, :key-153 false, 252 "value-490", "string-983" "value-197"}, 512 false}, :key-408 [nil nil {nil nil, false 0.12090808631453653, true 574, 228 true}], "string-965" {false {false false, "string-631" 289, nil nil, true nil}, 912 ["value-603" "value-18"], :key-267 [nil 393 89 nil true], 125 {981 true, "string-514" 75, nil "value-853", 548 372}, nil {false "value-58", :key-884 "value-120", "string-823" 0.44827646092315077, "string-46" 799, "string-571" nil}}, nil {true {:key-185 "value-893", nil true, 170 "value-263", :key-264 0.15786929654507653}, "string-579" [true nil], nil {"string-768" 321, :key-383 false, :key-452 0.08370064085899065, 434 "value-521", "string-952" "value-256"}}, :key-524 {437 {:key-203 475, :key-115 183, :key-54 307, true "value-404", 734 "value-844"}, "string-618" [nil true], 943 ["value-154" 0.7478451346298065 nil nil], true {true true, 788 false, "string-283" "value-287", 753 true}, nil [nil false nil]}}, nil {"string-415" {true "value-30", false [nil], 392 nil, 138 {"string-358" false, 679 "value-841", "string-171" nil, 155 866, false nil}}, :key-865 [{:key-872 "value-808", nil nil, "string-177" 0.2936531619331487, true true} [false nil 206]], true 0.7895151445790876, :key-363 723, nil {true {676 365, :key-24 "value-753", true false, nil 963, :key-574 627}, :key-826 [0.3993166388979038], false [372], "string-118" [false 0.3151475469223727 nil "value-681" 996], :key-944 [305 675]}}, 228 "value-249"} [[[{false "value-636", :key-446 "value-602", :key-710 32, 837 "value-604", :key-289 "value-565"}] "value-322" [{:key-41 nil, "string-616" false, true nil, "string-725" true, nil false} [0.3243773119151707 0.6563070744180147 "value-394" nil "value-258"] {"string-57" 793, :key-12 nil, :key-479 301, true nil, false true} nil {"string-192" 0.05326138163404803, "string-48" 0.27411074779934796, :key-546 true, nil nil}]] {true {nil {:key-833 false, "string-80" 0.9847589034073914, "string-298" nil, "string-620" nil, :key-885 0.8211166648949726}, :key-933 [nil "value-440" 0.7817410178152272 684], "string-810" true, "string-972" nil, "string-918" ["value-503" nil]}, :key-872 [{"string-388" true, "string-430" "value-40", :key-633 413, 60 nil, false true}], nil 0.8251046796365974, "string-727" false, 286 [false [0.19976873037518128 nil "value-177"] {:key-184 687, 370 0.24541308671412865, "string-703" 159, 601 72, nil false} [nil "value-178"]]} false]], "string-157" {nil ["value-292" 0.6262616621984142 396 67 {nil 0.746358642422997, :key-888 0.03750354819035384, :key-384 0.9931359282486701}], :key-218 ["value-740" {false {:key-984 {753 nil, false 432, nil 0.9042794302765009, "string-50" 146}, false [0.3034216199325197 false nil nil true], 323 [true 860 290 nil "value-766"], nil true, "string-553" 0.11252468022115847}, nil 238, "string-280" [["value-803" 0.7353725197480812]], "string-378" {true {false 425, 442 nil, "string-750" nil, "string-144" nil}, :key-241 [true], 322 {nil 750, "string-78" nil, 643 206}, 173 {nil nil, "string-209" "value-341", true false, :key-678 0.6712389689348812, "string-717" true}, nil ["value-828" "value-992" "value-485" false]}} [{686 {"string-705" "value-245", :key-869 true, false 0.29084306628816703, 425 0.9476711946919655, 818 true}, :key-790 [0.6920419805173879 296 "value-132" nil], :key-628 true, true nil, 198 "value-157"}] [0.8556427426326756 [[0.6215599028051805] [0.32880776725067506] [0.8229974397076927]]] nil]}, nil {nil {nil false, "string-807" [{"string-661" {372 "value-308", true 870, nil 472, false true}, 129 [448 "value-717" nil nil 0.7366133003557894], "string-689" 392, false {true "value-400", :key-166 654, nil nil, :key-133 nil, :key-558 "value-460"}, 976 {true "value-343", false nil, "string-459" false, "string-389" true, nil false}} [[30] [617 nil 0.5602220706172295] [315] "value-542" {"string-397" 424, nil 605, 934 "value-827", :key-924 0.07659606776416916}] "value-699" {:key-898 {:key-943 nil, nil true, :key-653 false, true 101}, :key-976 ["value-955" 737 true "value-284"], :key-631 699, nil [nil], 101 {:key-292 0.9127251089802901, nil true, false true, "string-626" false}} {nil {:key-735 "value-174", nil 347, "string-789" 0.3736566877789831, false 0.8044232458811463}, 346 [0.2959313504506478 true], "string-885" {:key-939 nil, "string-926" 824, "string-281" "value-638", false 827, nil "value-409"}, 590 [533 326 0.7985879323658602 "value-544"], false 0.8502503258558376}], "string-924" [{:key-245 {"string-723" "value-440", nil 0.5296736026755916, false nil, 543 false, 469 true}, 27 [nil], 220 {false 0.7639056317141059, nil 0.965253084873049, 679 "value-986"}, 82 "value-156", "string-538" {:key-911 false, :key-110 184, 882 false, 809 nil, false 0.07789119285685286}}], "string-259" 529}, false [{:key-579 {nil {184 nil, :key-932 nil, true 0.5875480316283855, false true}, false [0.3819575586603189 737], "string-186" [nil 0.24442960681839498 0.7289111143720822], "string-570" ["value-923" nil]}, :key-919 [[true 402 nil "value-899" "value-587"] {true 464, false nil, 320 "value-451", nil 0.3990684627472717} [nil true 0.8285106977460447] {nil false, :key-8 true, false "value-865", 702 0.39680655284628097} "value-453"], :key-923 false, "string-73" {"string-93" {:key-774 nil, "string-163" nil, 791 489, :key-796 true, 653 270}, false [279 "value-425" nil "value-139" true], "string-338" 298, nil {true false, :key-272 true, 209 true, 654 0.7889587739284534}, 237 {:key-796 61, nil nil, "string-832" "value-966", 22 0.6223855952814058}}, false [{:key-285 true, 787 487, :key-52 true, nil "value-975", "string-807" 220} {false "value-587", 275 42, true 0.854892072948271, "string-186" 0.9089391002311958}]} {"string-831" {false {:key-208 nil, "string-715" 0.40535960062577836, :key-799 "value-847", "string-878" nil, nil 212}, :key-984 [187], 405 true, true {false nil, :key-944 "value-973", nil 0.6095148025938789, :key-316 0.45467049529516024, 743 323}}, nil [{:key-624 0.8352336835496053, nil nil, "string-680" 682, "string-580" 816} {nil "value-835", 529 nil, "string-592" nil, "string-577" "value-201", 67 0.3314104285856674} {true false, 314 "value-393", nil 0.6533881693400809} {425 575, false 0.7094010852264275, "string-905" 307, 687 true} "value-843"], 343 {nil {false nil, 319 0.016740518663658666, "string-578" 318, 550 false, :key-513 0.5345404289105817}, true ["value-276" nil], 452 {nil "value-267", false "value-900", true 0.4837865301703602, "string-897" 543}, false [nil], 804 "value-396"}, "string-712" [[nil] {503 "value-551", "string-181" "value-148", 545 74, nil false, true false} {true 184, "string-952" true, false true, :key-886 0.302152238144451} {nil nil, :key-296 0.12953378734949994, "string-96" 389} ["value-495" 0.13924385501905479 nil nil nil]], "string-61" [true 64 [true] nil]} true 0.49033189708282654 [{nil {false true, :key-566 true, nil true, true true, 8 "value-238"}, "string-418" [false], :key-728 nil, 191 ["value-303"], false {851 nil, "string-685" nil, false 588, true nil, :key-294 nil}}]], "string-971" {:key-777 {743 {nil {true "value-982", false 0.7430640317346849, 804 "value-176"}, false "value-699", true [0.32841540648967493 "value-533" "value-119"]}, nil [{nil 194, :key-699 nil, "string-714" 0.6498983550194622, "string-65" nil} {:key-931 "value-1", "string-272" "value-790", "string-931" nil, 876 "value-435", :key-596 true} {"string-854" nil, nil nil, 331 0.07113812408667375, :key-37 0.5938509651433306} [106]], :key-802 {:key-302 {true "value-763", 491 "value-210", :key-76 0.4902554399336234, nil 467}, "string-947" [0.1356258763306024 nil nil 0.2591504277185045], 163 [210], :key-130 [nil "value-206" 877], 369 ["value-771" 0.6465941756214546 nil 0.7372249632313985 true]}, 995 {nil {"string-707" 0.1085559217213814, "string-432" nil, "string-889" 0.26883701714053687, 197 true, :key-926 503}, "string-66" [854 nil 0.05818619737699249 false], 22 0.9983155353598979, 332 119}}, :key-997 [{:key-139 {nil 863, "string-382" "value-416", 497 true}, nil [0.9570078128829738 0.4069338297276507 nil "value-876"], "string-693" "value-766", "string-55" [472 nil]} {:key-205 {nil "value-787", "string-609" nil, :key-875 101, 943 0.7536422297495697}, :key-621 ["value-702" true 727 0.4350773560443395 "value-940"], "string-319" 0.133935668169895, nil "value-430"} [[0.4778521292402864] {225 false, :key-899 false, 917 173, 606 "value-233", nil "value-209"} false {:key-348 732, true 634, 560 0.5102312578943122, nil nil, 708 nil}] {277 {393 nil, "string-333" "value-615", "string-661" false, nil "value-253"}, false [480 nil 883 788 615], nil {"string-742" false, 639 438, false 0.6450889758460099, nil true, "string-541" 0.9487133260533003}, "string-236" [nil], 916 [nil nil 63 849 0.8077643989875237]}], :key-69 {"string-11" {"string-955" {:key-511 0.4229351329384422, "string-64" 646, nil true, "string-21" true, "string-721" true}, nil [true "value-656"], true {true 0.013004276371833101, "string-427" 608, 257 237, :key-86 true, :key-750 nil}, "string-374" {694 0.45520499805931625, "string-937" "value-255", :key-123 true, nil 221, 626 0.5175756085661187}, :key-2 [false]}, "string-943" ["value-409"], false {"string-326" {:key-411 nil, :key-531 0.9555432319923948, "string-553" "value-546", :key-495 0.4014663045951943, false 95}, "string-963" [901 107], false 642, "string-103" {"string-76" true, nil 820, :key-811 "value-576", 482 0.610791021000254, :key-647 414}, 582 "value-991"}, :key-118 nil}, :key-880 "value-245", nil [{363 {nil "value-492", "string-204" true, 584 false, 696 "value-999", :key-619 0.19620429316021015}, false [true nil "value-376" nil], "string-910" {304 0.8503747614687732, nil 144, 248 0.8143889471386863, :key-366 "value-573"}, :key-232 [false "value-845" nil nil 781], "string-211" ["value-925" "value-235" 191 0.03457232629225715]} false]}, "string-672" [{nil {nil {255 nil, :key-549 0.8479773812421425, nil nil, true false}, "string-760" [0.15680143121951562], :key-959 {:key-383 nil, "string-6" true, nil nil, :key-241 false, "string-788" 495}, 407 ["value-14" nil 0.6425825899481409 nil "value-748"], 762 {:key-824 "value-183", false false, "string-665" "value-970", "string-588" 0.471959033973427, :key-122 51}}, "string-484" [nil], :key-984 "value-476", false [177 true], 563 {241 {nil 0.32869030898221485, 304 0.3358472968921289, true 818, :key-6 "value-926"}, true [true], :key-380 {nil false, false 135, 731 true, "string-461" false, :key-253 "value-357"}, 11 ["value-412" "value-886" true "value-275" 0.07558855357723337], "string-822" {"string-119" "value-489", "string-605" 0.5652191621758687, :key-280 false, nil 748}}} [{nil {"string-112" false, nil 319, "string-981" nil, :key-275 false}, 69 [0.4165245642641291 593 0.13122214776537822], "string-265" {:key-542 nil, 789 "value-302", nil "value-51", false 0.8909568938051173, :key-238 "value-367"}, "string-535" [238 547 false "value-517"], "string-566" 0.13540830605455034}] ["value-850" 0.4624359519889476]]}, false [[[980 [["value-651" "value-471" true "value-294" 450] {true false, 164 true, 928 nil, 481 0.8959689315278562} 0.1578594179340732] 0.901517183547884 443 [{"string-401" true, false false, nil 0.555077319427099, "string-652" 0.2523790240069471, 712 0.7945869045494819} {nil 960, "string-106" true, "string-843" false, 119 406, "string-412" nil} [281 "value-569" 938]]] {:key-221 {"string-335" {false 572, :key-810 nil, :key-872 0.2787239401406758, 110 0.8960555636452323, "string-883" 0.9231471797017025}, :key-584 [730 0.4764217431526384 536], nil {"string-693" nil, 292 0.2368854841411212, 759 0.6481986973751718, :key-574 nil, "string-734" "value-458"}, :key-816 0.7078032034543061, false [nil 797 0.9113330804689636 0.27462817444586207 nil]}, :key-771 [{nil 872, true "value-700", 761 0.991946722862311, false 0.3661318949635882} {757 true, 599 634, 517 924, true "value-581", :key-628 433}], 974 false, false ["value-809" [true 383] [0.18930131722186205 false 612] true "value-899"], true [{31 248, false false, :key-118 "value-271"} [nil nil 342]]} "value-725"] [[{"string-739" {nil false, false 0.9122064508003991, 304 0.41858269859663066, 0 "value-855"}, false [0.03519776942913222 "value-128" nil 0.4930973352566015 true], nil {"string-390" 922, "string-342" "value-954", true 458, nil 0.23550807926330442, 572 false}, "string-7" 266} true "value-779"] 990 {:key-493 {:key-83 {nil true, :key-842 nil, 321 516, 301 138, false 0.4746132146497579}, :key-5 [0.7781361311988927 false nil], false "value-145", :key-381 [380 229]}, "string-122" [{"string-211" nil, nil 0.8769417377485428, 610 "value-115"}], :key-247 [[0.3293808216083872] [true] "value-772" nil], "string-890" "value-3", 100 [nil [397] {36 "value-705", nil "value-484", false nil, :key-843 0.14738354487967753, "string-629" "value-587"} true]}] "value-10"]}, nil {false {nil {"string-415" {:key-927 {nil [0.9330662734188742 nil "value-399"], :key-481 [false 134 "value-464" "value-22" 956], "string-82" 456}, 603 [[0.9355368617905081 "value-840" true] ["value-651"] {:key-594 nil, false "value-851", nil "value-449", true "value-375"} {"string-754" 768, :key-489 nil, nil "value-113", :key-686 false, true true}], "string-459" [[846] true [0.7325047574645939] nil], true [nil false {:key-839 "value-156", 508 nil, nil "value-36", 583 18, 928 0.1602565755852663} false [nil false 0.8677105426415401 137]], 952 0.6759569106253477}, false [{"string-642" {203 "value-871", "string-581" true, :key-130 580, 5 531, :key-606 428}, true {nil 921, "string-645" 52, "string-739" nil, "string-479" 0.468107058597661}, false ["value-327" 0.040235227557332776], "string-865" "value-706"} "value-862" false nil], "string-970" {"string-7" {nil {896 nil, nil 0.16658186137740438, :key-236 251, true "value-31", false 691}, 48 [nil "value-930" true 101 nil], "string-216" [76 nil], "string-365" {635 0.10687476673838181, nil true, :key-174 0.7378149761122964, :key-94 nil, false 0.020909504529875056}, 205 {94 true, true 0.996191357616638, nil nil, false 265, 466 275}}, :key-159 [{false 0.6037979426711828, nil 870, "string-393" false, :key-725 nil, "string-805" "value-235"} {nil true, false nil, :key-856 "value-492", :key-861 nil, "string-760" nil}], "string-127" [{false "value-654", nil true, :key-875 "value-756", 636 true} {false 387, :key-658 true, nil 540, "string-611" true} nil [nil true nil] {78 472, true 0.042313535370618105, 901 nil, "string-885" 353, 18 nil}], 183 0.2727848485502081, false ["value-862" {false 698, 859 0.8347872456376112, 137 "value-119", nil true, :key-362 "value-544"} {:key-599 "value-931", :key-636 false, nil true, :key-689 0.9191044292780851, "string-135" false} 0.8230137998073368]}, :key-44 [0.0494475239786869 [[nil "value-56" 280 false] 566 {false 0.6913776568066744, :key-349 "value-799", nil nil, "string-711" 0.20191914951795464}] 305 "value-633"], nil [{nil [nil true true "value-475" "value-427"], 215 ["value-218" true 132 false], true [false 724 nil 328 true], "string-143" {"string-40" 0.43453181697752075, nil "value-48", false 989, 435 nil, 750 true}}]}, :key-704 [{true true, 33 [{:key-606 nil, :key-80 365, :key-749 47, nil nil, "string-261" 0.5296983215263293} 307], false [[0.7379427890862493 0.8855631446650617 nil nil] "value-697" "value-355"], nil {true {true "value-405", "string-810" 0.7133207643065183, "string-998" "value-895", :key-254 true, false "value-361"}, "string-791" [nil nil], 311 nil, :key-585 nil, false 0.1931119673408701}} [[nil [115 true nil "value-870" "value-657"]] nil 0.252748578042745 nil {:key-855 {"string-513" nil, nil 0.0693933241543715, :key-610 0.17082185021550933, :key-884 835}, false [true nil nil], nil nil, "string-595" [nil true nil 162], :key-722 {:key-642 "value-691", false nil, :key-231 nil, nil 336, :key-552 "value-29"}}] 770 [[{nil false, :key-590 true, false 0.543003443179052, 720 nil, :key-647 0.780607826799234} "value-72" {:key-936 "value-968", :key-699 "value-439", "string-162" "value-391", nil 0.5140004241996323, true 214}] false]], "string-644" [{true {nil {nil true, :key-974 true, false nil, :key-314 0.46212581752457815}, :key-837 [true "value-742" "value-381" false], "string-876" nil}, "string-401" [{"string-351" 0.9152670449723654, nil 309, 407 "value-262"} ["value-401" 860 false] false], false 0.49225814737627704, "string-837" {nil [true "value-332" 594 "value-100"], 82 {:key-274 458, :key-324 "value-710", nil 0.9312979776500414, 496 nil}, false {nil 0.31621305547297696, :key-593 true, "string-340" true, :key-312 833}, :key-728 {true "value-618", nil false, "string-538" 0.5638165147021866, "string-907" "value-66", "string-990" true}}, "string-755" nil}], "string-331" [[[[380 "value-203" false 0.168551208207409 nil] [nil] true] [[0.5868632220660708 false]]] {nil [320 {nil 0.40164986421691473, "string-326" true, false nil, true 0.06698298682733106, 908 true} [false nil] {nil true, 651 45, "string-407" "value-344", true nil} nil], :key-927 {:key-271 {353 254, nil nil, :key-28 0.5427031230100046, :key-328 "value-666", 726 173}, :key-692 [0.18364720064664553 920 "value-584" true], nil {:key-736 false, :key-617 true, 155 0.707204538918652, nil 0.9983016174026691, :key-340 911}, true {"string-830" 0.23262808127071888, 240 339, :key-906 99, 600 "value-248", 708 671}, :key-635 ["value-503" false]}, "string-188" [{true 327, nil 667, false false, :key-54 0.9336844852945749} 924 {"string-922" "value-30", "string-38" 800, "string-380" false, true 367, 739 588}], 570 {855 {"string-195" 923, "string-135" "value-892", 5 nil, nil 0.8392376947317229, "string-583" "value-770"}, nil "value-62", 182 {"string-921" true, :key-978 0.15149501817197775, nil 151, "string-548" 0.4738134736255798, :key-14 false}, "string-211" {false "value-72", nil 0.6673872049068323, :key-108 false, :key-635 true}}} true]}, true [{"string-512" {false {"string-957" {true nil, nil nil, false "value-946", 404 nil}, false [nil], :key-23 0.3405639732795718, "string-556" [nil 874 "value-530"], true {nil 0.6261968122294661, "string-606" 99, "string-473" "value-597", 362 false, false nil}}, :key-547 ["value-72" 196 0.3470182186502979], "string-414" [{nil nil, 614 nil, "string-585" false, 113 370, 520 "value-553"} [nil 295 nil false nil] nil "value-821"], "string-617" "value-866", nil [["value-203" 969 true 31 310] ["value-340" 0.4503842625917446 "value-879" 0.9589952099232232] nil]}, true [[true ["value-433"] "value-54" ["value-436" 0.8453177856137577 false] 0.539054575505457] 774 {:key-77 {"string-21" 178, :key-799 "value-38", true 0.741976517475851, nil true}, :key-113 ["value-918" true 0.8633052495436642], "string-93" 0.5742618953283902, 924 false, "string-969" {:key-206 true, true 827, nil true, "string-806" 931, :key-209 nil}} {true ["value-959" false 0.9619000170716855 true 0.868985328837762], :key-86 [false], nil nil, false 0.28804527446515904} {:key-156 {false 995, "string-740" nil, 647 "value-544"}, nil [true], :key-181 {"string-497" 694, "string-958" false, "string-505" 766, true true, 224 "value-16"}, "string-89" nil, 347 [nil nil 215 0.84163942946125]}], nil "value-917", :key-983 [[672 [nil nil "value-480" 0.4875607573872377 0.11090180235159597] {true 873, "string-137" "value-108", :key-27 27, "string-296" true, nil 0.17913127812438123}] ["value-3" [0.1704849995056147 nil true true] [0.03817522440725807 false] 8 147] "value-583" false [[0.584502796382633 121 286] [nil 0.9341922811844295]]], 488 {:key-637 {nil [0.4012021862396391], :key-78 false, "string-973" "value-995", "string-814" nil}, "string-282" [{42 330, "string-454" 956, "string-217" true, nil true, 627 "value-956"} [true] {480 574, :key-324 true, nil 479, 56 "value-382", "string-410" false} 895], 788 {nil {nil 0.506912977036597, 954 nil, 708 false, :key-325 true, :key-498 false}, 32 [nil nil false], 598 false, :key-144 {:key-866 0.35782380916123047, :key-721 0.0312600814820434, true 845, nil true}}, true 0.7963732681414336, :key-682 [{557 0.9229497880394856, :key-884 nil, 118 false, 310 true, :key-744 nil} {316 false, nil 0.16075179423641173, true 0.6268971120251465, false nil}]}}], nil {false {893 {"string-775" {true {:key-464 954, "string-524" 849, 488 601, :key-444 "value-686", "string-218" 0.3607800499641999}, false ["value-385" "value-735" 461 false "value-638"], nil {true 0.7554766415162734, nil 690, 344 0.6488568104932577, 319 nil, "string-594" false}, "string-137" {"string-675" 0.3322414810167459, nil "value-811", 156 752, 821 false, 718 "value-547"}, "string-115" 945}, 968 [{"string-206" false, true 0.009669588633997472, "string-630" false, :key-176 0.7688501005620507, 741 true} false], 70 [{:key-804 false, "string-812" "value-435", true 441, :key-995 "value-462", nil nil}], :key-632 nil, "string-515" nil}, :key-291 [{"string-901" {false true, true "value-394", nil 684}, "string-71" [nil nil 5.814846348727309E-4 false 0.2535686749519308], true [false 0.1106899155598049 309 "value-623" "value-895"]} {"string-555" {:key-876 0.053871817834651115, 449 nil, false 762, 740 539, nil "value-945"}, :key-419 [nil 180 nil], "string-124" {nil 0.3889093620746069, "string-219" nil, false 941}, "string-998" [635 nil true], nil nil}], nil {true {true {:key-156 0.4734112383681326, "string-159" nil, false 754, true "value-466", 238 862}, :key-87 [0.3308181789985636 0.09735069480485048 "value-6" 0.9124111751969913], nil 0.15890434201520298, 452 ["value-828" "value-289" 0.836910204872324 nil]}, 686 [{:key-121 "value-808", :key-448 "value-264", 85 687, false 843, 4 620} 100 false], false 565, "string-111" "value-227", :key-596 {:key-423 {false 668, 776 461, true 693, nil false, :key-92 "value-780"}, nil [77 0.6571386111005223], false [0.2307170110087402 "value-170" "value-18"], 347 {"string-161" 535, :key-588 "value-264", false "value-32", "string-276" "value-832", :key-316 "value-143"}}}, :key-163 false, :key-36 {true {911 {:key-772 481, nil false, 217 "value-306", 599 "value-237", true 0.719395017240623}, nil [true], "string-638" [55 "value-330" 808 241 0.3307102714466532], 248 {957 398, 679 0.9551694212892492, 778 456, :key-264 false, :key-702 "value-746"}, true ["value-658" 971 "value-625" true 591]}, :key-20 [[nil true] {802 0.5869532355119382, 724 nil, 62 179, "string-150" "value-533", :key-296 "value-855"} {"string-178" 955, :key-8 true, "string-150" "value-899", "string-829" 0.236382746562655, "string-751" true} {:key-118 nil, nil false, false false, :key-789 nil} {808 752, false false, true false, "string-110" 997, 100 0.6401753060987844}], 281 [["value-58" 0.8340902275041475 341] 37 "value-579" {250 nil, 10 "value-352", 210 0.590466084856483, nil 95, :key-637 "value-988"}], :key-688 "value-772", nil {"string-281" {false nil, :key-261 true, :key-402 nil, "string-723" false, nil nil}, nil nil, true {nil 0.1335671121147336, true 0.39799461284999227, false nil, "string-676" nil, 700 "value-4"}}}}, "string-707" [[[396] ["value-96" ["value-868" 697 782]] [{"string-347" nil, :key-379 nil, :key-314 "value-404", nil 285, 346 true} {"string-664" "value-838", :key-725 450, 629 0.38653732908525607, "string-384" 325, nil "value-560"} {true nil, nil nil, :key-319 82, :key-466 "value-499", "string-300" 443}] [[true] {"string-274" nil, 222 616, "string-861" true, 698 898, true "value-323"} [nil "value-978" true]]] [{"string-725" {nil true, 302 87, :key-816 155, :key-319 "value-897"}, "string-472" [true], false {:key-257 "value-778", nil 44, :key-32 0.5505458933658466, 926 0.05367713873348945}, :key-170 0.7460968953830417} 186 nil 902 "value-801"] 0.7474392865585155], 148 {true [[[nil 447 0.11034097997920944 24] [889]] 871], false [[false nil [217 nil] [false 461 0.7992778180255791]] nil], "string-191" {nil "value-602", "string-733" [{963 true, 983 694, nil 0.8576582547301973, :key-219 "value-969", 969 "value-593"} 535 ["value-577" 922 nil]], "string-582" [{true 0.9273619436977727, :key-217 137, 608 "value-758", nil nil} [982 447 0.43996542476978884 "value-271" 0.8857592080162848] {:key-16 false, "string-270" 400, nil 929, true true, :key-978 nil} {nil 0.8457137436610612, 855 nil, :key-443 886, 327 410} ["value-235" nil 837]]}, 742 {:key-20 {976 {:key-768 true, :key-730 true, "string-223" nil, false 0.5297895292984423, :key-14 810}, nil [0.6480234414716807 491 959 157], :key-530 nil, :key-770 {false "value-775", 863 false, :key-660 "value-380"}, :key-80 nil}, nil [[true 659 nil 16]], :key-349 {379 {:key-21 "value-809", false 123, 916 343, true true}, :key-892 [nil], 729 {"string-716" "value-993", "string-98" "value-238", "string-997" false, nil false, "string-39" nil}, "string-173" [0.8207146383657645 false 0.7326570362333988], true {nil "value-500", false nil, "string-800" true, :key-771 497, true 0.4593375277864371}}, false {nil {true 0.06113512214693906, 304 "value-504", nil nil, 938 490}, 118 [0.17687565893144963 false "value-841" 0.7134412510128642], 205 0.13024253863820978, 433 {nil 723, true 357, :key-898 780}, false [nil]}}}, nil [true {"string-274" {227 {"string-962" 681, 684 "value-514", 742 true, :key-931 "value-557", 198 false}, :key-149 [0.028771900761456415], :key-658 false, 113 [781 nil true], :key-507 [true nil nil true 977]}, 634 [{"string-704" nil, :key-669 441, :key-175 nil, 127 19, true nil} {false false, nil 464, "string-20" 304, true nil} 66 [603 "value-711"]], true {false [606 0.22273381939215686], nil [522 "value-242"], true "value-706"}, "string-42" [nil {nil false, "string-901" nil, "string-922" 0.8142561867907638, false 934, true nil} "value-850"], "string-14" ["value-273" {703 false, "string-733" 360, :key-992 0.4315340809565742, nil "value-435", true 990}]} [993 {nil {58 60, false "value-611", 519 nil, nil 92}, "string-819" [false "value-962" "value-109" "value-585"], 333 ["value-935" true], :key-949 nil, "string-399" nil} {:key-570 {"string-568" 449, 159 991, "string-117" "value-454", nil true, true 0.9315769675615816}, :key-532 [0.7655443352791097 51 nil], :key-496 [nil 0.9094704163060308 "value-614"], :key-119 ["value-198" 0.6930216882940219 957 "value-618"], false [false 0.24102872668014563 true]}] [false {"string-788" {true "value-21", "string-360" nil, 291 0.472465859902233, "string-194" nil, "string-836" true}, "string-677" [0.6024664651991348 883], "string-468" {nil 0.9281926613135586, 62 nil, :key-903 nil, 117 "value-510"}, :key-489 {"string-653" false, "string-642" nil, nil 503, :key-361 "value-67"}, 334 {true true, false nil, "string-139" 137, :key-15 true, :key-329 false}} {"string-494" {false nil, nil 433, "string-917" 923, true 0.145149041438686, 846 0.3221614272425667}, 626 [nil], :key-479 0.8790881686329802, nil nil, false 79} {true [40 0.36924204078761536 true false], nil {454 0.7696320981587638, "string-80" false, 396 nil, false "value-788"}, 436 ["value-961" 771]}]], :key-122 {67 {:key-735 {true {:key-936 "value-95", false 0.9390738414082119, 881 nil, 638 58, :key-804 nil}, 57 [0.8762986982071507 true "value-599" 186 0.43632697531144704], :key-869 {222 0.6663720674655913, false 0.8189391883586806, "string-52" 0.21009663393272626, :key-183 "value-645", true true}, false [nil "value-757"], nil ["value-218" 223]}, 840 [[nil] {true 215, 185 "value-486", "string-665" "value-992", 365 nil, :key-204 "value-230"} "value-886"], "string-531" nil, nil ["value-705" [nil] 695], "string-837" {true {false 0.29526640480190736, 498 124, nil 0.8197003963471307, 662 833}, 792 [212 "value-302" "value-867"], "string-411" [0.41365098400147415], "string-426" 0.9845638108375843, nil [0.1063160786596612]}}, 228 [[{864 36, "string-101" "value-244", true 0.5873962763238674, nil 0.8760525517117149, "string-421" false}] [nil true true {nil false, true 0.34162369647253177, :key-719 0.24011556194254768, "string-188" nil}] {nil 0.528195902678463, false 355, :key-424 {:key-621 698, "string-400" nil, nil 0.7823225104194431, "string-195" nil, 444 false}}], true [[{"string-815" 391, nil nil, false 0.5513329213196726, 721 true}] {"string-323" {nil nil, :key-703 "value-586", :key-188 "value-24", true 0.024447230202252923}, :key-465 [0.9036397582423945 312 "value-606" false "value-223"], true false, :key-635 [nil 360 nil "value-327" nil], :key-852 true} [0.9456983036781235 nil [526 true]] {:key-180 {"string-20" 117, 878 0.1449219188579277, nil 143, false 0.5637088103841638, 267 true}, true [980 0.9915835434031989], nil ["value-209" true true], "string-744" {:key-317 837, "string-931" false, 212 184, nil false, :key-48 nil}}], 117 994, "string-29" {nil {282 {480 0.9739597515654512, "string-878" nil, "string-646" 0.997257851752815, 287 nil, nil 388}, true {"string-24" "value-838", nil 0.9958809915520932, 931 false, :key-743 "value-328"}, false "value-708"}, :key-753 [nil {:key-330 732, 825 false, :key-400 nil, 278 0.5947884618715639, nil "value-52"}], 531 nil, true [{nil true, true nil, :key-487 nil, false 0.726463621474066, :key-727 "value-179"} ["value-639"] [nil true] [0.178376819670519] {nil 0.047203701749166216, "string-874" "value-64", 684 false, "string-501" 0.6670346687052693}]}}}, :key-111 ["value-563" [[[{:key-973 nil, 905 false, :key-188 0.603142561417479, 110 nil, "string-378" "value-567"} {false "value-880", nil nil, 385 "value-832", :key-756 false} [nil nil nil "value-146" true] {"string-696" "value-200", false "value-778", nil nil, "string-605" nil, 913 nil} {232 "value-313", :key-971 844, true 0.18723021335296397, nil nil, 368 true}]] 19 [204 {false {"string-203" nil, "string-831" "value-599", :key-762 nil, 427 0.8622909337862248, "string-774" 223}, :key-712 ["value-289"], "string-496" {"string-489" "value-971", :key-35 nil, :key-486 "value-159", 976 503, :key-678 0.5541040388962788}, "string-699" {true 0.25227679768183076, :key-642 false, nil "value-976", 798 true}, nil {nil 200, :key-379 "value-972", 726 0.3779025373904975, 397 "value-91", true nil}}]] "value-69" {"string-579" {false {nil [false 0.3204755502932426 0.7056387809764684 false], "string-4" [111 0.8209911959004986 0.28420176974851474 961], false 0.16324441566681758, 125 nil}, "string-369" [{:key-64 0.6723751247507367, true "value-913", "string-929" true, "string-459" "value-11", 908 "value-884"} [nil nil false 199] "value-850" [0.6846304250019183 "value-128" "value-715" 0.6906398588352738]], nil [false], true {558 {"string-433" false, :key-695 "value-831", 852 0.6875565963831629, nil false, 583 0.7188496086136674}, "string-286" [nil nil 900 0.6548021651231454 823], :key-341 [nil true], :key-116 {"string-840" nil, nil 142, true true, 459 0.9717061385179873, :key-38 nil}, true {"string-356" 978, "string-765" 0.568277412941587, "string-558" "value-378", nil 366, :key-566 "value-980"}}, 762 nil}, false [[0.7430295351634063] false false [0.26694372368068264 "value-63" [0.6178555107789585] nil [nil nil 0.5334362271300563]] 0.46575744111362605], :key-520 [[nil {"string-603" 690, 564 0.4488747458965874, :key-670 469, "string-49" 567, 485 false} {false false, nil 143, true nil} ["value-817" true] "value-646"]], 5 {nil [[true true "value-561" nil] 0.5239724220244286 true ["value-337" nil 220 "value-731"]], true {"string-90" {"string-296" 0.47432362931661953, 693 383, false 0.0740034934341276, nil false, true "value-224"}, :key-617 [nil true], true [nil 0.06442938640226925 nil], :key-447 {true 0.4741041177608524, false 0.1372970422098153, "string-37" 0.5748519299842493, 835 144, :key-617 true}}, "string-581" [nil nil {"string-111" nil, "string-167" 660, nil 0.501306855587283, false "value-609"}]}, :key-298 {961 {nil nil, "string-368" 882, "string-935" {true 0.32008646325883694, "string-489" 0.25188188675554013, "string-745" 0.2629044959281126, 625 "value-656", false false}}, false ["value-814" ["value-252" nil false] true [0.6578784189417519 true 320 true "value-252"] [0.38236196239669884 false]], :key-543 {"string-623" {nil nil, "string-498" 898, :key-31 0.724648783386005, "string-308" "value-106", 826 196}, 709 [true], 422 {802 0.501228459868283, nil false, "string-830" 0.7047272502667266}, nil [false 0.8083565757216299 0.9997086934194157 426], "string-753" "value-35"}, nil [{nil true, "string-612" 811, true nil} [nil false nil nil] {true 0.5769626831939609, "string-518" true, 941 nil, 584 744, "string-890" "value-848"} {330 44, nil "value-163", :key-351 false, "string-365" 0.6850698680625775} 0.11444257377548839], :key-996 {true {nil 482, 915 691, 734 0.6965386898255171}, "string-914" [false 636 0.1606985911307076 "value-90" "value-744"], nil 0.08312300742765999, false [91 "value-123" nil nil]}}}]}} -------------------------------------------------------------------------------- /test/xitdb/cursor_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.cursor-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.db :as xdb])) 5 | 6 | (deftest CursorTest 7 | (with-open [db (xdb/xit-db :memory)] 8 | (reset! db {:foo {:bar [1 2 3 {:hidden true} 5]}}) 9 | (let [cursor1 (xdb/xdb-cursor db [:foo :bar]) 10 | cursor2 (xdb/xdb-cursor db [:foo :bar 2]) 11 | cursor3 (xdb/xdb-cursor db [:foo :bar 3 :hidden])] 12 | (testing "Cursors return the value at keypath" 13 | (is (= [1 2 3 {:hidden true} 5] (xdb/materialize @cursor1))) 14 | (is (= 3 @cursor2)) 15 | (is (= true @cursor3))) 16 | 17 | (testing "reset! on the cursor changes the underlying database" 18 | (reset! cursor3 :changed) 19 | (is (= :changed @cursor3)) 20 | (is (= :changed (get-in @db [:foo :bar 3 :hidden]))) 21 | (is (= [1 2 3 {:hidden :changed} 5]) (xdb/materialize @cursor1))) 22 | 23 | (testing "swap! mutates the value at cursor" 24 | (swap! cursor1 assoc-in [3 :hidden] :changed-by-swap!) 25 | (is (= [1 2 3 {:hidden :changed-by-swap!} 5]) (xdb/materialize @cursor1)) 26 | (is (= :changed-by-swap! @cursor3)) 27 | (is (= 3 @cursor2)))))) 28 | -------------------------------------------------------------------------------- /test/xitdb/data_types_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.data-types-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | 7 | (deftest KeywordsAndStrings 8 | (testing "Should correctly handle key and string keys and values" 9 | (with-db [db (tu/test-db)] 10 | (reset! db {:foo "foo" 11 | "foo" "more-foo" 12 | ":foo" :foo}) 13 | (is (= {:foo "foo" 14 | "foo" "more-foo" 15 | ":foo" :foo} @db)) 16 | (swap! db dissoc "foo" ":foo") 17 | (is (= {:foo "foo"} @db)) 18 | (swap! db dissoc :foo) 19 | (is (= {} @db))))) 20 | 21 | (deftest KeywordsAndStringsInSets 22 | (testing "Should correctly handle keywords and strings in sets" 23 | (with-db [db (tu/test-db)] 24 | (reset! db #{:foo "foo" ":foo"}) 25 | (is (= #{:foo "foo" ":foo"} @db)) 26 | 27 | (swap! db conj :bar) 28 | (is (= #{:foo "foo" ":foo" :bar} @db)) 29 | 30 | (swap! db disj :foo ":foo") 31 | (is (= #{"foo" :bar} @db)) 32 | 33 | (swap! db disj "foo") 34 | (is (= #{:bar} @db))))) 35 | 36 | (deftest KeywordsAndStringsInVectors 37 | (testing "Should correctly handle keywords and strings in vectors" 38 | (with-db [db (tu/test-db)] 39 | (reset! db [:foo "foo" ":foo"]) 40 | (is (= [:foo "foo" ":foo"] @db)) 41 | 42 | (swap! db conj :bar) 43 | (is (= [:foo "foo" ":foo" :bar] @db)) 44 | 45 | (swap! db assoc 0 "changed") 46 | (is (= ["changed" "foo" ":foo" :bar] @db)) 47 | 48 | (swap! db #(into [] (remove #{":foo"} %))) 49 | (is (= ["changed" "foo" :bar] @db))))) 50 | 51 | (deftest KeywordsAndStringsInLists 52 | (testing "Should correctly handle keywords and strings in lists" 53 | (with-db [db (tu/test-db)] 54 | (reset! db '(:foo "foo" ":foo")) 55 | (is (= '(:foo "foo" ":foo") @db)) 56 | 57 | (swap! db conj :bar) 58 | (is (= '(:bar :foo "foo" ":foo") @db))))) 59 | 60 | (deftest NilValuesInMaps 61 | (testing "Should correctly handle nil values and distinguish from missing keys" 62 | (with-db [db (tu/test-db)] 63 | ;; Test nil values vs missing keys 64 | (reset! db {:existing-key nil :other-key "value"}) 65 | (is (contains? @db :existing-key)) 66 | (is (= nil (get @db :existing-key))) 67 | (is (= nil (get @db :missing-key))) 68 | (is (not (contains? @db :missing-key))) 69 | 70 | ;; Test entryAt with nil values 71 | (is (= [:existing-key nil] (find @db :existing-key))) 72 | (is (= nil (find @db :missing-key))) 73 | 74 | ;; Test nil as a key 75 | (reset! db {nil "nil-value" :other "other-value"}) 76 | (is (contains? @db nil)) 77 | (is (= "nil-value" (get @db nil))) 78 | (is (= [nil "nil-value"] (find @db nil)))))) 79 | 80 | -------------------------------------------------------------------------------- /test/xitdb/database_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.database-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | (deftest DatabaseTest 7 | (with-db [db (tu/test-db)] 8 | (testing "Resetting to map" 9 | (reset! db {:foo :bar}) 10 | (is (= {:foo :bar} @db)) 11 | 12 | (reset! db {:foo {:bar :Baz}}) 13 | (is (= {:foo {:bar :Baz}} @db)) 14 | 15 | (reset! db {:foo {:bar {:some "baz"}}}) 16 | (is (= {:foo {:bar {:some "baz"}}} @db)) 17 | 18 | (swap! db assoc-in [:foo :bar :some] [1 2 3 4]) 19 | (is (= {:foo {:bar {:some [1 2 3 4]}}} @db)) 20 | 21 | (swap! db assoc-in [:foo :bar :some] -42) 22 | (is (= (:foo {:bar {:some -42}}))) 23 | 24 | (is (tu/db-equal-to-atom? db))))) 25 | 26 | (deftest get-in-test 27 | (with-db [db (tu/test-memory-db-raw)] 28 | (reset! db {:foo {:bar :baz}}) 29 | (is (= :baz (get-in @db [:foo :bar]))) 30 | (reset! db {"foo" {:bar {1 :baz}}}) 31 | (is (= :baz (get-in @db ["foo" :bar 1]))) 32 | 33 | (testing "Negative keys and values" 34 | (reset! db {-999 {:bar -42}}) 35 | (is (= {-999 {:bar -42}} (tu/materialize @db))) 36 | (is (= -42 (get-in @db [-999 :bar]))) 37 | 38 | (reset! db [-39 23 "foo" :foo]) 39 | (is (= -39 (get-in @db [0]))) 40 | (is (= 23 (get-in @db [1]))) 41 | (is (= "foo" (get-in @db [2]))) 42 | (is (= :foo (get-in @db [3])))) 43 | 44 | (testing "Float values" 45 | (reset! db {-34 0.345542}) 46 | (is (= @db {-34 0.345542})) 47 | (swap! db assoc -424 3.949494958483) 48 | (is (= {-34 0.345542, -424 3.949494958483} 49 | (tu/materialize @db)))))) 50 | 51 | (deftest array-reset-test 52 | (with-db [db (tu/test-db)] 53 | (testing "Resetting to array" 54 | ;; Start with a fresh database and reset to vector 55 | (reset! db [1 2 3 4]) 56 | (is (= [1 2 3 4] @db)) 57 | 58 | ;; Test accessing elements 59 | (is (= 3 (get @db 2))) 60 | 61 | ;; Test updating an element 62 | (swap! db assoc-in [1] 20) 63 | (is (= [1 20 3 4] @db)) 64 | 65 | ;; Test adding elements 66 | (swap! db assoc-in [4] 5) 67 | (is (= [1 20 3 4 5] @db)) 68 | (is (tu/db-equal-to-atom? db))))) 69 | 70 | (deftest map-corner-cases-test 71 | (with-db [db (tu/test-db)] 72 | 73 | (testing "Empty map operations" 74 | (reset! db {}) 75 | (is (= {} @db)) 76 | (swap! db assoc-in [:foo] "bar") 77 | (is (= {:foo "bar"} @db))) 78 | 79 | (testing "Nested empty collections" 80 | (reset! db {:empty-map {} :empty-vec []}) 81 | (is (= {:empty-map {} :empty-vec []} @db)) 82 | (swap! db assoc-in [:empty-map :key] "value") 83 | (is (= {:empty-map {:key "value"} :empty-vec []} @db))) 84 | 85 | (testing "Special keys" 86 | (reset! db {}) 87 | (swap! db assoc-in [:ns/keyword] "namespaced") 88 | (swap! db assoc-in ["string-key"] "string") 89 | (is (= {:ns/keyword "namespaced" "string-key" "string"} @db))) 90 | 91 | (testing "Creating nested paths" 92 | (reset! db {}) 93 | (swap! db assoc-in [:a :b :c :d] "deep") 94 | (is (= {:a {:b {:c {:d "deep"}}}} @db))) 95 | 96 | (testing "Replacing data types" 97 | (reset! db {:foo {:bar "string"}}) 98 | (swap! db assoc-in [:foo :bar] [1 2 3]) 99 | (is (= {:foo {:bar [1 2 3]}} @db)) 100 | (swap! db assoc-in [:foo :bar] {:nested "map"}) 101 | (is (= {:foo {:bar {:nested "map"}}} @db))) 102 | 103 | (testing "Numeric and boolean keys" 104 | (reset! db {}) 105 | (swap! db assoc-in [1] "numeric") 106 | 107 | (is (= {1 "numeric"} @db) "Keys are preserved") 108 | (swap! db assoc-in [true] "boolean") 109 | (is (= {1 "numeric" true "boolean"} @db))) 110 | (is (tu/db-equal-to-atom? db)))) 111 | 112 | (deftest SwapTest 113 | (with-db [db (tu/test-db)] 114 | (reset! db {:foo :bar}) 115 | (is (= {:foo :bar} @db)) 116 | 117 | (testing "arity-1 assoc" 118 | (swap! db #(assoc % :some 43)) 119 | (is (= {:foo :bar :some 43} @db))) 120 | 121 | (testing "arity-2 assoc" 122 | (swap! db assoc :some 44) 123 | (is (= {:foo :bar :some 44} @db))) 124 | 125 | (testing "arity-3 assoc" 126 | (reset! db {:users {"1" {:name "john"}}}) 127 | (is (= {:users {"1" {:name "john"}}} @db)) 128 | 129 | (swap! db assoc-in [:users "1" :age] 44) 130 | (is (= {:users {"1" {:name "john" :age 44}}} @db))) 131 | 132 | (testing "dissoc" 133 | (reset! db {:users {"1" {:name "john"}}}) 134 | (swap! db dissoc :users) 135 | (is (= {} @db)) 136 | 137 | (reset! db {:users [] :foo :stays}) 138 | (swap! db dissoc :users) 139 | (is (= {:foo :stays} @db))) 140 | 141 | (testing "more dissoc" 142 | (reset! db {:users {"1" {:name "john"}}}) 143 | (swap! db dissoc :users) 144 | (is (= {} @db))) 145 | (is (tu/db-equal-to-atom? db)))) 146 | 147 | (deftest SwapArray 148 | (with-db [db (tu/test-db)] 149 | (testing "assoc" 150 | (reset! db [1 2 3]) 151 | (swap! db #(assoc % 0 44)) 152 | (is (= [44 2 3] @db)) 153 | 154 | (swap! db #(assoc % 3 99)) 155 | (is (= [44 2 3 99] @db))) 156 | 157 | (testing "conj" 158 | (reset! db [1 2 3]) 159 | (swap! db conj 55) 160 | (is (= [1 2 3 55] @db))) 161 | 162 | (testing "assoc-in" 163 | (reset! db [1 2 {:title "Untitled"} 3 4]) 164 | (swap! db assoc-in [2 :title] "Titled") 165 | (is (= [1 2 {:title "Titled"} 3 4] @db))) 166 | 167 | (testing "update-in" 168 | (reset! db [1 2 {:users [{:name "jp"} 169 | {:name "cj"}]} 3 4]) 170 | (swap! db update-in [2 :users] conj {:name "fl"}) 171 | (is (= [1 2 {:users [{:name "jp"} {:name "cj"} {:name "fl"}]} 3 4] 172 | @db))) 173 | 174 | (testing "update-in array" 175 | (reset! db [1 2 [{:name "jp"} {:name "cj"}] 3 4]) 176 | (swap! db update-in [2 1 :name ] str "-foo") 177 | (is (= [1 2 [{:name "jp"} {:name "cj-foo"}] 3 4] @db)) 178 | (swap! db update-in [2 1] dissoc :name) 179 | (is (= [1 2 [{:name "jp"} {}] 3 4] @db)) 180 | (swap! db update-in [2] butlast) 181 | (is (= [1 2 [{:name "jp"}] 3 4] @db))) 182 | 183 | (is (tu/db-equal-to-atom? db)))) 184 | 185 | (deftest AssocInTest 186 | (with-db [db (tu/test-db)] 187 | (testing "assoc-in more" 188 | (reset! db [1 2 {:users [{:name "jp"} 189 | {:name "cj"}]} 3 4]) 190 | (swap! db assoc-in [2 :users 1 :name] "maria") 191 | (is (= [1 2 {:users [{:name "jp"} {:name "maria"}]} 3 4] 192 | @db)) 193 | (is (tu/db-equal-to-atom? db))))) 194 | 195 | (deftest MergeTest 196 | (testing "Basic merges" 197 | (with-db [db (tu/test-db)] 198 | (reset! db {"1" {:name "jp"} 199 | "2" {:name "cj"}}) 200 | 201 | (swap! db merge {"3" {:name "maria"}}) 202 | 203 | (is (= {"1" {:name "jp"} 204 | "2" {:name "cj"} 205 | "3" {:name "maria"}} 206 | @db)) 207 | 208 | (swap! db merge {:foo [:bar]}) 209 | (is (= {"1" {:name "jp"} 210 | "2" {:name "cj"} 211 | "3" {:name "maria"} 212 | :foo [:bar]} 213 | @db)) 214 | (is (tu/db-equal-to-atom? db)))) 215 | (testing "merge-with" 216 | (with-db [db (tu/test-db)] 217 | (reset! db {"1" {:foo1 {:bar :baz}}}) 218 | (swap! db merge {"2" {:foo2 :bar2}}) 219 | (swap! db merge {"3" {:foo3 :bar3}}) 220 | (swap! db merge {"4" [1 2 3]}) 221 | (swap! db (fn [a] 222 | (merge-with into a {"4" [44]}))) 223 | (is (= {"2" {:foo2 :bar2} 224 | "4" [1 2 3 44] 225 | "1" {:foo1 {:bar :baz}} 226 | "3" {:foo3 :bar3}} 227 | (tu/materialize @db)))))) 228 | 229 | (deftest IntoTest 230 | (with-db [db (tu/test-db)] 231 | (reset! db [0 1 2 3 4]) 232 | (swap! db #(filterv even? %)) 233 | (is (= [0 2 4] 234 | @db)))) 235 | 236 | ;;; Tests below were AI generated 237 | 238 | (deftest UpdateTest 239 | (with-db [db (tu/test-db)] 240 | (testing "update map value" 241 | (reset! db {:count 5 :name "test"}) 242 | (swap! db update :count inc) 243 | (is (= {:count 6 :name "test"} @db))) 244 | 245 | (testing "update with multiple args" 246 | (reset! db {:list [1 2 3]}) 247 | (swap! db update :list conj 4 5) 248 | (is (= {:list [1 2 3 4 5]} @db))) 249 | 250 | (is (tu/db-equal-to-atom? db)))) 251 | 252 | (deftest SelectKeysTest 253 | (with-db [db (tu/test-db)] 254 | (reset! db {:a 1 :b 2 :c 3 :d 4}) 255 | (swap! db select-keys [:a :c]) 256 | (is (= {:a 1 :c 3} @db)) 257 | (is (tu/db-equal-to-atom? db)))) 258 | 259 | (deftest FilterRemoveTest 260 | (with-db [db (tu/test-db)] 261 | (testing "filter" 262 | (reset! db {:a 1 :b 2 :c 3 :d 4}) 263 | (swap! db #(into {} (filter (fn [[_ v]] (even? v)) %))) 264 | (is (= {:b 2 :d 4} @db))) 265 | 266 | (testing "remove with vector" 267 | (reset! db [1 2 3 4 5]) 268 | (swap! db #(vec (remove odd? %))) 269 | (is (= [2 4] @db))) 270 | 271 | (is (tu/db-equal-to-atom? db)))) 272 | 273 | (deftest MapReduceTest 274 | (with-db [db (tu/test-db)] 275 | (testing "map over vector" 276 | (reset! db [1 2 3 4]) 277 | (swap! db #(vec (map inc %))) 278 | (is (= [2 3 4 5] @db))) 279 | 280 | (testing "map over map values" 281 | (reset! db {:a 1 :b 2}) 282 | (swap! db #(zipmap (keys %) (map inc (vals %)))) 283 | (is (= {:a 2 :b 3} @db))) 284 | 285 | (testing "reduce" 286 | (reset! db [1 2 3 4 5]) 287 | (swap! db #(vector (reduce + %))) 288 | (is (= [15] @db))) 289 | 290 | (is (tu/db-equal-to-atom? db)))) 291 | 292 | (deftest SequenceOpsTest 293 | (with-db [db (tu/test-db)] 294 | (testing "concat" 295 | (reset! db [1 2 3]) 296 | (swap! db #(vec (concat % [4 5 6]))) 297 | (is (= [1 2 3 4 5 6] @db))) 298 | 299 | (testing "take/drop" 300 | (reset! db [1 2 3 4 5]) 301 | (swap! db #(vec (take 3 %))) 302 | (is (= [1 2 3] @db)) 303 | 304 | (reset! db [1 2 3 4 5]) 305 | (swap! db #(vec (drop 2 %))) 306 | (is (= [3 4 5] @db))) 307 | 308 | (testing "partition" 309 | (reset! db [1 2 3 4 5 6]) 310 | (swap! db #(vec (map vec (partition 2 %)))) 311 | (is (= [[1 2] [3 4] [5 6]] @db))) 312 | 313 | (is (tu/db-equal-to-atom? db)))) 314 | 315 | (deftest EmptyOnArray 316 | (with-db [db (tu/test-db)] 317 | (reset! db [1 2 3]) 318 | (swap! db empty) 319 | (is (= [] @db)) 320 | (reset! db {:one [1 2 4]}) 321 | (swap! db update :one empty) 322 | (is (= {:one []} @db)) 323 | @db)) 324 | 325 | (deftest MiscOpsTest 326 | (with-db [db (tu/test-db)] 327 | (testing "empty" 328 | (reset! db {:a 1 :b 2}) 329 | (swap! db empty) 330 | (is (= {} @db)) 331 | 332 | (reset! db [1 2 3]) 333 | (swap! db empty) 334 | (is (= [] @db))) 335 | 336 | (is (tu/db-equal-to-atom? db)))) 337 | 338 | (deftest JuxtTest 339 | (with-db [db (tu/test-db)] 340 | (testing "juxt" 341 | (reset! db {:users [{:name "John" :age 30} {:name "Alice" :age 25}]}) 342 | (swap! db update-in [:users] #(mapv (juxt :name :age) %)) 343 | (is (= {:users [["John" 30] ["Alice" 25]]} @db))) 344 | (is (tu/db-equal-to-atom? db)))) 345 | 346 | (deftest EmptyTest 347 | (with-db [db (tu/test-db)] 348 | (reset! db {}) 349 | (is (empty? @db)) 350 | 351 | (reset! db []) 352 | (is (empty? @db)) 353 | 354 | (swap! db conj 1) 355 | (is (false? (empty? @db))) 356 | 357 | (swap! db butlast) 358 | (is (empty? @db)) 359 | 360 | (is (tu/db-equal-to-atom? db)))) 361 | 362 | (deftest CountTest 363 | (with-db [db (tu/test-memory-db-raw)] 364 | (reset! db {:a :b :c :d :e :f}) 365 | (is (= 3 (count @db))) 366 | 367 | (swap! db assoc :x :y) 368 | (is (= 4 (count @db))) 369 | 370 | (swap! db dissoc :x :a :b :c :e) 371 | (is (= 0 (count @db))) 372 | (is (empty? @db)) 373 | 374 | (is (tu/db-equal-to-atom? db)))) 375 | 376 | (deftest NilTest 377 | (testing "nil values" 378 | (with-db [db (tu/test-db)] 379 | (reset! db {:one nil}) 380 | (is (= {:one nil} @db)) 381 | 382 | (swap! db update :one conj 1) 383 | (is (= {:one [1]} @db)) 384 | 385 | (reset! db [1 nil nil 2 3 nil]) 386 | (is (= [1 nil nil 2 3 nil] @db)) 387 | 388 | (reset! db nil) 389 | (is (= nil @db)) 390 | @db)) 391 | 392 | (testing "nil keys" 393 | (with-db [db (tu/test-db)] 394 | (reset! db {:one nil}) 395 | (is (= {:one nil} @db)) 396 | (swap! db merge {nil nil}) 397 | 398 | (is (= {:one nil nil nil} @db)) 399 | (is (= {nil nil} (select-keys @db [nil])))))) 400 | 401 | (deftest InstAndDateTest 402 | (with-db [db (tu/test-db)] 403 | (let [d (java.util.Date.) 404 | instant (.toInstant d)] 405 | (reset! db {:foo instant 406 | :bar d}) 407 | (is (= instant (:foo @db))) 408 | (is (= d (:bar @db))) 409 | 410 | (is (instance? java.util.Date (:bar @db)))))) 411 | 412 | (deftest IntoEfficiency 413 | (with-db [db (tu/test-db)] 414 | (reset! db [1 2 3]) 415 | (swap! db into [4 5]) 416 | (swap! db into [5 6]) 417 | (is (= [1 2 3 4 5 5 6] @db)))) 418 | 419 | (deftest LazySeqTest 420 | (with-db [db (tu/test-db)] 421 | (reset! db '(1 2 3)) 422 | (swap! db conj 44) 423 | 424 | (testing "Throws on lazy seqs by concat" 425 | (is (thrown? IllegalArgumentException (swap! db concat [1 2]))) 426 | (swap! db (comp vec concat) [1 2]) 427 | (is (= [44 1 2 3 1 2] @db)) 428 | (swap! db (comp seq concat) [99]) 429 | (is (= '(44 1 2 3 1 2 99) @db))) 430 | 431 | (testing "Throws on take or drop" 432 | (is (thrown? IllegalArgumentException (swap! db #(drop 3 %)))) 433 | (is (thrown? IllegalArgumentException (swap! db #(take 3 %)))) 434 | (swap! db #(vec (drop 3 %))) 435 | (is (= @db [3 1 2 99]))))) 436 | 437 | (deftest PopTest 438 | (with-db [db (tu/test-db)] 439 | (reset! db '(1 2 3 4 5)) 440 | (swap! db pop) 441 | (is (= '(2 3 4 5) @db)) 442 | (is (tu/db-equal-to-atom? db)))) 443 | 444 | 445 | 446 | 447 | 448 | -------------------------------------------------------------------------------- /test/xitdb/deep_merge_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.deep-merge-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | (deftest DeepMergeTest 7 | (testing "deep nested merges" 8 | (let [db (tu/test-db)] 9 | ;; Set up initial nested map 10 | (reset! db {"users" {:profile {:name "John" 11 | :settings {:theme "dark"}} 12 | :permissions {:read true 13 | :write false}}}) 14 | 15 | ;; Define a recursive deep merge function 16 | (let [deep-merge (fn deep-merge [& maps] 17 | (apply merge-with (fn [x y] 18 | (if (and (map? x) (map? y)) 19 | (deep-merge x y) 20 | y)) 21 | maps))] 22 | 23 | ;; Perform a deep merge 24 | (swap! db deep-merge {"users" {:profile {:settings {:notifications true}} 25 | :permissions {:admin false}}}) 26 | 27 | (is (= {"users" {:profile {:name "John" 28 | :settings {:theme "dark" 29 | :notifications true}} 30 | :permissions {:read true 31 | :write false 32 | :admin false}}} 33 | @db)) 34 | 35 | ;; Another deep merge, overwriting some values 36 | (swap! db deep-merge {"users" {:profile {:name "Jane" 37 | :settings {:theme "light"}} 38 | :stats {:visits 10}}}) 39 | 40 | (is (= {"users" {:profile {:name "Jane" 41 | :settings {:theme "light" 42 | :notifications true}} 43 | :permissions {:read true 44 | :write false 45 | :admin false} 46 | :stats {:visits 10}}} 47 | @db))) 48 | 49 | ;; Using update-in for targeted nested merges 50 | (swap! db update-in ["users" :profile :settings] 51 | merge {:language "en" :fontSize "medium"}) 52 | 53 | (is (= {"users" {:profile {:name "Jane" 54 | :settings {:theme "light" 55 | :notifications true 56 | :language "en" 57 | :fontSize "medium"}} 58 | :permissions {:read true 59 | :write false 60 | :admin false} 61 | :stats {:visits 10}}} 62 | @db)) 63 | (is (tu/db-equal-to-atom? db))))) -------------------------------------------------------------------------------- /test/xitdb/gen_map.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.gen-map 2 | (:require 3 | [clojure.string :as str])) 4 | 5 | ;;Code below Generated From Prompt by Claude 3.7 6 | ;;I need a Clojure function which generates a deeply nested map with random keys and random values. 7 | ;;I will use this function to generate maps to test a database implementation. 8 | ;;Each nested map has at least one key valued by a map and one key valued by a vector. 9 | ;;Keys can be keywords, integers, strings, boolean or nil. Values can also be maps or arrays. 10 | ;;I should be able to configure: num-keys-per-map and max-array-length and max-nesting-level 11 | 12 | (defn gen-random-map 13 | "Generate a deeply nested map with random keys and values. 14 | Each map has at least one key with a map value and one key with a vector value. 15 | 16 | Parameters: 17 | - num-keys-per-map: number of keys in each map 18 | - max-array-length: maximum length of generated arrays 19 | - max-nesting-level: maximum depth of nesting" 20 | [num-keys-per-map max-array-length max-nesting-level] 21 | 22 | (letfn [(random-key [] 23 | (let [key-type (rand-int 5)] 24 | (case key-type 25 | 0 (keyword (str "key-" (rand-int 1000))) ; keyword 26 | 1 (rand-int 1000) ; integer 27 | 2 (str "string-" (rand-int 1000)) ; string 28 | 3 (zero? (rand-int 2)) ; boolean 29 | 4 nil))) ; nil 30 | 31 | (random-primitive [] 32 | (let [value-type (rand-int 5)] 33 | (case value-type 34 | 0 (rand-int 1000) ; integer 35 | 1 (str "value-" (rand-int 1000)) ; string 36 | 2 (zero? (rand-int 2)) ; boolean 37 | 3 nil ; nil 38 | 4 (rand)))) ; float 39 | 40 | (generate-array [level] 41 | (let [length (inc (rand-int max-array-length))] 42 | (vec (repeatedly length #(generate-value level))))) 43 | 44 | (generate-value [level] 45 | (if (>= level max-nesting-level) 46 | (random-primitive) 47 | (let [value-type (rand-int 3)] 48 | (case value-type 49 | 0 (random-primitive) 50 | 1 (generate-nested-map (inc level)) 51 | 2 (generate-array (inc level)))))) 52 | 53 | (generate-nested-map [level] 54 | (if (>= level max-nesting-level) 55 | ;; At max level, just return primitives 56 | (into {} (for [_ (range (max 1 num-keys-per-map))] 57 | [(random-key) (random-primitive)])) 58 | ;; Otherwise ensure we have at least one map and one vector 59 | (let [keys-to-generate (max 2 num-keys-per-map) 60 | map-key (random-key) 61 | vector-key (random-key) 62 | 63 | ;; The rest of the entries 64 | remaining-keys (- keys-to-generate 2) 65 | regular-entries (for [_ (range remaining-keys)] 66 | [(random-key) (generate-value level)])] 67 | 68 | ;; Combine ensuring we have the required structure 69 | (into {} (concat 70 | [[map-key (generate-nested-map (inc level))]] 71 | [[vector-key (generate-array (inc level))]] 72 | regular-entries)))))] 73 | 74 | (generate-nested-map 0))) 75 | 76 | (defn random-map-paths 77 | "Given a deeply nested map, returns a collection of n [path value] tuples. 78 | Each tuple contains a path vector (for use with get-in) and its corresponding value. 79 | Paths are selected randomly from all possible paths in the map." 80 | [m n] 81 | (letfn [(collect-paths [data path] 82 | (cond 83 | (map? data) 84 | (mapcat (fn [[k v]] 85 | (collect-paths v (conj path k))) 86 | data) 87 | 88 | (and (coll? data) (not (string? data))) 89 | (mapcat (fn [i v] 90 | (collect-paths v (conj path i))) 91 | (range) data) 92 | 93 | :else [[path data]]))] 94 | 95 | (if (pos? n) 96 | (let [all-paths (collect-paths m []) 97 | shuffled-paths (shuffle all-paths)] 98 | (take n shuffled-paths)) 99 | []))) 100 | 101 | (comment 102 | 103 | (def rmap (gen-random-map 5 5 3)) 104 | (def paths (random-map-paths rmap 3)) 105 | 106 | (doseq [path paths] 107 | (let [keypath (first path) 108 | value (second path)] 109 | (= value (get-in rmap keypath)))) 110 | 111 | (spit "test-resources/map-2.edn" (gen-random-map 5 5 6))) -------------------------------------------------------------------------------- /test/xitdb/generated_data_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.generated-data-test 2 | (:require 3 | [clojure.edn :as edn] 4 | [clojure.test :refer :all] 5 | [xitdb.gen-map :as gen-map] 6 | [xitdb.test-utils :as tu :refer [with-db]])) 7 | 8 | (defn uuid [s] 9 | (java.util.UUID/fromString s)) 10 | 11 | (defn load-edn-file [filename] 12 | (edn/read-string {:readers {'inst #'clojure.instant/read-instant-date 13 | 'uuid #'uuid}} 14 | (slurp filename))) 15 | 16 | (deftest ComplexTest1 17 | (with-db [db (tu/test-db)] 18 | (let [m (load-edn-file "test-resources/map-1.edn")] 19 | (reset! db m) 20 | (is (= m @db))))) 21 | 22 | (deftest ComplexTest2 23 | (with-db [db (tu/test-db)] 24 | (let [m (load-edn-file "test-resources/map-2.edn")] 25 | (reset! db m) 26 | (is (= m @db)) 27 | 28 | (testing "Random seeking of keypaths" 29 | (let [paths (gen-map/random-map-paths m 30)] 30 | (doseq [path paths] 31 | (let [keypath (first path) 32 | value (second path)] 33 | (is (= value (get-in @db keypath) (get-in m keypath))))))) 34 | 35 | (testing "Merging two huge maps" 36 | (let [m2 (load-edn-file "test-resources/map-1.edn")] 37 | (swap! db merge m2) 38 | (is (= @db (merge m m2))) 39 | (is (= (count @db) (count (merge m m2))))))))) 40 | 41 | 42 | -------------------------------------------------------------------------------- /test/xitdb/kv_reduce_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.kv-reduce-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | (deftest kv-reduce-test 7 | (with-db [db (tu/test-db)] 8 | (testing "IKVReduce implementation for XITDBHashMap" 9 | (reset! db {:a 1 :b 2 :c 3}) 10 | 11 | ;; Test basic reduce operation 12 | (let [sum (reduce-kv (fn [acc k v] (+ acc v)) 0 @db)] 13 | (is (= 6 sum))) 14 | 15 | ;; Test early termination with reduced 16 | (let [first-key (reduce-kv (fn [acc k v] (reduced k)) nil @db)] 17 | (is (contains? #{:a :b :c} first-key))) 18 | 19 | ;; Test key-value accumulation 20 | (let [kvs (reduce-kv (fn [acc k v] (conj acc [k v])) [] @db)] 21 | (is (= 3 (count kvs))) 22 | (is (every? #(and (keyword? (first %)) (number? (second %))) kvs))) 23 | 24 | ;; Test with nested maps 25 | (reset! db {:outer {:inner {:value 42}}}) 26 | (let [nested-value (reduce-kv (fn [acc k v] 27 | (if (= k :outer) 28 | (reduce-kv (fn [acc2 k2 v2] 29 | (if (= k2 :inner) 30 | (get v2 :value) 31 | acc2)) acc v) 32 | acc)) 33 | nil @db)] 34 | (is (= 42 nested-value)))))) 35 | 36 | (deftest array-kv-reduce-test 37 | (with-db [db (tu/test-db)] 38 | (testing "IKVReduce implementation for XITDBArrayList" 39 | (reset! db [10 20 30 40]) 40 | 41 | ;; Test basic reduce-kv operation (sum of indices * values) 42 | (let [weighted-sum (reduce-kv (fn [acc idx val] (+ acc (* idx val))) 0 @db)] 43 | (is (= 200 weighted-sum))) ; 0*10 + 1*20 + 2*30 + 3*40 = 0 + 20 + 60 + 120 = 200 44 | 45 | ;; Test early termination with reduced 46 | (let [first-val (reduce-kv (fn [acc idx val] (reduced val)) nil @db)] 47 | (is (= 10 first-val))) 48 | 49 | ;; Test index-value accumulation 50 | (let [idx-vals (reduce-kv (fn [acc idx val] (conj acc [idx val])) [] @db)] 51 | (is (= [[0 10] [1 20] [2 30] [3 40]] idx-vals))) 52 | 53 | ;; Test with nested vectors 54 | (reset! db [[1 2] [3 4] [5 6]]) 55 | (let [sum-by-index (reduce-kv (fn [acc idx inner-vec] 56 | (+ acc (reduce-kv (fn [acc2 idx2 val2] 57 | (+ acc2 (* idx idx2 val2))) 58 | 0 inner-vec))) 59 | 0 @db)] 60 | ;; idx=0: 0*0*1 + 0*1*2 = 0 61 | ;; idx=1: 1*0*3 + 1*1*4 = 4 62 | ;; idx=2: 2*0*5 + 2*1*6 = 12 63 | ;; Total: 16 64 | (is (= 16 sum-by-index)))))) -------------------------------------------------------------------------------- /test/xitdb/map_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.map-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | (deftest map-with-complex-keys 7 | (with-db [db (tu/test-db)] 8 | (testing "Composite values as keys" 9 | (reset! db {:foo {{:bar :baz} 42}}) 10 | (is (= {:foo {{:bar :baz} 42}} 11 | (tu/materialize @db))) 12 | 13 | (reset! db {:foo {[1 :bar] 31 14 | [2 :baz] 42}}) 15 | (is (= {:foo {[1 :bar] 31 16 | [2 :baz] 42}} 17 | (tu/materialize @db))) 18 | 19 | (swap! db update :foo dissoc [2 :baz]) 20 | 21 | (is (= {:foo {[1 :bar] 31}} 22 | (tu/materialize @db)))))) -------------------------------------------------------------------------------- /test/xitdb/multi_threaded_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.multi-threaded-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.common :as common] 5 | [xitdb.db :as xdb])) 6 | 7 | (deftest ReturnsHistoryTest 8 | (with-open [db (xdb/xit-db :memory)] 9 | (reset! db {:foo {:bar {:baz {:vroo {:com 4}}}}}) 10 | 11 | (testing "should return new value by default" 12 | (let [retval (swap! db assoc-in [:foo :bar :baz :vroo] 5)] 13 | (is (= {:foo {:bar {:baz {:vroo 5}}}} 14 | (common/materialize retval))))) 15 | 16 | (testing "should return history when binding is set" 17 | (binding [xdb/*return-history?* true] 18 | (let [retval (swap! db assoc-in [:foo :bar] 42)] 19 | (is (= [3 {:foo {:bar {:baz {:vroo 5}}}} {:foo {:bar 42}}] 20 | (common/materialize retval)))))))) 21 | 22 | (defn temp-db-file [] 23 | (let [temp-file (java.io.File/createTempFile "xitdb-" ".db") 24 | temp-path (.getAbsolutePath temp-file)] 25 | (.deleteOnExit temp-file) 26 | temp-path)) 27 | 28 | (deftest multi-threaded-operations-test 29 | (testing "Concurrent reads and visible writes in multithreaded environment" 30 | (with-open [db (xdb/xit-db (temp-db-file))] 31 | ;; Initialize database 32 | (reset! db {:data {:value 0}}) 33 | 34 | ;; Create promises to track completion 35 | (let [read-before-promise (promise) 36 | write-complete-promise (promise) 37 | read-after-promise (promise) 38 | num-readers 5 39 | reader-before-values (atom []) 40 | reader-after-values (atom [])] 41 | 42 | ;; Start multiple reader threads (before write) 43 | (dotimes [i num-readers] 44 | (future 45 | (try 46 | (let [value (get-in @db [:data :value])] 47 | (swap! reader-before-values conj value)) 48 | (when (= (count @reader-before-values) num-readers) 49 | (deliver read-before-promise true)) 50 | (catch Exception e 51 | (println "Reader error:" e))))) 52 | 53 | ;; Wait for initial reads to complete 54 | (deref read-before-promise 1000 false) 55 | 56 | ;; Start writer thread 57 | (future 58 | (try 59 | (swap! db assoc-in [:data :value] 42) 60 | (deliver write-complete-promise true) 61 | (catch Exception e 62 | (println "Writer error:" e)))) 63 | 64 | ;; Wait for write to complete 65 | (deref write-complete-promise 1000 false) 66 | 67 | ;; Start multiple reader threads (after write) 68 | (dotimes [i num-readers] 69 | (future 70 | (try 71 | (let [value (get-in @db [:data :value])] 72 | (swap! reader-after-values conj value)) 73 | (when (= (count @reader-after-values) num-readers) 74 | (deliver read-after-promise true)) 75 | (catch Exception e 76 | (println "Reader error:" e))))) 77 | 78 | ;; Wait for final reads to complete 79 | (deref read-after-promise 1000 false) 80 | 81 | ;; Test assertions 82 | (is (= num-readers (count @reader-before-values)) "All initial readers should complete") 83 | (is (every? #(= 0 %) @reader-before-values) "All initial readers should see original value") 84 | 85 | (is (= num-readers (count @reader-after-values)) "All post-write readers should complete") 86 | (is (every? #(= 42 %) @reader-after-values) "All post-write readers should see updated value") 87 | 88 | (is (= 42 (get-in (common/materialize @db) [:data :value])) "Final value should be updated"))))) 89 | 90 | (deftest concurrent-readers-with-active-writer-test 91 | (testing "Continuous readers with periodic writes" 92 | (with-open [db (xdb/xit-db (temp-db-file))] 93 | ;; Initialize database 94 | (reset! db {:data {:value 0}}) 95 | 96 | (let [num-readers 5 97 | num-writes 10 98 | final-value 999 99 | running (atom true) 100 | reader-seen-values (atom (mapv (fn [_] #{}) (range num-readers))) 101 | writer-done (promise)] 102 | 103 | ;; Start reader threads that continuously read 104 | (dotimes [reader-id num-readers] 105 | (future 106 | (try 107 | (while @running 108 | (let [value (get-in @db [:data :value])] 109 | ;; Record each unique value this reader sees 110 | (swap! reader-seen-values update reader-id conj value) 111 | (Thread/sleep (+ 10 (rand-int 20))))) 112 | (catch Exception e 113 | (println "Reader" reader-id "error:" e))))) 114 | 115 | ;; Start writer thread that makes multiple updates 116 | (future 117 | (try 118 | (dotimes [i num-writes] 119 | (let [new-value (if (= i (dec num-writes)) 120 | final-value ; Final write has special value 121 | (+ 100 i))] ; Other writes have different values 122 | (swap! db assoc-in [:data :value] new-value) 123 | (Thread/sleep 50))) 124 | (deliver writer-done true) 125 | (catch Exception e 126 | (println "Writer error:" e)))) 127 | 128 | ;; Wait for writer to complete 129 | (deref writer-done 5000 false) 130 | 131 | ;; Let readers continue a bit longer to see the final value 132 | (Thread/sleep 200) 133 | 134 | ;; Signal readers to stop 135 | (reset! running false) 136 | 137 | ;; Wait for readers to fully stop 138 | (Thread/sleep 100) 139 | 140 | ;; Test assertions 141 | (let [all-expected-values (conj (set (map #(+ 100 %) (range (dec num-writes)))) 0 final-value)] 142 | ;; Check that all readers saw the final value 143 | (is (every? #(contains? % final-value) @reader-seen-values) 144 | "All readers should eventually see the final value") 145 | 146 | ;; Check that readers collectively saw all intermediate values 147 | (is (= all-expected-values 148 | (reduce clojure.set/union #{} @reader-seen-values)) 149 | "Readers collectively should see all written values")) 150 | 151 | (is (= final-value (get-in @db [:data :value])) 152 | "Database should contain the final value"))))) 153 | 154 | (comment 155 | (let [db (xitdb.db/xit-db "testing.xdb")] 156 | (reset! db {:foo {:bar {:baz {:vroo {:com 4}}}}}) 157 | (future 158 | (try 159 | (println "thread:" (tu/materialize @db)) 160 | (swap! db assoc-in [:foo :bar :baz] 42) 161 | (println "thread after:" (tu/materialize @db)) 162 | (catch Exception e 163 | (println "exception" e)))) 164 | (future 165 | (try 166 | (Thread/sleep 200) 167 | (println "thread 2 :" (tu/materialize @db)) 168 | (catch Exception e 169 | (println "exception" e)))) 170 | (future 171 | (try 172 | (println "thread 3:" (tu/materialize @db)) 173 | (catch Exception e 174 | (println "exception" e)))) 175 | (tu/materialize @db))) -------------------------------------------------------------------------------- /test/xitdb/set_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.set-test 2 | (:require 3 | [clojure.set :as set] 4 | [clojure.test :refer :all] 5 | [xitdb.common :as common] 6 | [xitdb.db :as xdb] 7 | [xitdb.test-utils :as tu :refer [with-db]])) 8 | 9 | (deftest SetTest 10 | (testing "Set works" 11 | (with-db [db (tu/test-db)] 12 | (reset! db #{1 2 3 4 5}) 13 | (swap! db conj 6) 14 | (swap! db disj 2 3) 15 | 16 | (is (= #{1 4 6 5} @db)) 17 | (is (= 4 (count @db))) 18 | (is (tu/db-equal-to-atom? db)))) 19 | 20 | (testing "Basic operations" 21 | (with-db [db (tu/test-db)] 22 | (testing "Creation" 23 | (reset! db #{1 2 3 4 5}) 24 | (is (= #{1 2 3 4 5} @db))) 25 | 26 | (testing "Membership" 27 | (is (= true (contains? @db 2))) 28 | (is (= 3 (get @db 3)))) 29 | 30 | (testing "Adding/Removing" 31 | (swap! db conj 7) 32 | (is (= #{1 2 3 4 5 7} @db)) 33 | (swap! db disj 3) 34 | (is (= #{1 2 4 5 7} @db)) 35 | 36 | (swap! db conj 7 5 1 4 4 4) 37 | (is (= #{1 2 4 5 7} @db))) 38 | 39 | (testing "Emptying" 40 | (swap! db empty) 41 | (is (= #{} @db)) 42 | (is (= 0 (count @db))) 43 | (is (tu/db-equal-to-atom? db)))))) 44 | 45 | (deftest DataTypes 46 | (testing "Supports nested types" 47 | (with-db [db (tu/test-db)] 48 | (reset! db #{1 {:foo :bar} [1 2 3 4] '(7 89)}) 49 | (is (= #{1 '(7 89) [1 2 3 4] {:foo :bar}} 50 | @db)) 51 | 52 | (testing "Adding a set to the set" 53 | (swap! db conj #{1 [3 4] {:new :map}}) 54 | (is (= #{1 '(7 89) [1 2 3 4] #{1 [3 4] {:new :map}} {:foo :bar}} 55 | @db))) 56 | (is (tu/db-equal-to-atom? db))))) 57 | 58 | 59 | (deftest SetOperations 60 | (testing "Union" 61 | (with-db [db (tu/test-db)] 62 | (reset! db #{1 2 3 4 5}) 63 | (swap! db set/union #{4 5 8}) 64 | (is (= #{1 4 3 2 5 8} @db)) 65 | (is (tu/db-equal-to-atom? db)))) 66 | 67 | (testing "Intersection" 68 | (with-db [db (tu/test-db)] 69 | (reset! db #{1 2 3 4 5}) 70 | (swap! db set/intersection #{4 5 8}) 71 | (is (= #{4 5} @db)) 72 | (is (tu/db-equal-to-atom? db)))) 73 | 74 | (testing "Union of two dbs" 75 | (let [db1 (tu/test-memory-db-raw) 76 | db2 (tu/test-memory-db-raw)] 77 | (reset! db1 {:sweets #{:tiramisu :mochi}}) 78 | (reset! db2 {:candies #{:chupa-chups :regular-candy}}) 79 | (swap! db1 update :sweets set/union (:candies @db2)) 80 | (is (= {:sweets #{:regular-candy :chupa-chups, :mochi :tiramisu}} 81 | (tu/materialize @db1)))))) 82 | 83 | (deftest nil-test 84 | (with-db [db (tu/test-db)] 85 | (reset! db {:sweets #{nil :mochi}}) 86 | (testing "Handles nil correctly" 87 | (let [sweets (:sweets @db)] 88 | (is (true? (contains? sweets nil))))))) 89 | 90 | (deftest HashCodeTest 91 | (with-db [db (tu/test-db)] 92 | (reset! db #{:one 1 []}) 93 | (is (= #{:one 1 []} @db)))) 94 | -------------------------------------------------------------------------------- /test/xitdb/test_utils.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.test-utils 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.db :as xdb] 5 | [xitdb.xitdb-types :as types])) 6 | 7 | (def materialize types/materialize) 8 | 9 | (defprotocol DbEqualToAtom 10 | (-db-equal-to-atom? [this])) 11 | 12 | (deftype DBWithAtom [db test-atom] 13 | DbEqualToAtom 14 | (-db-equal-to-atom? [this] 15 | (= (types/materialize @db) @test-atom)) 16 | 17 | java.io.Closeable 18 | (close [this] 19 | (.close db)) 20 | 21 | clojure.lang.IDeref 22 | (deref [_] 23 | (types/materialize 24 | (deref db))) 25 | 26 | clojure.lang.IAtom 27 | (reset [this new-value] 28 | (reset! test-atom new-value) 29 | (reset! db new-value)) 30 | (swap [this f] 31 | (swap! test-atom f) 32 | (swap! db f)) 33 | 34 | (swap [this f a] 35 | (swap! test-atom f a) 36 | (swap! db f a)) 37 | 38 | (swap [this f a1 a2] 39 | (swap! test-atom f a1 a2) 40 | (swap! db f a1 a2)) 41 | 42 | (swap [this f x y args] 43 | (apply swap! (concat [test-atom f x y] args)) 44 | (apply swap! (concat [db f x y] args)))) 45 | 46 | (defn instrumented-db [db] 47 | (let [a (atom nil)] 48 | (->DBWithAtom db a))) 49 | 50 | (defn db-equal-to-atom? 51 | [db] 52 | (if (satisfies? DbEqualToAtom db) 53 | (-db-equal-to-atom? db) 54 | true)) 55 | 56 | (def test-source :memory) ;; :memory or filename 57 | 58 | (defn test-db [] 59 | (instrumented-db (xdb/xit-db test-source))) 60 | 61 | (defn test-memory-db-raw [] 62 | (xdb/xit-db :memory)) 63 | 64 | (defn test-memory-db-a [] 65 | (instrumented-db (atom nil))) 66 | 67 | (defmacro with-db 68 | "Execute body with a database connection, then ensure database is closed. 69 | 70 | Usage: 71 | (with-db [db (xdb/open-db \"some.xdb\")] 72 | ... code using db ...)" 73 | [binding & body] 74 | (let [db-name (first binding) 75 | db-expr (second binding)] 76 | `(let [~db-name ~db-expr] 77 | (try 78 | ~@body 79 | (finally 80 | (.close ~db-name)))))) 81 | 82 | -------------------------------------------------------------------------------- /test/xitdb/transient_test.clj: -------------------------------------------------------------------------------- 1 | (ns xitdb.transient-test 2 | (:require 3 | [clojure.test :refer :all] 4 | [xitdb.test-utils :as tu :refer [with-db]])) 5 | 6 | 7 | (deftest ConjTest 8 | (with-db [db (tu/test-memory-db-raw)] 9 | (reset! db [1 2 3 4 5]) 10 | (swap! db (fn [V] 11 | (let [tv (transient V)] 12 | (conj! tv 42) 13 | (persistent! tv)))) 14 | 15 | (is (= [1 2 3 4 5 42] @db)))) 16 | 17 | 18 | (deftest TransientTest 19 | (with-db [db (tu/test-memory-db-raw)] 20 | (testing "conj!" 21 | (reset! db [1 2 3 4 5]) 22 | (swap! db (fn [V] 23 | (let [tv (transient V)] 24 | (conj! tv 42) 25 | (persistent! tv)))) 26 | 27 | (is (= [1 2 3 4 5 42] @db))) 28 | 29 | (testing "pop!" 30 | (reset! db [1 2 3 4 5]) 31 | (swap! db (fn [V] 32 | (let [tv (transient V)] 33 | (pop! tv) 34 | (pop! tv) 35 | (pop! tv) 36 | (persistent! tv)))) 37 | (is (= [1 2] @db))))) 38 | 39 | (let [tv (transient [1 2 3 4 5])] 40 | (pop! tv) 41 | (pop! tv) 42 | (pop! tv) 43 | (persistent! tv)) --------------------------------------------------------------------------------