├── .gitignore ├── LICENSE ├── README.md ├── build.clj ├── deps.edn ├── src └── net │ └── favila │ └── enhanced_entity_map.clj └── test └── net └── favila └── enhanced_entity_map_test.clj /.gitignore: -------------------------------------------------------------------------------- 1 | pom.xml 2 | pom.xml.asc 3 | *.jar 4 | *.class 5 | /lib/ 6 | /classes/ 7 | /target/ 8 | /checkouts/ 9 | .lein-deps-sum 10 | .lein-repl-history 11 | .lein-plugins/ 12 | .lein-failures 13 | .nrepl-port 14 | .cpcache/ 15 | .idea/ 16 | /*.iml 17 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Francis Avila 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Enhanced Entity Maps 2 | 3 | This provides a Datomic entity-map-like object (as returned from 4 | `datomic.api/entity`) which is compatible with it and has a few additional 5 | abilities: 6 | 7 | * Can support metadata. 8 | * Can assoc arbitrary keys and values on to it. 9 | * Can compute and cache derived attributes via a multimethod. 10 | * Can do database reads using the :aevt index selectively. 11 | 12 | ## Installation 13 | 14 | deps.edn jar 15 | 16 | ``` 17 | net.clojars.favila/enhanced-entity-map {:mvn/version "1.0.6"} 18 | ``` 19 | 20 | project.clj 21 | 22 | ``` 23 | [net.clojars.favila/enhanced-entity-map "1.0.6"] 24 | ``` 25 | 26 | ## Status 27 | 28 | This has not been used in production, but has extensive test coverage to ensure 29 | interface compatibility and behavior parity with normal Datomic entity maps. 30 | Evaluate this carefully before using for anything serious. 31 | 32 | ## Motivations 33 | 34 | You have a large codebase already committed to Datomic entity maps as a 35 | primary means of interacting with Datomic. Refactoring to pull and query 36 | would be a large and dangerous undertaking because you don't know what 37 | code depends on what attributes. This is either because of your own fault 38 | and poor planning, or because you use a framework which makes it difficult 39 | to predict what data you might need and the laziness of entity-maps was an 40 | ergonomic advantage. 41 | 42 | Symptoms you may be experiencing: 43 | 44 | * You have poor performance on map/filter style code over entities 45 | because of EAVT index use. 46 | (Should really be queries, but sometimes refactors are nontrivial.) 47 | * You are struggling against entity map closedness: you need a little bit extra 48 | on an entity sometimes (maybe just some metadata), but have to convert it to 49 | a map before you pass it along. 50 | The conversion removes the connection to the database, so attribute 51 | lookups on downstream code starts to return nil when they shouldn't. 52 | (Should really be plain pull with plain augmented maps, and you know your 53 | code's requirements.) 54 | * Related to closedness above: you can't maintain the "get an attribute" 55 | interface if you refactor your Datomic attributes. For example, you 56 | used to materialize an attribute but now want to compute it as part of 57 | a migration; or you want to provide a common attribute interface. 58 | (Should really be plain pull and maps; or pull with attribute renaming 59 | and/or xform.) 60 | * You compute some expensive value which is a pure function of an entity, 61 | and you end up computing it multiple times because there's no ergonomic way 62 | to keep that value with the entity. 63 | (Should just use [Pathom].) 64 | 65 | [Pathom]: https://pathom3.wsscode.com/ 66 | 67 | ## Abilities 68 | 69 | So what is "enhanced" about enhanced entity-maps vs normal Datomic entity-maps? 70 | 71 | ### Same Interfaces 72 | 73 | First lets talk about what is the same. 74 | 75 | Enhanced entity maps are a drop-in replacement for normal Datomic entity maps. 76 | It implements the same Entity interface and all the same behavior as normal 77 | entity maps, even the quirky stuff. 78 | (See the `basic-entity-map-and-aevt-parity` test.) 79 | 80 | ```clojure 81 | (require '[net.favila.enhanced-entity-map :as eem] 82 | '[datomic.api :as d]) 83 | 84 | ;; How you construct an enhanced entity map 85 | (def enhanced-em (eem/entity db [:my/id "e1"])) 86 | 87 | ;; You can also convert an existing entity-map 88 | (def normal-em (d/entity db [:my/id "e1"])) 89 | (d/touch normal-em) 90 | ;; Conversion will copy the cache of the entity map at the moment you convert it. 91 | (def enhanced-em-clone (eem/as-enhanced-entity normal-em)) 92 | 93 | ;; Enhanced entity maps also support Datomic entity-map functions 94 | (d/touch enhanced-em) 95 | (d/entity-db enhanced-em) 96 | 97 | ;; However normal and enhanced entity maps can never be equal to each other 98 | (= enhanced-em normal-em) 99 | ;; => false 100 | 101 | ;; But they do hash the same 102 | (= (hash enhanced-em) (hash normal-em)) 103 | ;; => true 104 | 105 | ;; However you should be really cautious about equality of even normal Datomic 106 | ;; entity maps--its semantics are a bit surprising. 107 | 108 | ;; Also assoc-ability changes equality and hash semantics; see below! 109 | ``` 110 | 111 | ### Metadata 112 | 113 | Enhanced entity maps support metadata. 114 | 115 | ```clojure 116 | (meta normal-em) 117 | ;; => nil 118 | (with-meta normal-em {:foo :bar}) 119 | ;; class datomic.query.EntityMap cannot be cast to class clojure.lang.IObj 120 | 121 | (meta (with-meta enhanced-em {:foo :bar})) 122 | ;; => {:foo :bar} 123 | ``` 124 | 125 | ### Assoc-ability 126 | 127 | You can assoc arbitrary keyword and value entries onto it, 128 | even keywords that are attribute idents. 129 | Lookups will inspect these values first before hitting the database. 130 | 131 | ```clojure 132 | ;; You can assoc any value you want, even types not supported by Datomic 133 | ;; such as nil. 134 | (def enhanced-em-assoc (assoc enhanced-em :not-a-real-attr [:value])) 135 | (:not-a-real-attr enhanced-em-assoc) 136 | ;; => [:value] 137 | 138 | ;; The return value is still an entity-map connected to the database, 139 | ;; so it can still perform lazy-lookups of values you haven't read yet. 140 | 141 | (:my/id enhanced-em) 142 | ;; => "e1" 143 | 144 | ;; But note assoc doesn't mutate! 145 | (:not-a-real-attr enhanced-em) 146 | ;; => nil 147 | 148 | ;; associng shadows attributes and derived-attributes (discussed below) 149 | (= :shadowed (:my/id (assoc enhanced-em :my/id :shadowed))) 150 | ;; => :shadowed 151 | 152 | ;; Associng also adds value-equality semantics. 153 | ;; An enhanced entity map which has been edited by assoc will never be equal 154 | ;; to or hash the same as an un-assoced map. 155 | 156 | (= enhanced-em (eem/entity db [:my/id "e1"])) 157 | (not= enhanced-em enhanced-em-assoc) 158 | ;; => true 159 | (not= (hash enhanced-em) (hash enhanced-em-assoc)) 160 | ;; => true 161 | 162 | ;; ... even if you assoc an attribute with the *same value it actually has*! 163 | (not= (assoc enhanced-em :my/id "e1") enhanced-em) 164 | ;; => true 165 | (= (:my/id enhanced-em-assoc) (:my/id enhanced-em)) 166 | ;; => true 167 | ``` 168 | 169 | Associng can be handy for: 170 | 171 | * adding novelty to an entity--completely new attributes and values the 172 | database doesn't know about. 173 | * Precaching existing attributes, e.g. from tabular results of a query into 174 | entity maps where you know most downstream code probably won't need 175 | any other values. (This avoids looking the value up from indexes twice.) 176 | * Shadowing or overriding actual attribute values the entity has. 177 | 178 | ## Optional AEVT index use 179 | 180 | Datomic entity maps only use two indexes for their reads: 181 | EAVT for forward attributes and VAET for reverse attributes. 182 | 183 | Normally EAVT is the right choice: if you are reading an attribute from an 184 | entity map, you are most likely to want another attribute from the same 185 | entity map, so EAVT will amortize the IO cost of that next read by using the 186 | same index segment. 187 | 188 | However, some code walks over many entities but only reads a few attributes 189 | from each. For example: 190 | 191 | ```clojure 192 | (->> (:my/high-cardinality-ref some-entity) 193 | (mapcat :my/other-ref) 194 | (map :my/scalar) 195 | (filter my-pred?)) 196 | ``` 197 | 198 | Code like this can get really slow with entity maps because of all the 199 | EAVT access. 200 | This *should* be a datalog query which will prefer AEVT indexes in most 201 | circumstances, but sometimes the refactor is nontrivial. 202 | 203 | Enhanced entity maps can selectively use AEVT indexes instead of EAVT for reads. 204 | This makes entity-maps more efficient for map-and-filter style 205 | work that reads a few attributes from many entities. 206 | 207 | The example above can be rewritten like this: 208 | 209 | ```clojure 210 | (eem/prefer-aevt 211 | (->> (:my/high-cardinality-ref some-entity) 212 | (mapcat :my/other-ref) 213 | (map :my/scalar) 214 | (filter my-pred?) 215 | ;; The "preference" is implemented with a dynamic binding, 216 | ;; so make sure you aren't lazy! 217 | vec)) 218 | ``` 219 | 220 | You can switch in and out of aevt mode at any level: 221 | 222 | ```clojure 223 | (eem/prefer-aevt 224 | (->> (eem/prefer-eavt (:my/high-cardinality-ref some-entity)) 225 | (mapcat :my/other-ref) 226 | (map :my/card1-ref) 227 | (filter #(eem/prefer-eavt (my-pred-that-reads-lots-of-attrs? %))) 228 | ;; `prefer-X` is implemented with a dynamic binding, 229 | ;; so look out for laziness. 230 | vec)) 231 | ``` 232 | 233 | Any values read while in any mode are cached on the entity map like normal, 234 | so you never have to pay to read the same value twice. 235 | 236 | ### Derived attributes 237 | 238 | Very often there's some value which is a pure function of an entity: 239 | for example, it's a normalized, defaulted, filtered or sorted view of an 240 | existing attribute, or it's a combination of two attribute's values. 241 | 242 | If you have such a value, you can now express that value as a "derived" 243 | attribute. No one has to know it isn't a real Datomic attribute! 244 | 245 | Implement the multimethod `eem/entity-map-derived-attribute` for your 246 | fully-qualified attribute. This method accepts the current enhanced entity map 247 | and the attribute you are looking up. 248 | 249 | ```clojure 250 | ;; To do this, implement the multimethod for your attribute: 251 | (defmethod eem/entity-map-derived-attribute :my.derived/ref+-non-enum 252 | [em _attr-kw] 253 | (into #{} (remove keyword?) (:my/ref+ em))) 254 | 255 | (def refer (eem/entity db [:my/id "refer"])) 256 | (:my/ref+ refer) 257 | ;; => #{#:db{:id 17592186045419} #:db{:id 17592186045418} :enum/e3} 258 | 259 | (:my.derived/ref+-non-em refer) 260 | ;; => #{#:db{:id 17592186045419} #:db{:id 17592186045418} :enum/e3} 261 | 262 | ;; The results of derived-attr calls are cached on the entity; 263 | ;; so are any other reads the method may happen to perform on the entity. 264 | 265 | ;; You can read a derived ref from a derived ref: 266 | 267 | (defmethod eem/entity-map-derived-attribute :my.derived/ref+-non-enum-sorted 268 | [em _attr-kw] 269 | (sort-by :my/id (:my.derived/ref+-non-enum em))) 270 | 271 | (:my.derived/ref+-non-enum-sorted refer) 272 | ;; => ({:db/id 17592186045418, :my/id "e1"} {:db/id 17592186045419, :my/id "e2"}) 273 | 274 | ;; Note that reverse refs are not magical like they are for normal attributes, 275 | ;; but you can implement a method with a reverse-ref-looking attribute. 276 | (defmethod eem/entity-map-derived-attribute :my.derived/_fake-reverse-ref 277 | [em _attribute-kw] 278 | #{(:my/real-forward-ref em)}) 279 | ``` 280 | 281 | This multimethod is only called if the attribute does not exist in the 282 | entity map's *database*! 283 | As a consequence, you can't use this feature compute a value for an 284 | existing attribute. 285 | 286 | ## Change Log 287 | 288 | ### v1.0.6 - 2024-05-04 289 | 290 | First release. 291 | 292 | ## Testing and Building 293 | 294 | (This is just to remind myself.) 295 | 296 | ```shell 297 | clojure -Xtest 298 | clojure -T:build clean 299 | clojure -T:build jar # cleans first 300 | 301 | # prints version, date, github compare link for changelog 302 | # Remember to change the compare link to the last release. 303 | clojure -T:build changelog-header 304 | 305 | # Go get a deploy token from https://clojars.org/tokens 306 | # deploy also cleans and builds jar 307 | export CLOJARS_USERNAME=username 308 | export CLOJARS_PASSWORD=token 309 | clojure -T:build deploy 310 | 311 | # If above succeeds, it will print a git tag command of the deployed version. 312 | # Run it and push. 313 | ``` 314 | 315 | ## License 316 | 317 | MIT License 318 | 319 | Copyright © 2024 Francis Avila 320 | -------------------------------------------------------------------------------- /build.clj: -------------------------------------------------------------------------------- 1 | (ns build 2 | (:refer-clojure :exclude [test]) 3 | (:require [clojure.tools.build.api :as b] 4 | [deps-deploy.deps-deploy :as dd]) 5 | (:import (java.time ZoneOffset ZonedDateTime) 6 | (java.time.format DateTimeFormatter))) 7 | 8 | (def lib 'net.clojars.favila/enhanced-entity-map) 9 | (def version (format "1.0.%s" (b/git-count-revs nil))) 10 | (def tag (str "v" version)) 11 | 12 | (def basis (delay (b/create-basis {}))) 13 | 14 | (def github-path "/favila/enhanced-entity-map") 15 | (def github-url (str "https://github.com" github-path)) 16 | 17 | (def jar-opts 18 | (delay 19 | ;; pom opts 20 | {:lib lib 21 | :version version 22 | :basis @basis 23 | :scm {:tag tag 24 | :url github-url 25 | :connection (str "scm:git:" github-url ".git") 26 | :developerConnection (str "scm:git:ssh://git@github.com" github-path ".git")} 27 | :pom-data [[:description "A datomic peer entity map with assoc-ability, index control, and computed attributes."] 28 | [:url github-url] 29 | [:licenses 30 | [:license 31 | [:name "MIT License"] 32 | [:url "https://opensource.org/license/mit"]]] 33 | [:developers 34 | [:developer 35 | [:name "Francis Avila"]]]] 36 | ;; copy-dir and jar opts 37 | :src-dirs ["src"] 38 | :target-dir "target/classes" 39 | :class-dir "target/classes" 40 | :jar-file (format "target/%s-%s.jar" (name lib) version)})) 41 | 42 | (defn clean [_] 43 | (b/delete {:path "target"})) 44 | 45 | (defn jar [_] 46 | (clean nil) 47 | (let [opts @jar-opts] 48 | (b/write-pom opts) 49 | (b/copy-dir opts) 50 | (b/jar opts))) 51 | 52 | (defn- today-YMD [] 53 | (-> (ZonedDateTime/now ZoneOffset/UTC) 54 | (.format DateTimeFormatter/ISO_LOCAL_DATE))) 55 | 56 | (defn changelog-header 57 | [_] 58 | (println (format "### [%s] - %s" tag (today-YMD))) 59 | (println (format "[%s]: %s/compare/%s...%s" tag github-url "FIXME" tag))) 60 | 61 | (defn git-tag 62 | [_] 63 | (println (format "git tag -a '%s' -m '%s'" tag tag))) 64 | 65 | (defn deploy 66 | [_] 67 | (clean nil) 68 | (jar nil) 69 | (let [{:keys [jar-file] :as opts} @jar-opts] 70 | (dd/deploy {:installer :remote 71 | :artifact (b/resolve-path jar-file) 72 | :pom-file (b/pom-path (select-keys opts [:lib :class-dir]))})) 73 | (git-tag nil)) 74 | -------------------------------------------------------------------------------- /deps.edn: -------------------------------------------------------------------------------- 1 | {:paths ["src"] 2 | :deps {org.clojure/clojure {:mvn/version "1.11.2"} 3 | org.clojure/core.cache {:mvn/version "1.0.225"} 4 | com.datomic/peer {:mvn/version "1.0.7075"}} 5 | 6 | :aliases {:build {:deps {io.github.clojure/tools.build {:mvn/version "0.10.3"} 7 | slipset/deps-deploy {:mvn/version "0.2.2"}} 8 | :ns-default build} 9 | 10 | :test {:extra-paths ["test"] 11 | :extra-deps {org.slf4j/slf4j-nop {:mvn/version "1.7.36"} 12 | io.github.cognitect-labs/test-runner 13 | {:git/tag "v0.5.1" :git/sha "dfb30dd"}} 14 | :main-opts ["-m" "cognitect.test-runner"] 15 | :exec-fn cognitect.test-runner.api/test}}} 16 | -------------------------------------------------------------------------------- /src/net/favila/enhanced_entity_map.clj: -------------------------------------------------------------------------------- 1 | (ns net.favila.enhanced-entity-map 2 | "Enhanced Entity Maps. 3 | 4 | Public interface: 5 | 6 | * entity: Make an entity map like d/entity but with special powers: 7 | * Can support metadata. 8 | * Can assoc values on to it. 9 | * Can compute and cache derived attributes. See entity-map-derived-attribute below. 10 | * Can do database reads using the :aevt index selectively. 11 | 12 | * as-enhanced-entity: Convert a d/entity to an enhanced entity map 13 | while preserving its cache. Note that unfortunately entity maps and 14 | enhanced entity maps do not compare equal using clojure.core/= because 15 | entity maps do an explicit class check. 16 | 17 | * prefer-aevt: Use AEVT index instead of EAVT index for enhanced-entity-map 18 | database reads done in its body. 19 | 20 | * prefer-eavt: Same, but prefers EAVT to AEVT. (This is the default, 21 | and what d/entity does.) 22 | 23 | * entity-map-derived-attribute: A multimethod which you can implement to make 24 | derived, computed attributes. The method accepts an enhanced entity map 25 | and an attribute (possibly reverse) and returns a computed value, which 26 | is then cached on the enhanced entity map. 27 | 28 | * entity-map?: Predicate to test for a normal or enhanced entity map." 29 | (:require 30 | [clojure.core.cache] 31 | [clojure.core.cache.wrapped :as cw] 32 | [datomic.api :as d] 33 | [datomic.db] 34 | [datomic.query]) 35 | (:import 36 | (clojure.core.cache BasicCache) 37 | (clojure.lang Associative Counted ILookup IObj IPersistentCollection IPersistentSet MapEntry Seqable Util) 38 | (datomic Datom Entity) 39 | (datomic.db IDbImpl) 40 | (datomic.query EMapImpl EntityMap) 41 | (java.io Writer))) 42 | 43 | (def entity-map-derived-attribute-registry (make-hierarchy)) 44 | 45 | (defn- entity-map-derived-attribute-dispatch 46 | [_enhanced-entity-map attribute-kw] attribute-kw) 47 | 48 | (defmulti entity-map-derived-attribute 49 | "Given an entity map and keyword from entity-map lookup, 50 | return a derived computed attribute value. 51 | 52 | This method is called implicitly by lookups on enhanced-entity-maps 53 | if the attribute wasn't manually assoc-ed and does not exist on the database." 54 | entity-map-derived-attribute-dispatch 55 | :default ::not-implemented 56 | :hierarchy #'entity-map-derived-attribute-registry) 57 | 58 | (defmethod entity-map-derived-attribute ::not-implemented [_eem _kw] 59 | ::not-implemented) 60 | 61 | (defn- make-derived-attr-cache [] 62 | (cw/basic-cache-factory {})) 63 | 64 | (defn- deref-derived-attr-cache [attr-cache] 65 | (.cache ^BasicCache @attr-cache)) 66 | 67 | (defn- find-or-miss-derived-attr [attr-cache em attr] 68 | (let [v (cw/lookup-or-miss attr-cache attr entity-map-derived-attribute em)] 69 | (when-not (identical? ::not-implemented v) 70 | (MapEntry/create attr v)))) 71 | 72 | (defn- not-nil-val [entry] 73 | (when (some? (val entry)) 74 | entry)) 75 | 76 | (declare enhanced-entity*) 77 | 78 | (def ^:dynamic *prefer-index* 79 | "Which index to prefer for enhanced-entity-map reads when there is a choice 80 | between :eavt or :aevt. 81 | 82 | Do not set this directly: use `prefer-aevt` or `prefer-eavt`." 83 | :eavt) 84 | 85 | (defn- iterable-first 86 | [^Iterable it] 87 | (let [i (.iterator it)] 88 | (when (.hasNext i) 89 | (.next i)))) 90 | 91 | (defn- not-cempty [^Counted x] 92 | (when-not (== 0 (.count x)) 93 | x)) 94 | 95 | (defn- enhanced-entity-or-ident 96 | [db eid] 97 | (or (d/ident db eid) 98 | (enhanced-entity* db eid))) 99 | 100 | (defn- lookup-vae [db e attr-rec] 101 | (let [{:keys [id is-component]} attr-rec 102 | xs (d/datoms db :vaet e id)] 103 | ;; Note: reverse refs never produce ident keywords! 104 | ;; This is what Datomic's EntityMap does. 105 | (if is-component 106 | (when-some [^Datom d (iterable-first xs)] 107 | (enhanced-entity* db (.e d))) 108 | (not-cempty 109 | (into #{} 110 | (map #(enhanced-entity* db (.e ^Datom %))) 111 | xs))))) 112 | 113 | (defn- lookup-eav [db e attr-rec] 114 | (let [{:keys [id cardinality value-type]} attr-rec 115 | xs (if (= :aevt *prefer-index*) 116 | (d/datoms db :aevt id e) 117 | (d/datoms db :eavt e id))] 118 | (cond 119 | (= :db.type/ref value-type) 120 | (if (= :db.cardinality/one cardinality) 121 | (when-some [^Datom d (iterable-first xs)] 122 | (enhanced-entity-or-ident db (.v d))) 123 | (not-cempty 124 | (into #{} 125 | (map #(enhanced-entity-or-ident db (.v ^Datom %))) 126 | xs))) 127 | 128 | (= :db.cardinality/one cardinality) 129 | (when-some [^Datom d (iterable-first xs)] 130 | (.v d)) 131 | 132 | :else 133 | (not-cempty (into #{} (map #(.v ^Datom %)) xs))))) 134 | 135 | (defn- find-eav [db e attr] 136 | ;; reverse-lookup? seems to handle special case of an ident whose name 137 | ;; actually starts with _ 138 | (let [rlookup? (datomic.db/reverse-lookup? db attr) 139 | fwd-attr (if rlookup? 140 | (datomic.db/reverse-key attr) 141 | attr)] 142 | (when-some [attr-rec (d/attribute db fwd-attr)] 143 | (if rlookup? 144 | (when (= :db.type/ref (:value-type attr-rec)) 145 | (MapEntry/create attr (lookup-vae db e attr-rec))) 146 | (MapEntry/create attr (lookup-eav db e attr-rec)))))) 147 | 148 | (defn- maybe-touch [x] 149 | (if (instance? Entity x) 150 | (d/touch x) 151 | x)) 152 | 153 | (defn- rfirst 154 | ([] nil) 155 | ([r] r) 156 | ([_ x] (reduced x))) 157 | 158 | (defn- touch-map 159 | [db eid touch-components?] 160 | (into {:db/id eid} 161 | (comp 162 | (partition-by :a) 163 | (map (fn [datoms] 164 | (let [{:keys [ident cardinality value-type is-component]} (d/attribute db (:a (first datoms))) 165 | ref? (= :db.type/ref value-type) 166 | many? (= :db.cardinality/many cardinality) 167 | xform (cond-> (map :v) 168 | 169 | ref? 170 | (comp (map (partial enhanced-entity-or-ident db))) 171 | 172 | (and touch-components? is-component ref?) 173 | (comp (map maybe-touch)))] 174 | [ident 175 | (if many? 176 | (into #{} xform datoms) 177 | (transduce xform rfirst datoms))])))) 178 | (d/datoms db :eavt eid))) 179 | 180 | (defprotocol AsEnhancedEntity 181 | (-as-enhanced-entity [o] "Coerce object to an enhanced entity map.")) 182 | 183 | (deftype EnhancedEntityMap 184 | ;; db is a non-history datomic db 185 | ;; eid is an entity id (nat-int? eid) => true 186 | ;; assoc-cache is a PersistentMap 187 | ;; datomic-attr-cache is a PersistentMap 188 | ;; derived-attr-cache is a clojure.core.cache.wrapped atom over a BasicCache. 189 | ;; (The core.cache wrapping is just for stampede protection, not eviction.) 190 | [db ^long eid assoc-cache ^:unsynchronized-mutable datomic-attr-cache derived-attr-cache metadata] 191 | AsEnhancedEntity 192 | (-as-enhanced-entity [this] this) 193 | 194 | IObj 195 | (meta [_] metadata) 196 | (withMeta [_ m] (EnhancedEntityMap. db eid assoc-cache datomic-attr-cache derived-attr-cache m)) 197 | 198 | Associative 199 | (entryAt [this attr] 200 | ;; Like d/entity we return no-entry if the value is nil. 201 | (if-some [entry (find assoc-cache attr)] 202 | (not-nil-val entry) 203 | (if-some [entry (find datomic-attr-cache attr)] 204 | entry ;; never nil value 205 | (if-some [new-entry (find-eav db eid attr)] 206 | ;; Unlike d/entity, we store nil values of "miss" lookups if 207 | ;; the database recognizes the attr. 208 | ;; This is to avoid hitting the derived-attr system and a possible 209 | ;; method-not-implemented exception. 210 | (do 211 | (set! datomic-attr-cache (conj datomic-attr-cache new-entry)) 212 | (not-nil-val new-entry)) 213 | (when-some [entry (find-or-miss-derived-attr derived-attr-cache this attr)] 214 | (not-nil-val entry)))))) 215 | (containsKey [this attr] 216 | (some? (.entryAt this attr))) 217 | (assoc [_ attr v] 218 | (assert (keyword? attr) "attr must be keyword") 219 | (EnhancedEntityMap. db eid (assoc assoc-cache attr v) datomic-attr-cache derived-attr-cache metadata)) 220 | 221 | EMapImpl 222 | (cache [_] datomic-attr-cache) 223 | 224 | Entity 225 | (get [this attr] (.valAt this attr)) 226 | (touch [this] 227 | (let [cache (touch-map db eid true)] 228 | (set! datomic-attr-cache cache) 229 | this)) 230 | (keySet [this] 231 | (into #{} 232 | (map (comp str key)) 233 | (.seq this))) 234 | (db [_] db) 235 | 236 | ILookup 237 | (valAt [this attr] 238 | (when-some [e (.entryAt this attr)] 239 | (val e))) 240 | (valAt [this attr notfound] 241 | (if-some [e (.entryAt this attr)] 242 | (val e) 243 | notfound)) 244 | 245 | IPersistentCollection 246 | (count [this] (count (.seq this))) 247 | ;; EntityMap does not implement IPersistentCollection.cons 248 | (empty [_] 249 | (enhanced-entity* db eid {})) 250 | (equiv [_ other] 251 | ;; We can never make this equivalent to EntityMaps 252 | ;; because EntityMaps do an explicit class check for EntityMap. 253 | (and 254 | (instance? EnhancedEntityMap other) 255 | (== eid (.eid ^EnhancedEntityMap other)) 256 | (= (.getRawId ^IDbImpl db) 257 | (.getRawId ^IDbImpl (.db ^EnhancedEntityMap other))) 258 | (= assoc-cache (.-assoc-cache ^EnhancedEntityMap other)))) 259 | 260 | Seqable 261 | (seq [_] 262 | (let [datomic-attrs (-> (touch-map db eid false) 263 | ;; Note: seq of d/entity does not include :db/id! 264 | (dissoc :db/id)) 265 | derived-attrs (deref-derived-attr-cache derived-attr-cache) 266 | not-assoc-attr-or-nil (fn [[k v]] (or (nil? v) 267 | (identical? ::not-implemented v) 268 | (contains? assoc-cache k)))] 269 | (concat 270 | assoc-cache 271 | (remove not-assoc-attr-or-nil datomic-attrs) 272 | ;; derived-attrs will never contain keys that are also datomic idents 273 | (remove not-assoc-attr-or-nil derived-attrs)))) 274 | 275 | Object 276 | (hashCode [_] 277 | (let [hc (Util/hashCombine 278 | (hash (.getRawId ^IDbImpl db)) 279 | (hash eid))] 280 | ;; If assoc-cache is empty, hashes the same way as an EntityMap 281 | ;; Otherwise includes the assoc-cache in its hash 282 | (if (== 0 (.count ^Counted assoc-cache)) 283 | hc 284 | (Util/hashCombine hc (hash assoc-cache))))) 285 | (toString [this] 286 | (binding [*print-length* 20 287 | *print-level* 3] 288 | (pr-str this)))) 289 | 290 | (defmethod print-method EnhancedEntityMap [^EnhancedEntityMap eem ^Writer w] 291 | (print-method 292 | (-> (.-assoc-cache eem) 293 | (into (remove (comp nil? val)) (.cache eem)) 294 | (into (remove (comp #(identical? ::not-implemented %) val)) 295 | (deref-derived-attr-cache (.-derived-attr-cache eem)))) 296 | w)) 297 | 298 | (defn- enhanced-entity* 299 | ([db eid] 300 | (->EnhancedEntityMap db eid {} {:db/id eid} 301 | (make-derived-attr-cache) 302 | nil)) 303 | ([db eid attr-cache] 304 | (->EnhancedEntityMap db eid {} 305 | ;; attr-cache should be a copy of another Entity cache 306 | ;; so this is expected to have :db/id already 307 | attr-cache 308 | (make-derived-attr-cache) 309 | nil))) 310 | 311 | (defn entity 312 | "Returns an \"enhanced\" entity map satisfying all the interfaces of 313 | datomic.api/entity, but with additional abilities: 314 | 315 | * It supports metadata. 316 | * It can optionally use AEVT indexes for database reads if it runs within 317 | the `prefer-aevt` macro, which makes it more suitable for map/filter-style 318 | code over many entity maps. 319 | * Arbitrary keywords and values can be assoc-ed on to the entity. 320 | The result of the assoc is a new enhanced-entity with that additional entry 321 | added to its cache. 322 | * It can be extended with \"virtual\" attributes via the 323 | `entity-map-derived-attribute` multimethod. 324 | This multimethod receives this enhanced-entity map and the attribute 325 | (possibly a reverse-attribute with underscore) and returns a computed 326 | value which will be cached on this entity. 327 | Concurrent access to the same derived attributes will only perform work 328 | once (i.e. no stampedes), and derived attributes may access other derived 329 | attributes in their method body. 330 | 331 | Keyword lookups are always performed from these sources in this order. 332 | A nil value is always regarded as \"not found\", but the fact that it was not 333 | found will still be cached. 334 | 335 | 1. Explicitly assoc-ed keys. Nil values will act as \"no entry\" to `find` 336 | and `get`, but still short-circuit this lookup process. 337 | 2. Datomic attributes or reverse-attributes in the entity-map's database. 338 | 3. Only if *not* a datomic attribute or reverse-attribute: the 339 | entity-map-derived-attribute multimethod. 340 | 341 | Note that this means you cannot define `entity-map-derived-attribute` methods 342 | usefully for keywords which are also attribute names in the entity map's 343 | database." 344 | [db eid] 345 | (when (d/is-history db) 346 | ;; Same exception as d/entity throws. 347 | (throw (IllegalStateException. "Can't create entity from history"))) 348 | (when-some [dbid (d/entid db eid)] 349 | (enhanced-entity* db dbid))) 350 | 351 | (extend-protocol AsEnhancedEntity 352 | ;; Datomic's entity maps 353 | EntityMap 354 | (-as-enhanced-entity [entity-map] 355 | (enhanced-entity* (d/entity-db entity-map) 356 | (.eid ^EntityMap entity-map) 357 | ;; Reuse whatever is already cached 358 | (update-vals (.cache ^EntityMap entity-map) 359 | -as-enhanced-entity))) 360 | 361 | ;; What `set?` tests. 362 | ;; These are presumed to be cardinality-many values inside entity maps. 363 | IPersistentSet 364 | (-as-enhanced-entity [xs] 365 | (into #{} (map -as-enhanced-entity) xs)) 366 | 367 | Object 368 | (-as-enhanced-entity [self] self)) 369 | 370 | (defn as-enhanced-entity 371 | "Returns an enhanced-entity from an existing normal d/entity object or 372 | set of such objects. Copies the cache of any d/entity objects found. 373 | 374 | Returns input unchanged if there are no d/entity objects." 375 | [entity-map] 376 | (-as-enhanced-entity entity-map)) 377 | 378 | (defmacro prefer-aevt 379 | "Use the :aevt index for enhanced-entity-map lookups which aren't already 380 | cached. 381 | 382 | This is generally better if performing a read of a few attributes over 383 | many entity maps, e.g. via map or filter." 384 | [& body] 385 | `(binding [*prefer-index* :aevt] 386 | ~@body)) 387 | 388 | (defmacro prefer-eavt 389 | "Use the :eavt index for enhanced-entity-map lookups which aren't already 390 | cached. 391 | 392 | This is generally better if performing a read of many attributes on one or a 393 | small number of entities. 394 | 395 | Note that this is the default behavior and the same behavior as native datomic 396 | entity maps; this macro is only necessary to \"undo\" a containing 397 | `prefer-aevt`." 398 | [& body] 399 | `(binding [*prefer-index* :eavt] 400 | ~@body)) 401 | 402 | (defn entity-map? 403 | "Returns true if x is a normal or enhanced entity map." 404 | [x] 405 | (instance? Entity x)) 406 | 407 | (defn enhanced-entity-map? 408 | "Returns true if x is an enhanced entity map." 409 | [x] 410 | (instance? EnhancedEntityMap x)) 411 | -------------------------------------------------------------------------------- /test/net/favila/enhanced_entity_map_test.clj: -------------------------------------------------------------------------------- 1 | (ns net.favila.enhanced-entity-map-test 2 | (:require 3 | [net.favila.enhanced-entity-map :as eem] 4 | [clojure.test :refer [deftest is testing use-fixtures]] 5 | [datomic.api :as d]) 6 | (:import (clojure.core.cache BasicCache) 7 | (datomic.query EMapImpl) 8 | (net.favila.enhanced_entity_map EnhancedEntityMap))) 9 | 10 | 11 | (def ^:dynamic *env* nil) 12 | 13 | (defn- test-env* [] 14 | (let [uri (str "datomic:mem://enhanced-entity-map-test-" 15 | (System/currentTimeMillis)) 16 | created? (d/create-database uri) 17 | conn (d/connect uri)] 18 | (assert created?) 19 | @(d/transact conn [{:db/ident :my/ref1 20 | :db/valueType :db.type/ref 21 | :db/cardinality :db.cardinality/one} 22 | {:db/ident :my/ref+ 23 | :db/valueType :db.type/ref 24 | :db/cardinality :db.cardinality/many} 25 | {:db/ident :my/id 26 | :db/valueType :db.type/string 27 | :db/cardinality :db.cardinality/one 28 | :db/unique :db.unique/value} 29 | {:db/ident :my/component-ref1 30 | :db/valueType :db.type/ref 31 | :db/cardinality :db.cardinality/one 32 | :db/isComponent true} 33 | {:db/ident :my/component-ref+ 34 | :db/valueType :db.type/ref 35 | :db/cardinality :db.cardinality/many 36 | :db/isComponent true} 37 | {:db/ident :my/counter 38 | :db/valueType :db.type/long 39 | :db/cardinality :db.cardinality/many} 40 | {:db/ident :my/str 41 | :db/valueType :db.type/string 42 | :db/cardinality :db.cardinality/many}]) 43 | @(d/transact conn [{:db/id "e1" 44 | :my/id "e1" 45 | :my/str ["e1-entity"] 46 | :my/counter 1} 47 | {:db/id "e2" 48 | :my/id "e2" 49 | :my/str ["e2-entity"] 50 | :my/counter 2} 51 | {:db/id "e3" 52 | :my/id "e3"} 53 | {:db/id "e3" 54 | :my/id "e3" 55 | :my/str ["e3-entity"] 56 | :my/counter 3 57 | :db/ident :enum/e3} 58 | {:db/id "refer" 59 | :my/id "refer" 60 | :my/ref1 "e1" 61 | :my/ref+ ["e1" "e2" "e3"] 62 | :my/component-ref1 "e1" 63 | :my/component-ref+ ["e1" "e2" "e3"]} 64 | {:db/id "refer-ident" 65 | :my/id "refer-ident" 66 | :my/ref1 "e3" 67 | :my/component-ref1 "e3"} 68 | {:db/id "e4" 69 | :my/id "e4"} 70 | {:db/id "ident-ref" 71 | :my/id "ident-ref" 72 | :db/ident :enum/ident-ref 73 | :my/ref1 "e4" 74 | :my/ref+ ["e4"] 75 | :my/component-ref1 "e4" 76 | :my/component-ref+ ["e4"]} 77 | {:db/id "extra-e4-component-ref" 78 | :my/id "extra-e4-component-ref" 79 | :my/component-ref1 "e4" 80 | :my/component-ref+ ["e4"]}]) 81 | {:uri uri 82 | :conn conn 83 | :db (d/db conn)})) 84 | 85 | (defn- test-env-fixture [f] 86 | (let [{:keys [uri conn] :as env} (test-env*)] 87 | (binding [*env* env] 88 | (f) 89 | (d/release conn) 90 | (d/delete-database uri)))) 91 | 92 | (use-fixtures :once test-env-fixture) 93 | 94 | (defmethod eem/entity-map-derived-attribute :my.derived/str+counter 95 | [em _] 96 | (set (for [s (:my/str em) 97 | c (:my/counter em)] 98 | (str s c)))) 99 | 100 | (defmethod eem/entity-map-derived-attribute :my.derived/str+counter-sorted 101 | [em _] 102 | (sort (:my.derived/str+counter em))) 103 | 104 | (defmethod eem/entity-map-derived-attribute :my.derived/ref+-non-enum 105 | [em _] 106 | (into #{} 107 | (remove keyword?) 108 | (:my/ref+ em))) 109 | 110 | (defmethod eem/entity-map-derived-attribute :my.derived/circular-dep-a 111 | [em _] 112 | (:my.derived/circular-dep-b em)) 113 | 114 | (defmethod eem/entity-map-derived-attribute :my.derived/circular-dep-b 115 | [em _] 116 | (:my.derived/circular-dep-a em)) 117 | 118 | (deftest basic-entity-map-and-aevt-parity 119 | ;; This is testing behavior that should be the *same* 120 | ;; on normal and enhanced entity maps, 121 | ;; even when the enhanced-entity-map is using :aevt 122 | (let [{:keys [db]} *env* 123 | e1-eid (d/entid db [:my/id "e1"]) 124 | refer-lr [:my/id "refer"] 125 | refer-eid (d/entid db refer-lr)] 126 | ;; Checking test setup assumptions 127 | (is (pos-int? e1-eid)) 128 | (is (pos-int? refer-eid)) 129 | (doseq [[entity-ctor index] [[#'d/entity] 130 | [#'eem/entity :eavt] 131 | [#'eem/entity :aevt]]] 132 | (testing (str "Using " entity-ctor 133 | (when index (str " with index" index)) 134 | "\n") 135 | (binding [eem/*prefer-index* index] 136 | (let [refer-em (entity-ctor db refer-lr) 137 | refer-ident-em (entity-ctor db [:my/id "refer-ident"]) 138 | e4-em (entity-ctor db [:my/id "e4"])] 139 | 140 | (is (true? (eem/entity-map? refer-em)) 141 | "Both kinds of entity-maps should be `entity-map?`") 142 | 143 | (is (= refer-eid (:db/id refer-em)) 144 | ":db/id of both kinds of entity-maps should be the same when created via a lookup ref") 145 | 146 | (is (= nil (:my/str refer-em)) 147 | "Lookup of unasserted attribute should have nil value.") 148 | 149 | (is (= nil (::non-existent-attribute-or-multimethod-key refer-em)) 150 | "Lookup of non-existent key should have nil value.") 151 | 152 | (is (= [:db/id refer-eid] (find refer-em :db/id)) 153 | "Both kinds of entity-map should support find on :db/id") 154 | 155 | (testing "Both kinds of entity-map should support find on a datomic attribute that has not been cached." 156 | (let [e1-em (entity-ctor db [:my/id "e1"]) 157 | val-my-id (find e1-em :my/id) 158 | val-my-str (find e1-em :my/str) 159 | val-my-counter (find e1-em :my/counter)] 160 | (is (= [:my/id "e1"] val-my-id)) 161 | (is (= [:my/str #{"e1-entity"}] val-my-str)) 162 | (is (= [:my/counter #{1}] val-my-counter)))) 163 | 164 | (testing "Cached lookup should produce the same (identical) objects" 165 | (is (identical? (:my/id refer-em) (:my/id refer-em))) 166 | (is (identical? (:my/ref1 refer-em) (:my/ref1 refer-em))) 167 | (is (identical? (:my/ref+ refer-em) (:my/ref+ refer-em)))) 168 | 169 | (testing "Cardinality-one forward-refs should return entity-maps if the target entity is not an ident" 170 | (is (eem/entity-map? (:my/ref1 refer-em))) 171 | (is (= e1-eid (-> refer-em :my/ref1 :db/id)))) 172 | 173 | (testing "Cardinality-one forward-refs should return idents if the target entity is an ident" 174 | (is (= :enum/e3 (:my/ref1 refer-ident-em))) 175 | (is (= :enum/e3 (:my/component-ref1 refer-ident-em)))) 176 | 177 | (is (= #{(entity-ctor db [:my/id "e1"]) 178 | (entity-ctor db [:my/id "e2"]) 179 | :enum/e3} 180 | (:my/ref+ refer-em)) 181 | "Cardinality-many forward-refs should return sets of entity-maps or idents.") 182 | 183 | (testing "Non-component reverse-refs should always return entity-map sets, even if the reference has an ident." 184 | (is (set? (:my/_ref1 e4-em))) 185 | (is (== 1 (count (:my/_ref1 e4-em)))) 186 | (is (= :enum/ident-ref (:db/ident (first (:my/_ref1 e4-em)))))) 187 | 188 | (testing "Component reverse-refs should be a single entity-map, even if there are multiple references, even if the referer has an ident." 189 | (is (eem/entity-map? (:my/_component-ref1 e4-em))) 190 | (is (= :enum/ident-ref (:db/ident (:my/_component-ref1 e4-em)))) 191 | 192 | (is (eem/entity-map? (:my/_component-ref+ e4-em))) 193 | (is (= :enum/ident-ref (:db/ident (:my/_component-ref+ e4-em))))))))))) 194 | 195 | (deftest failed-lookup-parity 196 | (let [{:keys [db]} *env* 197 | lr-em (d/entity db [:my/id "does-not-exist"]) 198 | lr-eem (eem/entity db [:my/id "does-not-exist"]) 199 | kw-em (d/entity db :enum/does-not-exist) 200 | kw-eem (eem/entity db :enum/does-not-exist)] 201 | (is (= nil lr-em lr-eem kw-em kw-eem) 202 | "Unresolvable lookup refs and idents should return nil from `entity`."))) 203 | 204 | (deftest ident-lookup-parity 205 | (let [{:keys [db]} *env* 206 | em (d/entity db :enum/e3) 207 | eem (eem/entity db :enum/e3)] 208 | (is (= "e3" (:my/id em) (:my/id eem)) 209 | "Resolvable idents should return entity-map from `entity`."))) 210 | 211 | (deftest history-db-parity 212 | (let [{:keys [db]} *env* 213 | hdb (d/history db)] 214 | (is (thrown? IllegalStateException (d/entity hdb [:my/id "e1"]))) 215 | (is (thrown? IllegalStateException (eem/entity hdb [:my/id "e1"]))))) 216 | 217 | (deftest hash-and-equality 218 | (let [{:keys [db]} *env* 219 | e1-em (d/entity db [:my/id "e1"]) 220 | e1-eem (eem/entity db [:my/id "e1"])] 221 | 222 | (is (= (hash e1-em) 223 | (hash e1-eem)) 224 | "Hash of both kinds of entity-maps should be equal.") 225 | 226 | ;; Cache an attribute read 227 | (:my/counter e1-em) 228 | (:my/counter e1-eem) 229 | 230 | (is (= (hash e1-em) 231 | (hash e1-eem)) 232 | "Hash of both kinds of entity-maps should not be altered by attribute access.") 233 | 234 | (let [e1-eem-assoced (assoc e1-eem :my/counter (:my/counter e1-eem))] 235 | (is (not= (hash e1-em) 236 | (hash e1-eem-assoced)) 237 | "Hash of assoc-ed and not-assoc-ed enhanced entity maps should not be equal, even if the effective value is the same.") 238 | 239 | (is (not= e1-em 240 | e1-eem-assoced) 241 | "Assoc-ed enhanced entity map is not equal to non-assoc-ed eem, even if the effective value is the same.")) 242 | 243 | (is (= e1-em (d/entity db [:my/id "e1"])) 244 | "Normal entity maps are equal if their database-id and entity-id are the same.") 245 | 246 | (is (= e1-eem (eem/entity db [:my/id "e1"])) 247 | "Enhanced entity maps are equal if their database-id and entity-id are the same.") 248 | 249 | (is (not= e1-em e1-eem) 250 | "Normal and enhanced entity-maps are not equal.") 251 | 252 | (is (= e1-eem (eem/entity db [:my/id "e1"])) 253 | "Enhanced entity maps are equal if neither was assoc-ed.") 254 | 255 | (is (= (assoc e1-eem :my/counter #{99} 256 | :not-an-attr "value") 257 | (assoc e1-eem :my/counter #{99} 258 | :not-an-attr "value")) 259 | "Assoc-ed enhanced entity-maps are equal to each-other if their assoc-es and entity-maps are equal."))) 260 | 261 | (defn- entity-cache [^EMapImpl m] 262 | (.cache m)) 263 | 264 | (defn- derived-attr-cache [^EnhancedEntityMap m] 265 | (.cache ^BasicCache @(.derived_attr_cache m))) 266 | 267 | (deftest entity-map-conversion-copies-cache 268 | (let [{:keys [db]} *env* 269 | e1-eid (d/entid db [:my/id "e1"]) 270 | e2-eid (d/entid db [:my/id "e2"]) 271 | e1-em (d/entity db [:my/id "e1"]) 272 | 273 | ;; Perform cached reads 274 | _ (:my/str e1-em) 275 | 276 | e1-eem (eem/as-enhanced-entity e1-em) 277 | 278 | refer-em (d/entity db [:my/id "refer"]) 279 | 280 | ;; Perform cached read on other entities 281 | _ (-> refer-em :my/ref1 :my/counter) 282 | _ (->> refer-em :my/ref+ (mapv :my/id)) 283 | refer-eem (eem/as-enhanced-entity refer-em)] 284 | 285 | (is (= (entity-cache e1-em) 286 | (entity-cache e1-eem) 287 | {:db/id e1-eid 288 | :my/str #{"e1-entity"}}) 289 | "Entity cache is copied shallowly.") 290 | 291 | (testing "deep copy" 292 | (is (= #{:db/id :my/ref1 :my/ref+} 293 | (set (keys (entity-cache refer-em))) 294 | (set (keys (entity-cache refer-eem))))) 295 | 296 | (is (= {:db/id e1-eid :my/counter #{1}} 297 | (entity-cache (:my/ref1 refer-em)) 298 | (entity-cache (:my/ref1 refer-eem)))) 299 | 300 | (is (eem/enhanced-entity-map? (:my/ref1 refer-eem))) 301 | 302 | (is (= #{{:db/id e1-eid :my/id "e1"} 303 | {:db/id e2-eid :my/id "e2"} 304 | :enum/e3} 305 | (into #{} (map (fn [x] 306 | (if (keyword? x) 307 | x 308 | (entity-cache x)))) (:my/ref+ refer-em)) 309 | (into #{} (map (fn [x] 310 | (if (keyword? x) 311 | x 312 | (entity-cache x)))) (:my/ref+ refer-eem)))) 313 | 314 | (is (every? #(or (keyword? %) (eem/enhanced-entity-map? %)) (:my/ref+ refer-eem)))))) 315 | 316 | (deftest enhanced-entity-map-metadata 317 | (let [{:keys [db]} *env* 318 | e1 (eem/entity db [:my/id "e1"]) 319 | e1-m (with-meta e1 {:foo :bar})] 320 | (is (nil? (meta e1))) 321 | (is (= {:foo :bar} (meta e1-m))))) 322 | 323 | (deftest prefer-aevt-caches 324 | (let [{:keys [db]} *env* 325 | refer-em (eem/entity db [:my/id "refer"])] 326 | 327 | (is (= {} (dissoc (entity-cache refer-em) :db/id))) 328 | 329 | (eem/prefer-aevt 330 | (is (= [nil "e1" "e2"] 331 | (sort (map :my/id (:my/ref+ refer-em)))))) 332 | 333 | (is (= #{{:my/id "e1"} 334 | {:my/id "e2"} 335 | :enum/e3} 336 | (into #{} (map (fn [x] 337 | (if (keyword? x) 338 | x 339 | (dissoc (entity-cache x) :db/id)))) 340 | (:my/ref+ refer-em)))))) 341 | 342 | (deftest entity-db-works 343 | (let [{:keys [db]} *env* 344 | refer-em (eem/entity db [:my/id "refer"])] 345 | (is (identical? db (d/entity-db refer-em))) 346 | (is (identical? db (d/entity-db (:my/ref1 refer-em)))))) 347 | 348 | (deftest touch-works 349 | (let [{:keys [db]} *env* 350 | refer-eid (d/entid db [:my/id "refer"]) 351 | refer-em (eem/entity db refer-eid) 352 | _ (d/touch refer-em) 353 | refer-ec (entity-cache refer-em)] 354 | (is (= #{:db/id :my/id :my/ref1 :my/ref+ :my/component-ref1 :my/component-ref+} 355 | (set (keys refer-ec))) 356 | "Touch realizes all forward attributes on the entity.") 357 | 358 | (testing "Touch does not follow non-component refs" 359 | (is (= {} (-> refer-ec :my/ref1 entity-cache (dissoc :db/id)))) 360 | (is (= #{{} :enum/e3} 361 | (into #{} (map (fn [x] 362 | (if (keyword? x) 363 | x 364 | (dissoc (entity-cache x) :db/id)))) 365 | (:my/ref+ refer-em))))) 366 | 367 | (testing "Touch follows component refs" 368 | (is (= {:my/id "e1" 369 | :my/str #{"e1-entity"} 370 | :my/counter #{1}} 371 | (-> refer-ec :my/component-ref1 entity-cache (dissoc :db/id)))) 372 | (is (= #{{:my/id "e1" 373 | :my/str #{"e1-entity"} 374 | :my/counter #{1}} 375 | {:my/id "e2" 376 | :my/str #{"e2-entity"} 377 | :my/counter #{2}} 378 | :enum/e3} 379 | (into #{} (map (fn [x] 380 | (if (keyword? x) 381 | x 382 | (dissoc (entity-cache x) :db/id)))) 383 | (:my/component-ref+ refer-em))) 384 | "Touch follows components recursively.")))) 385 | 386 | (deftest ident-aliasing-works 387 | (let [{:keys [db]} *env* 388 | {db :db-after} (d/with db [[:db/add :my/str :db/ident :my/str-renamed]]) 389 | e1-eem (eem/entity db [:my/id "e1"])] 390 | (is (= #{"e1-entity"} (:my/str e1-eem))) 391 | (is (= {:my/str #{"e1-entity"}} 392 | (dissoc (entity-cache e1-eem) :db/id))) 393 | 394 | (d/touch e1-eem) 395 | 396 | (is (= [:my/str-renamed #{"e1-entity"}] 397 | (find (entity-cache e1-eem) :my/str-renamed)) 398 | "Touch adds the newer ident when an attribute ident value has changed."))) 399 | 400 | (deftest cache-failed-lookups 401 | (let [{:keys [db]} *env* 402 | e1-em (d/entity db [:my/id "e1"]) 403 | e1-eem (eem/entity db [:my/id "e1"]) 404 | refer-em (d/entity db [:my/id "refer"]) 405 | refer-eem (eem/entity db [:my/id "refer"])] 406 | 407 | (testing "Failed lookups on completely unknown keys" 408 | (::non-existent-attribute-or-multimethod-key e1-em) 409 | (::non-existent-attribute-or-multimethod-key e1-eem) 410 | (is (= {} (dissoc (entity-cache e1-em) :db/id)) 411 | "Normal entity maps do not cache in entity-cache.") 412 | (is (= {} (dissoc (entity-cache e1-eem) :db/id)) 413 | "Enhanced entity maps do not cache in entity-cache.") 414 | (is (= {::non-existent-attribute-or-multimethod-key ::eem/not-implemented} 415 | (derived-attr-cache e1-eem)) 416 | "Enhanced entity maps cache unknown keys in the derived-attr-cache")) 417 | 418 | (testing "Failed lookups on known attribute idents" 419 | (:my/ref1 e1-em) 420 | (:my/ref1 e1-eem) 421 | (:db/ident e1-em) 422 | (:db/ident e1-eem) 423 | (is (= {} (dissoc (entity-cache e1-em) :db/id)) 424 | "Normal entity maps do not cache in forward direction.") 425 | (is (= {:my/ref1 nil :db/ident nil} (dissoc (entity-cache e1-eem) :db/id)) 426 | "Enhanced entity maps do cache in forward direction") 427 | 428 | (:my/_ref1 refer-em) 429 | (:my/_ref1 refer-eem) 430 | (is (= nil (:my/_ref1 refer-em))) 431 | (is (= {} (dissoc (entity-cache refer-em) :db/id)) 432 | "Normal entity maps cache in forward direction") 433 | (is (= {:my/_ref1 nil} (dissoc (entity-cache refer-eem) :db/id)) 434 | "Enhanced entity maps do cache in reverse direction")) 435 | 436 | (testing "Derived attribute value caching" 437 | (:my.derived/str+counter-sorted refer-eem) 438 | ;; :my/str is read by the derived attr method 439 | (is (= {:my/str nil} 440 | (dissoc (entity-cache refer-eem) :db/id :my/_ref1)) 441 | "Derived attributes are not stored in the entity-cache.") 442 | 443 | (is (= {:my.derived/str+counter #{} 444 | :my.derived/str+counter-sorted ()} 445 | (derived-attr-cache refer-eem)))))) 446 | 447 | (deftest derived-attributes-work 448 | (let [{:keys [db]} *env* 449 | e1 (eem/entity db [:my/id "e1"]) 450 | refer (eem/entity db [:my/id "refer"])] 451 | 452 | (is (= ["e1-entity1"] 453 | (:my.derived/str+counter-sorted e1)) 454 | "Second order derived attributes work.") 455 | (is (= {:my/str #{"e1-entity"} :my/counter #{1}} 456 | (dissoc (entity-cache e1) :db/id)) 457 | "Intermediate attribute lookups are cached.") 458 | (is (= {:my.derived/str+counter #{"e1-entity1"} 459 | :my.derived/str+counter-sorted ["e1-entity1"]} 460 | (derived-attr-cache e1)) 461 | "Intermediate derived-attribute lookups are cached.") 462 | 463 | (is (= #{"e1" "e2"} 464 | (into #{} 465 | (map :my/id) 466 | (:my.derived/ref+-non-enum refer))) 467 | "Derived attribute implementations can walk to other entities."))) 468 | 469 | (deftest derived-attribute-circular-dep 470 | (let [{:keys [db]} *env* 471 | e1 (eem/entity db [:my/id "e1"])] 472 | (is (thrown? StackOverflowError (:my.derived/circular-dep-a e1)) 473 | "Circular derived attribute dependencies will eventually StackOverflow."))) 474 | 475 | (comment 476 | (def env (test-env*)) 477 | (def db (d/db (:conn env))) 478 | ) 479 | 480 | ;; Below here is code from the readme, here to test the docs actually run. 481 | 482 | (comment 483 | 484 | (require '[net.favila.enhanced-entity-map :as eem] 485 | '[datomic.api :as d]) 486 | 487 | ;; How you construct one 488 | (def enhanced-em (eem/entity db [:my/id "e1"])) 489 | 490 | ;; You can also convert an existing entity-map 491 | (def normal-em (d/entity db [:my/id "e1"])) 492 | (d/touch normal-em) 493 | ;; Conversion will copy the cache of the entity map at the moment you convert it. 494 | (def enhanced-em-clone (eem/as-enhanced-entity normal-em)) 495 | 496 | ;; Enhanced entity maps also support Datomic entity-map functions 497 | (d/touch enhanced-em) 498 | (d/entity-db enhanced-em) 499 | 500 | ;; However they can never be equal to each other 501 | (= enhanced-em normal-em) 502 | ;; => false 503 | 504 | (= (hash enhanced-em) (hash normal-em)) 505 | ;; => true 506 | ) 507 | 508 | (comment 509 | (meta normal-em) 510 | ;; => nil 511 | (with-meta normal-em {:foo :bar}) 512 | ;; class datomic.query.EntityMap cannot be cast to class clojure.lang.IObj 513 | 514 | (meta (with-meta enhanced-em {:foo :bar})) 515 | ;; => {:foo :bar} 516 | 517 | ) 518 | 519 | (comment 520 | ;; You can assoc any value you want, even types not supported by Datomic. 521 | (def enhanced-em-assoc (assoc enhanced-em :not-a-real-attr [:value])) 522 | (:not-a-real-attr enhanced-em-assoc) 523 | ;; => [:value] 524 | 525 | ;; The return value is still an entity-map connected to the database, 526 | ;; so it can still perform lazy-lookups of values you haven't read yet. 527 | 528 | (:my/id enhanced-em) 529 | ;; => "e1" 530 | 531 | ;; But note assoc doesn't mutate! 532 | (:not-a-real-attr enhanced-em) 533 | ;; => nil 534 | 535 | ;; associng shadows attributes and derived-attributes (discussed below) 536 | (= :shadowed (:my/id (assoc enhanced-em :my/id :shadowed))) 537 | ;; => :shadowed 538 | 539 | ;; Associng also adds value-equality semantics. 540 | ;; An enhanced entity map which has been edited by assoc will never be equal 541 | ;; to or hash the same as an un-assoced map. 542 | 543 | (= enhanced-em (eem/entity db [:my/id "e1"])) 544 | (not= enhanced-em enhanced-em-assoc) 545 | ;; => true 546 | (not= (hash enhanced-em) (hash enhanced-em-assoc)) 547 | ;; => true 548 | 549 | ;; EVEN IF you assoc an attribute with the *same value it actually has*: 550 | (not= (assoc enhanced-em :my/id "e1") enhanced-em) 551 | ;; => true 552 | 553 | (= (:my/id enhanced-em-assoc) (:my/id enhanced-em)) 554 | 555 | ) 556 | 557 | (comment 558 | 559 | (defmethod eem/entity-map-derived-attribute :my.derived/ref+-non-enum 560 | [em _attr-kw] 561 | (into #{} (remove keyword?) (:my/ref+ em))) 562 | 563 | (def refer (eem/entity db [:my/id "refer"])) 564 | (:my/ref+ refer) 565 | ;; => #{#:db{:id 17592186045419} #:db{:id 17592186045418} :enum/e3} 566 | 567 | (:my.derived/ref+-non-em refer) 568 | ;; => #{#:db{:id 17592186045419} #:db{:id 17592186045418} :enum/e3} 569 | 570 | ;; The results of derived-attr calls are cached on the entity; 571 | ;; so are any other reads the method may happen to perform on the entity. 572 | 573 | ;; You can read a derived ref from a derived ref: 574 | 575 | (defmethod eem/entity-map-derived-attribute :my.derived/ref+-non-enum-sorted 576 | [em _attr-kw] 577 | (sort-by :my/id (:my.derived/ref+-non-enum em))) 578 | 579 | (:my.derived/ref+-non-enum-sorted refer) 580 | ;; => ({:db/id 17592186045418, :my/id "e1"} {:db/id 17592186045419, :my/id "e2"}) 581 | 582 | ;; Note that reverse refs are not magical like they are for normal attributes, 583 | ;; but you can implement a method with a reverse-ref-looking attribute. 584 | (defmethod eem/entity-map-derived-attribute :my.derived/_fake-reverse-ref 585 | [em _attribute-kw] 586 | #{(:my/real-forward-ref em)}) 587 | 588 | ) 589 | --------------------------------------------------------------------------------