├── README.md ├── README.org ├── cljs-4-graphic-designers.org ├── datascript.org ├── datomic-tutorial.md ├── datomic-tutorial.org ├── om-next-client-server-tutorial.md ├── om-next-client-server-tutorial.org └── writing-specs.org /README.md: -------------------------------------------------------------------------------- 1 |
2 |

Table of Contents

3 |
4 | 8 |
9 |
10 | 11 | 12 | # About Missing Link Tutorials 13 | 14 | I find a lot of technology doesn't have documentation that I can 15 | relate to. The style of Missing Link Tutorials is to be clear, 16 | succint, easy, and to cover concepts and essential practical aspects 17 | of the topic. Without further ado… 18 | 19 | # Tutorials 20 | 21 | - [Beginner Datomic Tutorial](datomic-tutorial.md) 22 | - [Om Next Client/Server Tutorial](om-next-client-server-tutorial.md) 23 | -------------------------------------------------------------------------------- /README.org: -------------------------------------------------------------------------------- 1 | #+TITLE: Missing Link Tutorials 2 | 3 | * About Missing Link Tutorials 4 | 5 | I find a lot of technology doesn't have documentation that I can 6 | relate to. The style of Missing Link Tutorials is to be clear, 7 | succint, easy, and to cover concepts and essential practical aspects 8 | of the topic. Without further ado... 9 | 10 | 11 | * Tutorials 12 | 13 | + [[file:datomic-tutorial.md][Beginner Datomic Tutorial]] 14 | + [[file:om-next-client-server-tutorial.md][Om Next Client/Server Tutorial]] 15 | 16 | 17 | 18 | -------------------------------------------------------------------------------- /cljs-4-graphic-designers.org: -------------------------------------------------------------------------------- 1 | * Overview 2 | 3 | This document will try to bridge the gap between graphic designers and 4 | frontend developers in the context of clojurescript (cljs) 5 | development. Essentially this document is aimed at graphic designers 6 | who are ok with learning some basics of cljs to more 7 | seemlessly iterate with front end cljs devs. 8 | 9 | * The HTML 10 | 11 | In cljs we write html in a format called [[https://github.com/teropa/hiccups][hiccup]]. Lets compare some 12 | standard html with the hiccup format. 13 | 14 | #+BEGIN_SRC html 15 |

Hello

16 | #+END_SRC 17 | 18 | equivalent hiccup: 19 | 20 | #+BEGIN_SRC clojure 21 | [:p "hello"] 22 | #+END_SRC 23 | 24 | Here is a table of equivalents: 25 | 26 | | HTML | HICCUP | 27 | |-----------------------------+--------------------------------| 28 | |

fenton
travers

| [:p "fenton" [:br] "travers] | 29 | | hello | [:a {:href "abc.com"} "hello"] | 30 | |
bye
| [:div {:class "a1"} "bye"] | 31 | 32 | That should get us started. Lets look at the workflow. How could you 33 | as a designer actually get some work done. 34 | 35 | * Test Project 36 | 37 | Clone the following repository: 38 | 39 | : git clone git@github.com:ftravers/cljs_crypto.git 40 | 41 | Checkout the following tag: 42 | 43 | : git checkout basic-cljs-page 44 | 45 | Now run the project, put the following command into the terminal in the 46 | project root: 47 | 48 | : clj -A:fe-dev:fig -b dev -r 49 | 50 | A browser window should pop up that displays this very basic, 51 | rudimentary project. 52 | 53 | * Project Structure 54 | 55 | Lets look at the files in the project: 56 | 57 | #+BEGIN_SRC shell 58 | % tree 59 | . 60 | |-- deps.edn 61 | |-- dev.cljs.edn 62 | |-- LICENSE 63 | |-- README.org 64 | |-- resources 65 | | `-- public 66 | | `-- index.html 67 | `-- src 68 | `-- cljs_crypto 69 | `-- core.cljs 70 | #+END_SRC 71 | 72 | The first file to look at is: ~src/cljs_crypto/core.cljs~. Here are 73 | the contents: 74 | 75 | #+BEGIN_SRC clojure 76 | 1 (ns cljs-crypto.core 77 | 2 (:require 78 | 3 [rum.core :refer [defc mount] :as rum])) 79 | 4 80 | 5 (defc hello-world-component 81 | 6 [] 82 | 7 [:h1 "Hello my friendly, lovely world!"]) 83 | 8 84 | 9 (defn main-page [] 85 | 10 (mount 86 | 11 (hello-world-component) 87 | 12 js/document.body)) 88 | 13 89 | 14 (main-page) 90 | #+END_SRC 91 | 92 | The only line we are going to pay attention to at the moment is 93 | line 7. 94 | 95 | : [:h1 "Hello my friendly, lovely world!"] 96 | 97 | This is the hiccup. Open this file in the atom editor and make a 98 | change, and see that get automatically reflected in web browser. 99 | 100 | * A Clojure Friendly Editor 101 | 102 | Everyone has their favorite editor. I'm going to recommend using the 103 | Atom editor for a few reasons. There are a few handy features that 104 | we'll take advantage of. 105 | 106 | (1) rainbow delimiters: this will color our parens, brackets and curly 107 | braces, so we can visually see opening and closing delimiters easily. 108 | 109 | (2) parinfer: this will prevent us from creating un-balanced 110 | delimiters, i.e. we can't have an open paren ~(~, without a closing 111 | paren ~)~. 112 | 113 | These two things will go a long way into preventing you from getting 114 | frustrated with counting parenthesis. Install [[https://atom.io][atom]], for your 115 | platform. Afterwards add a few plugins to make the editor better for 116 | cljs by running the following command from the terminal: 117 | 118 | #+BEGIN_SRC shell 119 | apm install proto-repl ink parinfer lisp-paredit rainbow-delimiters atom-beautify atom-file-icons hasklig 120 | #+END_SRC 121 | 122 | * Adding in CSS 123 | 124 | First kill the terminal process if it is still running. ~Ctrl-d~ will 125 | kill it. 126 | 127 | To add CSS we use a library called [[https://github.com/noprompt/garden][garden]]. Now lets add some basic 128 | CSS. Check out the tag: ~basic-css~. 129 | 130 | : git checkout basic-css 131 | 132 | Fire up the CLJS process in the terminal again. 133 | 134 | : clj -A:fe-dev:fig -b dev -r 135 | 136 | Open the file: ~src/cljs_crypto/core.cljs~ again. 137 | 138 | #+BEGIN_SRC clojure 139 | 1 (ns cljs-crypto.core 140 | 2 (:require 141 | 3 [garden.core :refer [css]] 142 | 4 [rum.core :refer [defc mount] :as rum])) 143 | 5 144 | 6 (def my-style 145 | 7 [[:.my-1st-class 146 | 8 {:font-size "2em"}]]) 147 | 9 148 | 10 (defc hello-world-component 149 | 11 [] 150 | 12 [:div 151 | 13 [:style (css my-style)] 152 | 14 [:p {:class :my-1st-class} 153 | 15 "Hello lovely world!"]]) 154 | 16 155 | 17 (defn main-page [] 156 | 18 (mount 157 | 19 (hello-world-component) 158 | 20 js/document.body)) 159 | 21 160 | 22 (main-page) 161 | #+END_SRC 162 | 163 | Now look at lines 6-8. 164 | 165 | #+BEGIN_SRC clojure 166 | [[:.my-1st-class 167 | {:font-size "2em"}]] 168 | #+END_SRC 169 | 170 | The equivalent in normal CSS would look like: 171 | 172 | #+BEGIN_SRC css 173 | .my-1st-class { 174 | font-size: 2em; 175 | } 176 | #+END_SRC 177 | 178 | Try changing the font size to "3em", see this automatically reflected 179 | in the webpage. 180 | 181 | ** Lets Add a Second Class 182 | 183 | Checkout the tag: css-2nd-class 184 | 185 | : git checkout css-2nd-class 186 | 187 | The relevant parts of the file are: 188 | 189 | #+BEGIN_SRC clojure 190 | 1 (def my-style 191 | 2 [[:.my-1st-class 192 | 3 {:font-size "3em"}] 193 | 4 [:.my-2nd-class 194 | 5 {:color "red"}]]) 195 | 6 196 | 7 (defc hello-world-component 197 | 8 [] 198 | 9 [:div 199 | 10 [:style (css my-style)] 200 | 11 [:p {:class :my-1st-class} 201 | 12 "Hello lovely world!"] 202 | 13 [:p {:class :my-2nd-class} 203 | 14 "Nice to meet you."]]) 204 | #+END_SRC 205 | 206 | The main difference is on lines 4-5. 207 | 208 | #+BEGIN_SRC clojure 209 | [:.my-2nd-class 210 | {:color "red"}] 211 | #+END_SRC 212 | 213 | Just add each new class between square brackets []. 214 | -------------------------------------------------------------------------------- /datascript.org: -------------------------------------------------------------------------------- 1 | * Background 2 | 3 | If you are unfamiliar with datomic, probably visit that tutorial 4 | first. 5 | 6 | * Setup 7 | 8 | 9 | 10 | * Motivation 11 | 12 | With clojurescript applications, some people keep their app state in a 13 | global atom in the form of a large map. For example: 14 | 15 | #+BEGIN_SRC clojure 16 | (def app-state 17 | {:user-logged-in true 18 | :secure-token "adslkfj23l4jskdlfj42w5j" 19 | :email "joe.blow@gmail.com" 20 | :user-id 1 21 | :friends [{:user-id 2 :name "sally" :status :active} 22 | {:user-id 3 :name "bob"}] 23 | :co-workers [{:user-id 2 :name "sally" :status :active]}) 24 | #+END_SRC 25 | 26 | Why on earth would we ever want to have a small database on the client 27 | side? Seems like overkill right? Well say in our above app, "sally" 28 | signs off and is no longer active. I have to update that in two 29 | places, there is no master record. 30 | 31 | #+BEGIN_SRC clojure 32 | (def app-state 33 | {:user-logged-in true 34 | :secure-token "adslkfj23l4jskdlfj42w5j" 35 | :email "joe.blow@gmail.com" 36 | :user-id 1 37 | :users [{:user-id 2 :name "sally" :active :away}] 38 | :friends [2 3] 39 | :co-workers [2]) 40 | #+END_SRC 41 | 42 | I wanted 43 | to pay "sally" back the 50.50 I owed her. Since I'm storing that in 44 | two locations, I have to make sure I update both locations. 45 | -------------------------------------------------------------------------------- /datomic-tutorial.md: -------------------------------------------------------------------------------- 1 |
2 |

Table of Contents

3 |
4 | 41 |
42 |
43 | 44 | # About Missing Link Tutorials 45 | 46 | I find a lot of technology doesn't have documentation that I can 47 | relate to. The style of Missing Link Tutorials is to be clear, 48 | succint, easy, and to cover concepts and essential practical aspects 49 | of the topic. Without further ado… 50 | 51 | # Datomic Tutorial 52 | 53 | This tutorial was written for Datomic version: 0.9.5561, which was 54 | released on February 13, 2017. 55 | 56 | ## Data Shape 57 | 58 | You can think about the data that is stored in datomic as just a bunch 59 | of maps. Datomic doesn't have tables, like a relational database. 60 | Records (rows) are just maps chucked into a large pile. Datomic calls 61 | these maps Entities. The keys that these maps use are special, as we'll 62 | explain later. 63 | 64 | Below is an explicit visual sample of how you can conceive of a 65 | datomic database: 66 | 67 | [{:db/id 1 68 | :car/make "toyota" 69 | :car/model "tacoma" 70 | :year 2014} 71 | 72 | {:db/id 2 73 | :car/make "BMW" 74 | :car/model "325xi" 75 | :year 2001} 76 | 77 | {:db/id 3 78 | :user/name "ftravers" 79 | :user/age 54 80 | :cars [{:db/id 1} 81 | {:db/id 2}]}] 82 | 83 | So this datomic database has 3 entries (entities/maps/rows). A user, 84 | `ftravers`, who owns 2 cars. 85 | 86 | Every map (entity) will get a `:db/id`. This is what uniquely 87 | identifies that entity to datomic. Datomic `:db/id`'s are actually 88 | very large integers, so the data above is actually a bit fake, but I 89 | keep it simple to communicate the concept. 90 | 91 | As we can see in the above example, the `:cars` field (key) of the 92 | user `ftravers` points (refers/links) to the cars he owns using the 93 | `:db/id` field. The `:db/id` field allows one entity to refer to 94 | another entity, or as in this case, multiple other entities. 95 | 96 | ## Map Fields 97 | 98 | Entities (maps) in datomic, like idiomatic clojure, use keywords for 99 | its keys (fields). 100 | 101 | Looking at all three records (maps/entities), in our sample database 102 | we can see that the collective set of keys (fields) used are: 103 | 104 | :db/id 105 | :car/make 106 | :car/model 107 | :year 108 | :user/name 109 | :user/age 110 | :cars 111 | 112 | Datomic doesn't allow you to just go ahead and pick any old keyword as 113 | a field (or key) to entity maps, like you could in plain old clojure. 114 | Rather we have to specify ahead of time which keywords entities in 115 | datomic are allowed to use. Defining which keys (fields) can be used 116 | by maps (entities) is the process of creating a datomic *schema*. 117 | 118 | In the SQL world, creating a schema means defining table names, column 119 | names and column data types. 120 | 121 | In datomic, we do away with the concept of a table. You could say 122 | datomic ONLY specifies 'columns'. Additionally, these columns have no 123 | relationship to (or grouping with) any other column. Contrast this 124 | with an RDBMS which groups columns with the concept of a table. 125 | 126 | Here a column in SQL is equivalent to a field in datomic. When we 127 | specify a column in SQL we give it a name, and we indicate what it 128 | will hold, a string, integer, etc… 129 | 130 | In datomic we do the same thing, we define fields stating what their 131 | name is and what type of data they hold. In Datomic nomenclature 132 | fields are referred to as **attributes**. 133 | 134 | So this is a bit of a big deal. In RDBMS our columns are stuck 135 | together in a table. In datomic we define a bunch of fields that 136 | don't necessarily have anything to do with one another. We can 137 | randomly use any field we define for any record (map/entity) we want 138 | to store! Remember datomic is just a big pile of maps. 139 | 140 | Again, we can only use fields that have been predefined (the schema), 141 | but other than that, we can create maps with any combinations of those 142 | fields. We'll revist this idea later on. 143 | 144 | One more note, I've used both namespace qualified keyword fields like: 145 | `:user/name` and non-namespace qualified keyword fields like: 146 | `:cars`. I do this just to show that the keywords dont need to be 147 | namespace qualified, but it is best practice to do so. Why you may 148 | ask? One person suggested that they can be easier to refactor since 149 | they are more specific. 150 | 151 | Okay enough concepts, let's see how to define a field. 152 | 153 | ## Basic Schema 154 | 155 | Here we create a field (define an attribute) in datomic. We'll start 156 | with creating just one field. This field will *hold* an email value. 157 | 158 | (def schema 159 | [{:db/doc "A users email." 160 | :db/ident :user/email 161 | :db/valueType :db.type/string 162 | :db/cardinality :db.cardinality/one 163 | :db.install/_attribute :db.part/db}]) 164 | 165 | `:db/ident` is the name of the field. So when we want to use this 166 | field to store data, this is the keyword that you would use. 167 | 168 | `:db/valueType` is the type of data that this field will hold. Here 169 | we use the `string` datatype to store an email string. 170 | 171 | `:db/cardinality` can be either `one` or `many`. Basically should 172 | this field hold a single item or a list of items. 173 | 174 | Those are the important fields to understand conceptually. `:db/doc` 175 | is a documentation string, `:db.install/_attribute` instructs datomic 176 | to treat this data as schema field creation data. 177 | 178 | Before we can start adding schema to a database, we need to create the 179 | database! 180 | 181 | (def db-url "datomic:free://127.0.0.1:4334/omn-dev") 182 | (d/create-database db-url) 183 | 184 | Now we can load this schema definition into the database by 185 | transacting it like so: 186 | 187 | (d/transact (d/connect db-url) schema) 188 | 189 | or written a bit more functionally 190 | 191 | (-> db-url 192 | d/connect 193 | (d/transact schema)) 194 | 195 | ## Testdata 196 | 197 | Now that we've defined a field, let's make use of it by 198 | creating/inserting an entity that makes use of the newly created 199 | field. Remember data inside datomic is just a map, so let's just 200 | create that map: 201 | 202 | (def test-data 203 | [{:user/email "fred.jones@gmail.com"}]) 204 | 205 | Let's transact this data into the DB: 206 | 207 | (-> db-url 208 | d/connect 209 | (d/transact test-data)) 210 | 211 | ## Blow away and recreate DB 212 | 213 | When experimenting with datomic, I like to blow the database away, so 214 | I know I'm starting with a clean slate each time. 215 | 216 | (d/delete-database db-url) 217 | (d/create-database db-url) 218 | (d/transact (d/connect db-url) schema) 219 | (d/transact (d/connect db-url) test-data) 220 | 221 | Here I blow it away, recreate a blank DB, recreate the connection, 222 | transact the schema and testdata. 223 | 224 | Working code can be found under the 225 | 226 | GIT BRANCH: basic-schema-insert 227 | 228 | ## Better Testdata 229 | 230 | Okay a DB with only one record (row/entity/map) in it is pretty 231 | boring. Also a db with only one string column (field) is next to 232 | useless! Let's create a DB with two entities (records/maps) in it. 233 | Also let's create a second field, age, so we can query the database for 234 | people 21 and older! 235 | 236 | The schema: 237 | 238 | (def schema 239 | [{:db/doc "A users email." 240 | :db/ident :user/email 241 | :db/valueType :db.type/string 242 | :db/cardinality :db.cardinality/one 243 | :db.install/_attribute :db.part/db} 244 | 245 | {:db/doc "A users age." 246 | :db/ident :user/age 247 | :db/valueType :db.type/long 248 | :db/cardinality :db.cardinality/one 249 | :db.install/_attribute :db.part/db}]) 250 | 251 | So we've added another field, age, that is type: `:db.type/long`. Now 252 | let's add some actual data: 253 | 254 | (def test-data 255 | [{:user/email "sally.jones@gmail.com" 256 | :user/age 34} 257 | 258 | {:user/email "franklin.rosevelt@gmail.com" 259 | :user/age 14}]) 260 | 261 | GIT BRANCH: better-testdata 262 | 263 | **REMEMBER** to transact this schema and testdata into your cleaned up 264 | DB! Otherwise you'll get an error for trying to add the `:user/email` 265 | field twice. 266 | 267 | # Query the database 268 | 269 | ## Concept 270 | 271 | Now we have seen how to add data to datomic, the interesting part is 272 | the querying of the data. A query might be: "Give me the users who 273 | are over 21", if you are making an app to see who is legal to drink 274 | in the United States, for example. 275 | 276 | In regular RDBMS we compare rows of a table based on the values in a 277 | given column. The SQL query might look like: 278 | 279 | SELECT email FROM users WHERE age > 21 280 | 281 | In datomic we don't have tables, just a bunch of maps. So we don't 282 | have a `FROM` clause. In our case we want to inspect the `:user/age` 283 | field. This means, ANY entity (map), which has the `:user/age` field 284 | will be included in our query. This is a very important idea which we 285 | will revisit later to re-inforce. 286 | 287 | Let's reinforce this concept. When maps use the same field, then any 288 | query on that field will pull in those maps. It **doesn't** matter if 289 | they have **ANY** other fields in common. 290 | 291 | Contrast this with an RDBMS. First of all, all rows that belong to a 292 | given table will by definition have **ALL** the same exact fields. 293 | Second, if you had a column in another table that you'd like to apply 294 | the same query to, well there isn't a reasonable way to do that. 295 | 296 | Often you'll find rows in an RDBMS that have `null` values, because 297 | for whatever reason, for those rows, having a value in that column 298 | doesn't make sense. This sometimes becomes a problem with modeling 299 | data in an RDBMS. If you have objects that have some fields in common 300 | but not other fields, you often have to break this up into multiple 301 | tables, and life gets complex. Like you might have a user table, an 302 | administrator table, a customer table, a person table, etc… This 303 | rigidity of RDBMS, can often make modeling data very counter-intuitive. 304 | 305 | What do we gain by having this restriction? I would argue nothing. 306 | Datomic does away with this needless restriction of tables. Removing 307 | unneccessary restrictions IMO, is always a good thing. 308 | 309 | ## Breaking down a datomic query 310 | 311 | A query takes *datalog* for its first argument and a *database* to 312 | execute that datalog on, as the second argument. Let's just look at 313 | the datalog part first: 314 | 315 | [:find ?e 316 | :where [?e :user/email]] 317 | 318 | Datalog at a minimum has a `:find` part, and a `:where` part. First 319 | we'll examine the where part. 320 | 321 | ## Datalog :where 322 | 323 | The query (`:where`) part selects (narrows down) the records 324 | (entities). This is truly the querying part. So this corresponds to 325 | the `WHERE` clause in SQL. 326 | 327 | The `:find` part, is basically dictates what to show from the found 328 | records. So this naturally corresponds to the `SELECT` part of SQL. 329 | Let's focus on the `:where` part first. 330 | 331 | Where clauses take one or more vector clauses that are of the form: 332 | 333 | [entity field-name field-value] 334 | 335 | or in datomic speak: 336 | 337 | [entity attribute value] 338 | 339 | Working backwards in our example `[?e :user/email]`, it only specifies 340 | the entity and attribute (field) aspects. It doesn't specify a 341 | field-value. What this means, is that the field-value doesn't matter, 342 | we dont care what it is, it can be anything. 343 | 344 | Next we say we want maps (entities) that have the field (attribute): 345 | `:user/email`. 346 | 347 | Finally, the `?e`, means each entity (map) we find, store it in the 348 | variable `?e`, because we are going to use it in the `:find` part of 349 | our datalog. 350 | 351 | In summary this query reads like: "Get us all the entities in the DB 352 | that have the field: `:user/email`. 353 | 354 | ## Datalog :find 355 | 356 | Finally we have the `:find` part of the datalog. The correlates 357 | directly to the `SELECT` aspect of SQL, and it basically indicates 358 | what fields of the found records to return. 359 | 360 | We just say: `:find ?e`, which can be read as: "Just return the entity 361 | itself to me." Datomic, kind of makes a short cut at this point and 362 | actually returns the entity-id instead of the entity itself. We will 363 | show later how to convert an entity-id, which is just an integer, into 364 | a clojure map that better reflects what that entity actually consists 365 | of. 366 | 367 | ## Tangent, getting the database 368 | 369 | This line: 370 | 371 | (-> db-url d/connect d/db) 372 | 373 | Is a tricky way to get the database. It basically reads, starting 374 | with the the database URL, pass that to the function `d/connect` which 375 | returns a database connection, then pass that connection to the 376 | function `d/db` which returns the actual database. The *thread first* 377 | `->` saves us from having to type: 378 | 379 | (d/db (d/connect db-url)) 380 | 381 | which we kind of have to read inside out to make sense. 382 | 383 | ## Back to our query, tangent over! 384 | 385 | Here is the full query, 386 | 387 | (defn query1 [db] 388 | (d/q '[:find ?e 389 | :where 390 | [?e :user/email]] 391 | db)) 392 | 393 | and the result of running it: 394 | 395 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 396 | #{[17592186045418] [17592186045419]} 397 | 398 | GIT BRANCH: simple-first-query 399 | 400 | Hmmm… Okay this is kind of far from what we put in. Below is the 401 | original data we trasacted into the DB: 402 | 403 | (def test-data 404 | [{:user/email "sally.jones@gmail.com" 405 | :user/age 34} 406 | 407 | {:user/email "franklin.rosevelt@gmail.com" 408 | :user/age 14}]) 409 | 410 | The numbers returned by the query are the entity id's (`:db/id`) of 411 | the two records (maps) we transacted into the database. 412 | 413 | We are going to convert these entity ids into familiar clojure maps 414 | using two approaches. The first approach is a bit more instinctive, 415 | and the second approach is more enlightened (elegant). 416 | 417 | Instinctively, I'd look for an API to convert a `:db/id` into the 418 | actual entity that the id represents. So datomic has a function: 419 | `(entity db entity-id)`, which is documented like so: 420 | 421 | "Returns a dynamic map of the entity's attributes for the given id" 422 | 423 | Okay that looks promising. A bit more research on google reveals the 424 | following works: 425 | 426 | datomic-tutorial.core> (map #(seq (d/entity (d/db @db-conn) (first %))) (query1 (-> db-url d/connect d/db))) 427 | (([:user/email "sally.jones@gmail.com"] [:user/age 34]) 428 | ([:user/email "franklin.rosevelt@gmail.com"] [:user/age 14])) 429 | 430 | Okay, that is the instinctual approach to extract the data we are 431 | looking for, but it isn't very elegant. Now let me introduce a more 432 | enlightened approach, **pull syntax**! 433 | 434 | ## Pull Syntax 435 | 436 | Instead of having the find clause look like: 437 | 438 | :find ?e 439 | 440 | we can convert that into pull syntax like so: 441 | 442 | :find (pull ?e [:user/email :user/age]) 443 | 444 | and our output will now look like: 445 | 446 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 447 | [[#:user{:email "sally.jones@gmail.com", :age 34}] 448 | [#:user{:email "franklin.rosevelt@gmail.com", :age 14}]] 449 | 450 | Okay, that looks a lot nicer! 451 | 452 | The way to understand pull syntax is that the first argument is the 453 | entity that you want to apply a pull pattern to. The second part is 454 | the **pull pattern**. 455 | 456 | Let's remind ourselves of the shape of the data in the DB: 457 | 458 | (def test-data 459 | [{:user/email "sally.jones@gmail.com" 460 | :user/age 34} 461 | 462 | {:user/email "franklin.rosevelt@gmail.com" 463 | :user/age 14}]) 464 | 465 | The pull pattern we use is: `[:user/email :user/age]`. Here we 466 | declare the fields from the entity that we want returned to us. Once 467 | again the result of the pull syntax: 468 | 469 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 470 | [[#:user{:email "sally.jones@gmail.com", :age 34}] 471 | [#:user{:email "franklin.rosevelt@gmail.com", :age 14}]] 472 | 473 | Much more user friendly! 474 | 475 | Our query is a little boring, let's make a query that is more 476 | interesting than just "get all entities who have the `:user/email` 477 | field! 478 | 479 | Let's modify this query to only return people who are 21 and over. 480 | Franklin, you aren't allowed to drink! 481 | 482 | To achieve this we use the following TWO where clauses: 483 | 484 | :where 485 | [?e :user/age ?age] 486 | [(>= ?age 21)] 487 | 488 | The first thing to note about this :where query is that it contains 489 | two clauses. Where clauses are implicitly **AND**-ed together. So both 490 | criteria need to be true for a given entity to be included in the 491 | results. 492 | 493 | Let's breakdown the first part of the query: 494 | 495 | [?e :user/age ?age] 496 | 497 | Remember where clauses are in the format: [entity field-name 498 | field-value] or in datomic nomeclature [entity attribute value]. 499 | 500 | The `[?e :user/age ?age]` where clause reads like: "Find all entities 501 | that have the field (attribute) `:user/age`, and stick the entity into 502 | the variable `?e` and stick the value of the attribute `:user/age`, 503 | into the variable `?age`. 504 | 505 | So for each entity that meets this criteria will have the entity 506 | stored in the `?e` variable, and the age in the `?age` variable. Now 507 | we can make use of the age value in the second where clause: 508 | 509 | [(>= ?age 21)] 510 | 511 | Okay this is a special, and super cool variant on normal where 512 | clauses. We can run **ANY** function here that returns a boolean 513 | result. We know the function `>=` is a boolean value returning 514 | function, so its legit. 515 | 516 | Second, for each entity, the users age will be stored in the variable 517 | `?age`, so we can simply pass the value of that variable into the 518 | function to get our bool result! This just says, we want "entities who 519 | have an age >= 21". Great! 520 | 521 | So here is the full new query: 522 | 523 | (defn query1 [db] 524 | (d/q '[:find (pull ?e [:user/email :user/age]) 525 | :where 526 | [?e :user/age ?age] 527 | [(>= ?age 21)]] 528 | db)) 529 | 530 | And now we get the desired result, nicely formatted by our pull 531 | syntax: 532 | 533 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 534 | [[#:user{:email "sally.jones@gmail.com", :age 34}]] 535 | 536 | GIT BRANCH: query-pull-filter 537 | 538 | # Parent Child Data 539 | 540 | Often we have data that owns other data. For example going back to 541 | our slightly modified first example: 542 | 543 | [{:db/id "taco" 544 | :car/make "toyota" 545 | :car/model "tacoma" 546 | :year 2014} 547 | 548 | {:db/id "325" 549 | :car/make "BMW" 550 | :car/model "325xi" 551 | :year 2001} 552 | 553 | {:db/id 3 554 | :user/name "ftravers" 555 | :user/age 54 556 | :cars [{:db/id "taco"} 557 | {:db/id "325"}]}] 558 | 559 | Because the `ftravers` entity map needs to refer to the `toyota` and 560 | `BMW` entity maps, we include a `:db/id` field. You can put in any 561 | string here for your convenience. After transacting into datomic, 562 | they'll get converted to large integers as we've seen before. 563 | 564 | This data says `ftravers`, owns two cars, a `toyota` and a `BMW` 565 | . So how do we model this? First we start with the schema. We'll 566 | need to define the fields: `:car/make`, `:car/model`, `:year`, 567 | `:user/name`, `:user/age`, and `:cars`. 568 | 569 | `:car/make`, `:car/model`, and `:user/name` are all of type `string` 570 | and cardinality one. For `:year` and `:user/age` we can use integers. 571 | `:cars` is the new one. 572 | 573 | The field `:cars` has a cardinality of `many`; also the type that it 574 | will hold is of a type that points to other entities. We'll need a 575 | type that is like a pointer, reference or link. 576 | 577 | Let's look only at the schema for `:cars`. You should be able to piece 578 | together the other fields from previous schema examples, or just look 579 | at the: 580 | 581 | GIT BRANCH: parent-child-modeling 582 | 583 | ## Many Refs Schema 584 | 585 | For the `:cars` field, the schema definition will look like: 586 | 587 | {:db/doc "List of cars a user owns" 588 | :db/ident :cars 589 | :db/valueType :db.type/ref 590 | :db/cardinality :db.cardinality/many 591 | :db.install/_attribute :db.part/db} 592 | 593 | Take special note of the values for `cardinality` and `valueType`. 594 | 595 | We have used a `valueType` of `:db.type/ref`. This is how we point to 596 | (refer/link) to other entities in the DB. This is the critical 597 | difference between a database and regular old clojure data structures 598 | that don't support references. 599 | 600 | The second thing to note is that the `cardinality` is set to `many`. 601 | That means this field will hold a list of values, not just a single 602 | value. 603 | 604 | ## Testdata 605 | 606 | Now let's make some testdata that can be transacted into the DB: 607 | 608 | (def test-data 609 | [{:db/id "taco" 610 | :car/make "toyota" 611 | :car/model "tacoma" 612 | :year 2014} 613 | 614 | {:db/id "325" 615 | :car/make "BMW" 616 | :car/model "325xi" 617 | :year 2001} 618 | 619 | {:db/id 3 620 | :user/name "ftravers" 621 | :user/age 54 622 | :cars [{:db/id "taco"} 623 | {:db/id "325"}]}]) 624 | 625 | GIT BRANCH: parent-child-modeling 626 | 627 | Now that we have some parent/child data in the DB, let's see how to 628 | query and display it nicely. 629 | 630 | ## Querying Parent Child Data 631 | 632 | First we'll find the record we care about with a where clause that 633 | looks like: 634 | 635 | [?e :user/name "ftravers"] 636 | 637 | This reads: "find all the entities that have the `:user/name` 638 | attribute that has as its value `ftravers`". 639 | 640 | Now let's demonstrate how to format the results nicely with a slightly 641 | more advance pull pattern. 642 | 643 | ## Parent Child Pull Syntax 644 | 645 | We have already learned how to extract entity fields with a basic pull 646 | pattern: 647 | 648 | (pull ?e [:user/name :user/age]) 649 | 650 | retrieves the `:user/name` and `:user/age` fields from the found, 651 | `?e`, entity/entities. Again the result of this look like: 652 | 653 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 654 | [[#:user{:name "ftravers", :age 54}]] 655 | 656 | but what we really want is something that looks like: 657 | 658 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 659 | [[{:user/name "ftravers", 660 | :user/age 54, 661 | :cars 662 | [#:car{:make "toyota", :model "tacoma"} 663 | #:car{:make "BMW", :model "325xi"}]}]] 664 | 665 | So we want more than just the simple fields that an entity has, but we 666 | want to follow any references it has to other entities and get values 667 | from those entities. 668 | 669 | To get the above we change the pull pattern to be: 670 | 671 | [:user/name 672 | :user/age 673 | {:cars [:car/make :car/model]}] 674 | 675 | So to get the children, and print out their fields, you start a new 676 | map, whose key is the parent field that points to the child. In our 677 | case `:cars`. Then you start a vector and list the properties of the 678 | child you wish to grab. 679 | 680 | This is an extremely elegant way to extract arbitrary levels of data 681 | from datomic. Just imagine the mess this would look like with SQL. 682 | Maybe here is a stab just for comparison. 683 | 684 | SELECT users.id users.name, users.age, cars.make, cars.model, cars.year 685 | FROM users cars 686 | WHERE users.id == cars.userid AND users.name == "ftravers" 687 | 688 | And this would produce a result like: 689 | 690 | [[1 ftravers 54 "toyota" "tacoma" 2013] 691 | [1 ftravers 54 "BMW" "325xi" 2001]] 692 | 693 | for comparison the equivalent datalog is: 694 | 695 | '[:find (pull ?e 696 | [:user/name 697 | :user/age 698 | {:cars [:car/make :car/model]}]) 699 | :where [?e :user/name "ftravers"]] 700 | 701 | and its result, is nicely normalized: 702 | 703 | [[{:user/name "ftravers", 704 | :user/age 54, 705 | :cars 706 | [#:car{:make "toyota", :model "tacoma"} 707 | #:car{:make "BMW", :model "325xi"}]}]] 708 | 709 | # Deeper Understanding 710 | 711 | ## Fields cross SQL Table boundaries 712 | 713 | So pretend we have two entities like: 714 | 715 | {:user/name "ftravers" 716 | :year 1945} 717 | 718 | {:car/make "BMW 325xi" 719 | :year 2001} 720 | 721 | In datomic we can compare these two seemingly quite different objects 722 | with each other because they share a field: `:year`. So I could write 723 | a query that returns **ALL THINGS** that are older than 35 years old. 724 | As I write this, it is 2017, so a 35 year old thing would be born 725 | (made) in approximately the year: 1982. So the where clause would 726 | look like: 727 | 728 | [?e :year ?year] 729 | [(<= ?year 1982)] 730 | 731 | In RDBMS you normally are only ever comparing things that exist in the 732 | same table. So it'd be awkward to try a similar thing in an RDBMS. 733 | Primarily because they wouldn't have a combined index for fields in 734 | two separate tables. So your performance would die. In datomic each 735 | field has its own index, so a query like the above, would still be 736 | performant. 737 | -------------------------------------------------------------------------------- /datomic-tutorial.org: -------------------------------------------------------------------------------- 1 | * About Missing Link Tutorials 2 | 3 | I find a lot of technology doesn't have documentation that I can 4 | relate to. The style of Missing Link Tutorials is to be clear, 5 | succint, easy, and to cover concepts and essential practical aspects 6 | of the topic. Without further ado... 7 | 8 | * Datomic Tutorial 9 | 10 | This tutorial was written for Datomic version: 0.9.5561, which was 11 | released on February 13, 2017. 12 | 13 | ** Data Shape 14 | 15 | You can think about the data that is stored in datomic as just a bunch 16 | of maps. Datomic doesn't have tables, like a relational database. 17 | Records (rows) are just maps chucked into a large pile. Datomic calls 18 | these maps Entities. The keys that these maps use are special, as we'll 19 | explain later. 20 | 21 | Below is an explicit visual sample of how you can conceive of a 22 | datomic database: 23 | 24 | #+BEGIN_SRC clojure 25 | [{:db/id 1 26 | :car/make "toyota" 27 | :car/model "tacoma" 28 | :year 2014} 29 | 30 | {:db/id 2 31 | :car/make "BMW" 32 | :car/model "325xi" 33 | :year 2001} 34 | 35 | {:db/id 3 36 | :user/name "ftravers" 37 | :user/age 54 38 | :cars [{:db/id 1} 39 | {:db/id 2}]}] 40 | #+END_SRC 41 | 42 | So this datomic database has 3 entries (entities/maps/rows). A user, 43 | ~ftravers~, who owns 2 cars. 44 | 45 | Every map (entity) will get a ~:db/id~. This is what uniquely 46 | identifies that entity to datomic. Datomic ~:db/id~'s are actually 47 | very large integers, so the data above is actually a bit fake, but I 48 | keep it simple to communicate the concept. 49 | 50 | As we can see in the above example, the ~:cars~ field (key) of the 51 | user ~ftravers~ points (refers/links) to the cars he owns using the 52 | ~:db/id~ field. The ~:db/id~ field allows one entity to refer to 53 | another entity, or as in this case, multiple other entities. 54 | 55 | ** Map Fields 56 | 57 | Entities (maps) in datomic, like idiomatic clojure, use keywords for 58 | its keys (fields). 59 | 60 | Looking at all three records (maps/entities), in our sample database 61 | we can see that the collective set of keys (fields) used are: 62 | 63 | #+BEGIN_SRC clojure 64 | :db/id 65 | :car/make 66 | :car/model 67 | :year 68 | :user/name 69 | :user/age 70 | :cars 71 | #+END_SRC 72 | 73 | Datomic doesn't allow you to just go ahead and pick any old keyword as 74 | a field (or key) to entity maps, like you could in plain old clojure. 75 | Rather we have to specify ahead of time which keywords entities in 76 | datomic are allowed to use. Defining which keys (fields) can be used 77 | by maps (entities) is the process of creating a datomic /schema/. 78 | 79 | In the SQL world, creating a schema means defining table names, column 80 | names and column data types. 81 | 82 | In datomic, we do away with the concept of a table. You could say 83 | datomic ONLY specifies 'columns'. Additionally, these columns have no 84 | relationship to (or grouping with) any other column. Contrast this 85 | with an RDBMS which groups columns with the concept of a table. 86 | 87 | Here a column in SQL is equivalent to a field in datomic. When we 88 | specify a column in SQL we give it a name, and we indicate what it 89 | will hold, a string, integer, etc... 90 | 91 | In datomic we do the same thing, we define fields stating what their 92 | name is and what type of data they hold. In Datomic nomenclature 93 | fields are referred to as *attributes*. 94 | 95 | So this is a bit of a big deal. In RDBMS our columns are stuck 96 | together in a table. In datomic we define a bunch of fields that 97 | don't necessarily have anything to do with one another. We can 98 | randomly use any field we define for any record (map/entity) we want 99 | to store! Remember datomic is just a big pile of maps. 100 | 101 | Again, we can only use fields that have been predefined (the schema), 102 | but other than that, we can create maps with any combinations of those 103 | fields. We'll revist this idea later on. 104 | 105 | One more note, I've used both namespace qualified keyword fields like: 106 | ~:user/name~ and non-namespace qualified keyword fields like: 107 | ~:cars~. I do this just to show that the keywords dont need to be 108 | namespace qualified, but it is best practice to do so. Why you may 109 | ask? One person suggested that they can be easier to refactor since 110 | they are more specific. 111 | 112 | Okay enough concepts, let's see how to define a field. 113 | 114 | ** Basic Schema 115 | 116 | Here we create a field (define an attribute) in datomic. We'll start 117 | with creating just one field. This field will /hold/ an email value. 118 | 119 | #+BEGIN_SRC clojure 120 | (def schema 121 | [{:db/doc "A users email." 122 | :db/ident :user/email 123 | :db/valueType :db.type/string 124 | :db/cardinality :db.cardinality/one 125 | :db.install/_attribute :db.part/db}]) 126 | #+END_SRC 127 | 128 | ~:db/ident~ is the name of the field. So when we want to use this 129 | field to store data, this is the keyword that you would use. 130 | 131 | ~:db/valueType~ is the type of data that this field will hold. Here 132 | we use the ~string~ datatype to store an email string. 133 | 134 | ~:db/cardinality~ can be either ~one~ or ~many~. Basically should 135 | this field hold a single item or a list of items. 136 | 137 | Those are the important fields to understand conceptually. ~:db/doc~ 138 | is a documentation string, ~:db.install/_attribute~ instructs datomic 139 | to treat this data as schema field creation data. 140 | 141 | Before we can start adding schema to a database, we need to create the 142 | database! 143 | 144 | #+BEGIN_SRC clojure 145 | (def db-url "datomic:free://127.0.0.1:4334/omn-dev") 146 | (d/create-database db-url) 147 | #+END_SRC 148 | 149 | Now we can load this schema definition into the database by 150 | transacting it like so: 151 | 152 | #+BEGIN_SRC clojure 153 | (d/transact (d/connect db-url) schema) 154 | #+END_SRC 155 | 156 | or written a bit more functionally 157 | 158 | #+BEGIN_SRC clojure 159 | (-> db-url 160 | d/connect 161 | (d/transact schema)) 162 | #+END_SRC 163 | 164 | ** Testdata 165 | 166 | Now that we've defined a field, let's make use of it by 167 | creating/inserting an entity that makes use of the newly created 168 | field. Remember data inside datomic is just a map, so let's just 169 | create that map: 170 | 171 | #+BEGIN_SRC clojure 172 | (def test-data 173 | [{:user/email "fred.jones@gmail.com"}]) 174 | #+END_SRC 175 | 176 | Let's transact this data into the DB: 177 | 178 | #+BEGIN_SRC clojure 179 | (-> db-url 180 | d/connect 181 | (d/transact test-data)) 182 | #+END_SRC 183 | 184 | ** Blow away and recreate DB 185 | 186 | When experimenting with datomic, I like to blow the database away, so 187 | I know I'm starting with a clean slate each time. 188 | 189 | #+BEGIN_SRC clojure 190 | (d/delete-database db-url) 191 | (d/create-database db-url) 192 | (d/transact (d/connect db-url) schema) 193 | (d/transact (d/connect db-url) test-data) 194 | #+END_SRC 195 | 196 | Here I blow it away, recreate a blank DB, recreate the connection, 197 | transact the schema and testdata. 198 | 199 | Working code can be found under the 200 | 201 | GIT BRANCH: basic-schema-insert 202 | 203 | ** Better Testdata 204 | 205 | Okay a DB with only one record (row/entity/map) in it is pretty 206 | boring. Also a db with only one string column (field) is next to 207 | useless! Let's create a DB with two entities (records/maps) in it. 208 | Also let's create a second field, age, so we can query the database for 209 | people 21 and older! 210 | 211 | The schema: 212 | 213 | #+BEGIN_SRC clojure 214 | (def schema 215 | [{:db/doc "A users email." 216 | :db/ident :user/email 217 | :db/valueType :db.type/string 218 | :db/cardinality :db.cardinality/one 219 | :db.install/_attribute :db.part/db} 220 | 221 | {:db/doc "A users age." 222 | :db/ident :user/age 223 | :db/valueType :db.type/long 224 | :db/cardinality :db.cardinality/one 225 | :db.install/_attribute :db.part/db}]) 226 | #+END_SRC 227 | 228 | So we've added another field, age, that is type: ~:db.type/long~. Now 229 | let's add some actual data: 230 | 231 | #+BEGIN_SRC clojure 232 | (def test-data 233 | [{:user/email "sally.jones@gmail.com" 234 | :user/age 34} 235 | 236 | {:user/email "franklin.rosevelt@gmail.com" 237 | :user/age 14}]) 238 | #+END_SRC 239 | 240 | GIT BRANCH: better-testdata 241 | 242 | *REMEMBER* to transact this schema and testdata into your cleaned up 243 | DB! Otherwise you'll get an error for trying to add the ~:user/email~ 244 | field twice. 245 | 246 | * Query the database 247 | 248 | ** Concept 249 | 250 | Now we have seen how to add data to datomic, the interesting part is 251 | the querying of the data. A query might be: "Give me the users who 252 | are over 21", if you are making an app to see who is legal to drink 253 | in the United States, for example. 254 | 255 | In regular RDBMS we compare rows of a table based on the values in a 256 | given column. The SQL query might look like: 257 | 258 | #+BEGIN_SRC SQL 259 | SELECT email FROM users WHERE age > 21 260 | #+END_SRC 261 | 262 | In datomic we don't have tables, just a bunch of maps. So we don't 263 | have a ~FROM~ clause. In our case we want to inspect the ~:user/age~ 264 | field. This means, ANY entity (map), which has the ~:user/age~ field 265 | will be included in our query. This is a very important idea which we 266 | will revisit later to re-inforce. 267 | 268 | Let's reinforce this concept. When maps use the same field, then any 269 | query on that field will pull in those maps. It *doesn't* matter if 270 | they have *ANY* other fields in common. 271 | 272 | Contrast this with an RDBMS. First of all, all rows that belong to a 273 | given table will by definition have *ALL* the same exact fields. 274 | Second, if you had a column in another table that you'd like to apply 275 | the same query to, well there isn't a reasonable way to do that. 276 | 277 | Often you'll find rows in an RDBMS that have ~null~ values, because 278 | for whatever reason, for those rows, having a value in that column 279 | doesn't make sense. This sometimes becomes a problem with modeling 280 | data in an RDBMS. If you have objects that have some fields in common 281 | but not other fields, you often have to break this up into multiple 282 | tables, and life gets complex. Like you might have a user table, an 283 | administrator table, a customer table, a person table, etc... This 284 | rigidity of RDBMS, can often make modeling data very counter-intuitive. 285 | 286 | What do we gain by having this restriction? I would argue nothing. 287 | Datomic does away with this needless restriction of tables. Removing 288 | unneccessary restrictions IMO, is always a good thing. 289 | 290 | ** Breaking down a datomic query 291 | 292 | A query takes /datalog/ for its first argument and a /database/ to 293 | execute that datalog on, as the second argument. Let's just look at 294 | the datalog part first: 295 | 296 | #+BEGIN_SRC clojure 297 | [:find ?e 298 | :where [?e :user/email]] 299 | #+END_SRC 300 | 301 | Datalog at a minimum has a ~:find~ part, and a ~:where~ part. First 302 | we'll examine the where part. 303 | 304 | ** Datalog :where 305 | 306 | The query (~:where~) part selects (narrows down) the records 307 | (entities). This is truly the querying part. So this corresponds to 308 | the ~WHERE~ clause in SQL. 309 | 310 | The ~:find~ part, is basically dictates what to show from the found 311 | records. So this naturally corresponds to the ~SELECT~ part of SQL. 312 | Let's focus on the ~:where~ part first. 313 | 314 | Where clauses take one or more vector clauses that are of the form: 315 | 316 | #+BEGIN_SRC clojure 317 | [entity field-name field-value] 318 | #+END_SRC 319 | 320 | or in datomic speak: 321 | 322 | #+BEGIN_SRC clojure 323 | [entity attribute value] 324 | #+END_SRC 325 | 326 | Working backwards in our example ~[?e :user/email]~, it only specifies 327 | the entity and attribute (field) aspects. It doesn't specify a 328 | field-value. What this means, is that the field-value doesn't matter, 329 | we dont care what it is, it can be anything. 330 | 331 | Next we say we want maps (entities) that have the field (attribute): 332 | ~:user/email~. 333 | 334 | Finally, the ~?e~, means each entity (map) we find, store it in the 335 | variable ~?e~, because we are going to use it in the ~:find~ part of 336 | our datalog. 337 | 338 | In summary this query reads like: "Get us all the entities in the DB 339 | that have the field: ~:user/email~. 340 | 341 | ** Datalog :find 342 | 343 | Finally we have the ~:find~ part of the datalog. The correlates 344 | directly to the ~SELECT~ aspect of SQL, and it basically indicates 345 | what fields of the found records to return. 346 | 347 | We just say: ~:find ?e~, which can be read as: "Just return the entity 348 | itself to me." Datomic, kind of makes a short cut at this point and 349 | actually returns the entity-id instead of the entity itself. We will 350 | show later how to convert an entity-id, which is just an integer, into 351 | a clojure map that better reflects what that entity actually consists 352 | of. 353 | 354 | ** Tangent, getting the database 355 | 356 | This line: 357 | 358 | #+BEGIN_SRC clojure 359 | (-> db-url d/connect d/db) 360 | #+END_SRC 361 | 362 | Is a tricky way to get the database. It basically reads, starting 363 | with the the database URL, pass that to the function ~d/connect~ which 364 | returns a database connection, then pass that connection to the 365 | function ~d/db~ which returns the actual database. The /thread first/ 366 | ~->~ saves us from having to type: 367 | 368 | #+BEGIN_SRC clojure 369 | (d/db (d/connect db-url)) 370 | #+END_SRC 371 | 372 | which we kind of have to read inside out to make sense. 373 | 374 | ** Back to our query, tangent over! 375 | 376 | Here is the full query, 377 | 378 | #+BEGIN_SRC clojure 379 | (defn query1 [db] 380 | (d/q '[:find ?e 381 | :where 382 | [?e :user/email]] 383 | db)) 384 | #+END_SRC 385 | 386 | and the result of running it: 387 | 388 | #+BEGIN_SRC clojure 389 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 390 | #{[17592186045418] [17592186045419]} 391 | #+END_SRC 392 | 393 | GIT BRANCH: simple-first-query 394 | 395 | Hmmm... Okay this is kind of far from what we put in. Below is the 396 | original data we trasacted into the DB: 397 | 398 | #+BEGIN_SRC clojure 399 | (def test-data 400 | [{:user/email "sally.jones@gmail.com" 401 | :user/age 34} 402 | 403 | {:user/email "franklin.rosevelt@gmail.com" 404 | :user/age 14}]) 405 | #+END_SRC 406 | 407 | The numbers returned by the query are the entity id's (~:db/id~) of 408 | the two records (maps) we transacted into the database. 409 | 410 | We are going to convert these entity ids into familiar clojure maps 411 | using two approaches. The first approach is a bit more instinctive, 412 | and the second approach is more enlightened (elegant). 413 | 414 | Instinctively, I'd look for an API to convert a ~:db/id~ into the 415 | actual entity that the id represents. So datomic has a function: 416 | ~(entity db entity-id)~, which is documented like so: 417 | 418 | "Returns a dynamic map of the entity's attributes for the given id" 419 | 420 | Okay that looks promising. A bit more research on google reveals the 421 | following works: 422 | 423 | #+BEGIN_SRC clojure 424 | datomic-tutorial.core> (map #(seq (d/entity (d/db @db-conn) (first %))) (query1 (-> db-url d/connect d/db))) 425 | (([:user/email "sally.jones@gmail.com"] [:user/age 34]) 426 | ([:user/email "franklin.rosevelt@gmail.com"] [:user/age 14])) 427 | #+END_SRC 428 | 429 | Okay, that is the instinctual approach to extract the data we are 430 | looking for, but it isn't very elegant. Now let me introduce a more 431 | enlightened approach, *pull syntax*! 432 | 433 | ** Pull Syntax 434 | 435 | Instead of having the find clause look like: 436 | 437 | #+BEGIN_SRC clojure 438 | :find ?e 439 | #+END_SRC 440 | 441 | we can convert that into pull syntax like so: 442 | 443 | #+BEGIN_SRC clojure 444 | :find (pull ?e [:user/email :user/age]) 445 | #+END_SRC 446 | 447 | and our output will now look like: 448 | 449 | #+BEGIN_SRC clojure 450 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 451 | [[#:user{:email "sally.jones@gmail.com", :age 34}] 452 | [#:user{:email "franklin.rosevelt@gmail.com", :age 14}]] 453 | #+END_SRC 454 | 455 | Okay, that looks a lot nicer! 456 | 457 | The way to understand pull syntax is that the first argument is the 458 | entity that you want to apply a pull pattern to. The second part is 459 | the *pull pattern*. 460 | 461 | Let's remind ourselves of the shape of the data in the DB: 462 | 463 | #+BEGIN_SRC clojure 464 | (def test-data 465 | [{:user/email "sally.jones@gmail.com" 466 | :user/age 34} 467 | 468 | {:user/email "franklin.rosevelt@gmail.com" 469 | :user/age 14}]) 470 | #+END_SRC 471 | 472 | The pull pattern we use is: ~[:user/email :user/age]~. Here we 473 | declare the fields from the entity that we want returned to us. Once 474 | again the result of the pull syntax: 475 | 476 | #+BEGIN_SRC clojure 477 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 478 | [[#:user{:email "sally.jones@gmail.com", :age 34}] 479 | [#:user{:email "franklin.rosevelt@gmail.com", :age 14}]] 480 | #+END_SRC 481 | 482 | Much more user friendly! 483 | 484 | Our query is a little boring, let's make a query that is more 485 | interesting than just "get all entities who have the ~:user/email~ 486 | field! 487 | 488 | Let's modify this query to only return people who are 21 and over. 489 | Franklin, you aren't allowed to drink! 490 | 491 | To achieve this we use the following TWO where clauses: 492 | 493 | #+BEGIN_SRC clojure 494 | :where 495 | [?e :user/age ?age] 496 | [(>= ?age 21)] 497 | #+END_SRC 498 | 499 | The first thing to note about this :where query is that it contains 500 | two clauses. Where clauses are implicitly *AND*-ed together. So both 501 | criteria need to be true for a given entity to be included in the 502 | results. 503 | 504 | Let's breakdown the first part of the query: 505 | 506 | #+BEGIN_SRC clojure 507 | [?e :user/age ?age] 508 | #+END_SRC 509 | 510 | Remember where clauses are in the format: [entity field-name 511 | field-value] or in datomic nomeclature [entity attribute value]. 512 | 513 | The ~[?e :user/age ?age]~ where clause reads like: "Find all entities 514 | that have the field (attribute) ~:user/age~, and stick the entity into 515 | the variable ~?e~ and stick the value of the attribute ~:user/age~, 516 | into the variable ~?age~. 517 | 518 | So for each entity that meets this criteria will have the entity 519 | stored in the ~?e~ variable, and the age in the ~?age~ variable. Now 520 | we can make use of the age value in the second where clause: 521 | 522 | #+BEGIN_SRC clojure 523 | [(>= ?age 21)] 524 | #+END_SRC 525 | 526 | Okay this is a special, and super cool variant on normal where 527 | clauses. We can run *ANY* function here that returns a boolean 528 | result. We know the function ~>=~ is a boolean value returning 529 | function, so its legit. 530 | 531 | Second, for each entity, the users age will be stored in the variable 532 | ~?age~, so we can simply pass the value of that variable into the 533 | function to get our bool result! This just says, we want "entities who 534 | have an age >= 21". Great! 535 | 536 | So here is the full new query: 537 | 538 | #+BEGIN_SRC clojure 539 | (defn query1 [db] 540 | (d/q '[:find (pull ?e [:user/email :user/age]) 541 | :where 542 | [?e :user/age ?age] 543 | [(>= ?age 21)]] 544 | db)) 545 | #+END_SRC 546 | 547 | And now we get the desired result, nicely formatted by our pull 548 | syntax: 549 | 550 | #+BEGIN_SRC clojure 551 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 552 | [[#:user{:email "sally.jones@gmail.com", :age 34}]] 553 | #+END_SRC 554 | 555 | GIT BRANCH: query-pull-filter 556 | 557 | * Parent Child Data 558 | 559 | Often we have data that owns other data. For example going back to 560 | our slightly modified first example: 561 | 562 | #+BEGIN_SRC clojure 563 | [{:db/id "taco" 564 | :car/make "toyota" 565 | :car/model "tacoma" 566 | :year 2014} 567 | 568 | {:db/id "325" 569 | :car/make "BMW" 570 | :car/model "325xi" 571 | :year 2001} 572 | 573 | {:db/id 3 574 | :user/name "ftravers" 575 | :user/age 54 576 | :cars [{:db/id "taco"} 577 | {:db/id "325"}]}] 578 | #+END_SRC 579 | 580 | Because the ~ftravers~ entity map needs to refer to the ~toyota~ and 581 | ~BMW~ entity maps, we include a ~:db/id~ field. You can put in any 582 | string here for your convenience. After transacting into datomic, 583 | they'll get converted to large integers as we've seen before. 584 | 585 | This data says ~ftravers~, owns two cars, a ~toyota~ and a ~BMW~ 586 | . So how do we model this? First we start with the schema. We'll 587 | need to define the fields: ~:car/make~, ~:car/model~, ~:year~, 588 | ~:user/name~, ~:user/age~, and ~:cars~. 589 | 590 | ~:car/make~, ~:car/model~, and ~:user/name~ are all of type ~string~ 591 | and cardinality one. For ~:year~ and ~:user/age~ we can use integers. 592 | ~:cars~ is the new one. 593 | 594 | The field ~:cars~ has a cardinality of ~many~; also the type that it 595 | will hold is of a type that points to other entities. We'll need a 596 | type that is like a pointer, reference or link. 597 | 598 | Let's look only at the schema for ~:cars~. You should be able to piece 599 | together the other fields from previous schema examples, or just look 600 | at the: 601 | 602 | GIT BRANCH: parent-child-modeling 603 | 604 | ** Many Refs Schema 605 | 606 | For the ~:cars~ field, the schema definition will look like: 607 | 608 | #+BEGIN_SRC clojure 609 | {:db/doc "List of cars a user owns" 610 | :db/ident :cars 611 | :db/valueType :db.type/ref 612 | :db/cardinality :db.cardinality/many 613 | :db.install/_attribute :db.part/db} 614 | #+END_SRC 615 | 616 | Take special note of the values for ~cardinality~ and ~valueType~. 617 | 618 | We have used a ~valueType~ of ~:db.type/ref~. This is how we point to 619 | (refer/link) to other entities in the DB. This is the critical 620 | difference between a database and regular old clojure data structures 621 | that don't support references. 622 | 623 | The second thing to note is that the ~cardinality~ is set to ~many~. 624 | That means this field will hold a list of values, not just a single 625 | value. 626 | 627 | ** Testdata 628 | 629 | Now let's make some testdata that can be transacted into the DB: 630 | 631 | #+BEGIN_SRC clojure 632 | (def test-data 633 | [{:db/id "taco" 634 | :car/make "toyota" 635 | :car/model "tacoma" 636 | :year 2014} 637 | 638 | {:db/id "325" 639 | :car/make "BMW" 640 | :car/model "325xi" 641 | :year 2001} 642 | 643 | {:db/id 3 644 | :user/name "ftravers" 645 | :user/age 54 646 | :cars [{:db/id "taco"} 647 | {:db/id "325"}]}]) 648 | #+END_SRC 649 | 650 | GIT BRANCH: parent-child-modeling 651 | 652 | Now that we have some parent/child data in the DB, let's see how to 653 | query and display it nicely. 654 | 655 | ** Querying Parent Child Data 656 | 657 | First we'll find the record we care about with a where clause that 658 | looks like: 659 | 660 | #+BEGIN_SRC clojure 661 | [?e :user/name "ftravers"] 662 | #+END_SRC 663 | 664 | This reads: "find all the entities that have the ~:user/name~ 665 | attribute that has as its value ~ftravers~". 666 | 667 | Now let's demonstrate how to format the results nicely with a slightly 668 | more advance pull pattern. 669 | 670 | ** Parent Child Pull Syntax 671 | 672 | We have already learned how to extract entity fields with a basic pull 673 | pattern: 674 | 675 | #+BEGIN_SRC clojure 676 | (pull ?e [:user/name :user/age]) 677 | #+END_SRC 678 | 679 | retrieves the ~:user/name~ and ~:user/age~ fields from the found, 680 | ~?e~, entity/entities. Again the result of this look like: 681 | 682 | #+BEGIN_SRC clojure 683 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 684 | [[#:user{:name "ftravers", :age 54}]] 685 | #+END_SRC 686 | 687 | but what we really want is something that looks like: 688 | 689 | #+BEGIN_SRC clojure 690 | datomic-tutorial.core> (query1 (-> db-url d/connect d/db)) 691 | [[{:user/name "ftravers", 692 | :user/age 54, 693 | :cars 694 | [#:car{:make "toyota", :model "tacoma"} 695 | #:car{:make "BMW", :model "325xi"}]}]] 696 | #+END_SRC 697 | 698 | So we want more than just the simple fields that an entity has, but we 699 | want to follow any references it has to other entities and get values 700 | from those entities. 701 | 702 | To get the above we change the pull pattern to be: 703 | 704 | #+BEGIN_SRC clojure 705 | [:user/name 706 | :user/age 707 | {:cars [:car/make :car/model]}] 708 | #+END_SRC 709 | 710 | So to get the children, and print out their fields, you start a new 711 | map, whose key is the parent field that points to the child. In our 712 | case ~:cars~. Then you start a vector and list the properties of the 713 | child you wish to grab. 714 | 715 | This is an extremely elegant way to extract arbitrary levels of data 716 | from datomic. Just imagine the mess this would look like with SQL. 717 | Maybe here is a stab just for comparison. 718 | 719 | #+BEGIN_SRC sql 720 | SELECT users.id users.name, users.age, cars.make, cars.model, cars.year 721 | FROM users cars 722 | WHERE users.id == cars.userid AND users.name == "ftravers" 723 | #+END_SRC 724 | 725 | And this would produce a result like: 726 | 727 | #+BEGIN_SRC clojure 728 | [[1 ftravers 54 "toyota" "tacoma" 2013] 729 | [1 ftravers 54 "BMW" "325xi" 2001]] 730 | #+END_SRC 731 | 732 | for comparison the equivalent datalog is: 733 | 734 | #+BEGIN_SRC clojure 735 | '[:find (pull ?e 736 | [:user/name 737 | :user/age 738 | {:cars [:car/make :car/model]}]) 739 | :where [?e :user/name "ftravers"]] 740 | #+END_SRC 741 | 742 | and its result, is nicely normalized: 743 | 744 | #+BEGIN_SRC clojure 745 | [[{:user/name "ftravers", 746 | :user/age 54, 747 | :cars 748 | [#:car{:make "toyota", :model "tacoma"} 749 | #:car{:make "BMW", :model "325xi"}]}]] 750 | #+END_SRC 751 | 752 | * Deeper Understanding 753 | 754 | ** Fields cross SQL Table boundaries 755 | 756 | So pretend we have two entities like: 757 | 758 | #+BEGIN_SRC clojure 759 | {:user/name "ftravers" 760 | :year 1945} 761 | 762 | {:car/make "BMW 325xi" 763 | :year 2001} 764 | #+END_SRC 765 | 766 | In datomic we can compare these two seemingly quite different objects 767 | with each other because they share a field: ~:year~. So I could write 768 | a query that returns *ALL THINGS* that are older than 35 years old. 769 | As I write this, it is 2017, so a 35 year old thing would be born 770 | (made) in approximately the year: 1982. So the where clause would 771 | look like: 772 | 773 | #+BEGIN_SRC clojure 774 | [?e :year ?year] 775 | [(<= ?year 1982)] 776 | #+END_SRC 777 | 778 | In RDBMS you normally are only ever comparing things that exist in the 779 | same table. So it'd be awkward to try a similar thing in an RDBMS. 780 | Primarily because they wouldn't have a combined index for fields in 781 | two separate tables. So your performance would die. In datomic each 782 | field has its own index, so a query like the above, would still be 783 | performant. 784 | 785 | -------------------------------------------------------------------------------- /om-next-client-server-tutorial.md: -------------------------------------------------------------------------------- 1 |
2 |

Table of Contents

3 |
4 | 44 |
45 |
46 | 47 | 48 | # Overview 49 | 50 | This tutorial will try to explain how to make the simplest 51 | client/server app using om-next and a simple backend. 52 | 53 | We'll start the tutorial with building a simple login page. The user 54 | is prompted for a username and a password, and when they supply the 55 | correct one, as verified by the backend database, the login form is 56 | replaced by a success message. 57 | 58 | We'll break down each step of this simple om-next app as it proceeds. 59 | 60 | # Get the Code 61 | 62 | The code is contained in two git repositories: 63 | 64 | The client code: 65 | 66 | 67 | `omn1` stands for: "Om Next, 1st tutorial" 68 | 69 | The server code: 70 | 71 | `omn1be` stands for: "Om Next Back End, 1st tutorial" 72 | 73 | Throughout the tutorial we'll reference branches in these 74 | repositories. Check that branch out to follow along. 75 | 76 | # In the Beginning 77 | 78 | *Git Repository*: `omn1` 79 | 80 | *Git Branch*: `in-the-beginning` 81 | 82 | Here is a super simple om-next web-app. All it does is print "Hello 83 | World". 84 | 85 | ```clojure 86 | 1 (ns omn1.webpage 87 | 2 (:require 88 | 3 [om.next :as om :refer-macros [defui]] 89 | 4 [om.dom :as dom :refer [div]] 90 | 5 [goog.dom :as gdom])) 91 | 6 92 | 7 (defui SimpleUI 93 | 8 Object 94 | 9 (render 95 | 10 [this] 96 | 11 (div nil "Hello World"))) 97 | 12 98 | 13 (om/add-root! 99 | 14 (om/reconciler {}) 100 | 15 SimpleUI 101 | 16 (gdom/getElement "app")) 102 | 103 | ``` 104 | 105 | **Line 14:** is a bit redundant in this example, but its a required argument to the 106 | `add-root!` function, so we include it. It doesn't do anything at 107 | this point, but later on we'll see what it does. 108 | 109 | **Line 7:** we refer to the user interface created by the `defui` macro as a 110 | *Component*. 111 | 112 | **Line 11:** we've hard-coded "Hello World" into the UI component, which is bad 113 | form, lets extract it now. 114 | 115 | # Remove State from Component 116 | 117 | *Git Branch*: `remove-state` 118 | 119 | Now we move the data from being hard coded inside the component to an 120 | external light weight database. Parts that are redundant from 121 | previous examples are elided. 122 | 123 | ```clojure 124 | 1 ;; ... 125 | 2 (defui SimpleUI 126 | 3 Object 127 | 4 (render 128 | 5 [this] 129 | 6 (div nil (:greeting (om/props this))))) 130 | 7 131 | 8 (def app-state (atom {:greeting "Hello World"})) 132 | 9 133 | 10 (def reconciler 134 | 11 (om/reconciler 135 | 12 {:state app-state})) 136 | 13 ;; ... 137 | 138 | ``` 139 | 140 | **Line 8:** here we create a state atom that holds the state 141 | for the application. 142 | 143 | **Line 12:** here we add the state atom into the 144 | application by wiring it into the `reconciler`. 145 | 146 | **Line 6:** finally, we access the state in the `root` component. 147 | 148 | # Add Query, Parser, Reader 149 | 150 | *Git Branch*: `add-reader-query-parser` 151 | 152 | ```clojure 153 | 1 ;; ... 154 | 2 (defui SimpleUI 155 | 3 static om/IQuery 156 | 4 (query [_] [:greeting]) 157 | 5 ;; ... 158 | 6 ) 159 | 7 ;; ... 160 | 8 (defn my-reader 161 | 9 [env kee parms] 162 | 10 (.log js/console (:target env)) 163 | 11 {:value "abc"}) 164 | 12 165 | 13 (def parser 166 | 14 (om/parser {:read my-reader})) 167 | 15 168 | 16 (def reconciler 169 | 17 (om/reconciler 170 | 18 {:state app-state 171 | 19 :parser parser})) 172 | 20 ;; ... 173 | 174 | ``` 175 | 176 | Now the application looks quite a bit more complicated. We've added a 177 | query to the component, a reader function and a parser. 178 | 179 | Run the program and inspect the console. The code: 180 | 181 | **Line 10**: causes the following output: 182 | 183 | ```clojure 184 | null 185 | :remote 186 | 187 | ``` 188 | 189 | Om-next will run the reader function once for a local query, and once 190 | for any remotes that are defined. We haven't define any remote end 191 | points, but om-next out of the box provides one remote called: 192 | `:remote`. A remote is a mechanism to wire in calls to a backend 193 | server. 194 | 195 | Our reader function `my-reader`, has the function parameter `kee`, set 196 | to the keyword `:greeting`. Then the reader result is a map with a 197 | key `:value` set to the string `abc`. 198 | 199 | Reader functions should always return a map with a `:value` key, that 200 | is set to whatever the value for the passed in `kee` is. 201 | 202 | As you can see `{:greeting "abc"}` gets printed out on the webpage. 203 | 204 | So we have a lot of ceremony already, and it is a bit hard to percieve 205 | the benefits of this approach at this point. Unfortunately, we'll 206 | just need to chug through this and hopefully in the end you can start 207 | to appreciate the benefits. 208 | 209 | # A Parameterized Query 210 | 211 | Our eventual goal is to create a login page that passes a username and 212 | password to a backend database, and if the username/password pair 213 | matches what is in the database, then we display a "login successful" 214 | page. 215 | 216 | Our query is going to be: `:user/authenticated`. This value will 217 | initially be `false`, but eventually, when the correct 218 | username/password pair is supplied, be changed to be `true`. 219 | 220 | *Git Branch*: `parameterize-query` 221 | 222 | ```clojure 223 | 1 (defui SimpleUI 224 | 2 static om/IQuery 225 | 3 (query [_] 226 | 4 '[(:user/authenticated 227 | 5 {:user/name ?name 228 | 6 :user/password ?pword})]) 229 | 7 230 | 8 static om/IQueryParams 231 | 9 (params [this] 232 | 10 {:name "" :pword ""}) 233 | 11 ;; ... 234 | 12 ) 235 | 13 236 | 14 (defn my-reader 237 | 15 [env kee parms] 238 | 16 (.log js/console parms) 239 | 17 ;; ... 240 | 18 ) 241 | 242 | ``` 243 | 244 | The `IQueryParams` indicate which parameters are available to this 245 | component and query. Our `IQuery` section has been updated to make 246 | use of these parameters. 247 | 248 | **Line 16:** We are dumping the `parms` parameter of the reader 249 | function to the console. Go inspect the console to see the shape of 250 | the data. 251 | 252 | # Adding in a remote 253 | 254 | *Git Branch*: `add-remote` 255 | 256 | ```clojure 257 | 1 ;; ... 258 | 2 (defui SimpleUI 259 | 3 static om/IQuery 260 | 4 (query [_] '[(:user/authenticated {:user/name ?name :user/password ?pword})]) 261 | 5 262 | 6 static om/IQueryParams 263 | 7 (params [this] 264 | 8 {:name "fenton" :pword "passwErd"}) 265 | 9 ;; ... 266 | 10 ) 267 | 11 268 | 12 (defn my-reader 269 | 13 [env kee parms] 270 | 14 (let [st (:state env)] 271 | 15 {:value (get @st kee) 272 | 16 :remote true 273 | 17 })) 274 | 18 275 | 19 (defn remote-connection 276 | 20 [qry cb] 277 | 21 (.log js/console (str (:remote qry))) 278 | 22 (cb {:user/authenticated true})) 279 | 23 280 | 24 (def reconciler 281 | 25 (om/reconciler 282 | 26 {:state app-state 283 | 27 :parser parser 284 | 28 :send remote-connection 285 | 29 })) 286 | 30 ;; ... 287 | 288 | ``` 289 | 290 | **Line 16:** Here we return `true` from our reader 291 | function to trigger the remote call. Here we return the name of the 292 | remote as the key, `:remote`, and set it's value to `true`. Om-next 293 | gives us this remote by default. We could add other remotes if we 294 | wanted to. 295 | 296 | **Line 28:** We must wire up our remote function in the 297 | `reconciler` with the `:send` keyword parameter. 298 | 299 | Now we have added a function that is stubbing out what will eventually 300 | be an actual call to a remote server. Our `remote-connection` 301 | function responds with the key `:user/authenticate` to `true`. 302 | 303 | **Line 8:** Finally lets hardcode in a username password 304 | pair. If you look at the console of the browser then, you'll see the 305 | following data spit out: 306 | 307 | ```clojure 308 | [(:user/authenticated 309 | {:user/name "fenton" 310 | :user/password "passwErd"})] 311 | 312 | ``` 313 | 314 | So this is the data that our client will send to our server. This is 315 | EDN. 316 | 317 | # The Architecture 318 | 319 | Om-next has nothing to say about how you would communicate with a 320 | backend server. So you can use any of the methods available to a 321 | browser to do this. Some examples of technologies you could use: 322 | http, REST, json, websockets, EDN, transit, blah, blah, blah. 323 | 324 | The key to understand is that the client has a piece of Clojure EDN 325 | data that it will give to you, and you have to send that back to the 326 | server somehow. This example happens to use EDN over websockets. 327 | Transit with REST might be another good way. 328 | 329 | In our example we are using this data: 330 | 331 | ```clojure 332 | [(:user/authenticated 333 | {:user/name "fenton" 334 | :user/password "passwErd"})] 335 | 336 | ``` 337 | 338 | Please keep this front and center in your mind. Any good integration 339 | is going to be all about data and only data. Here we have a classic 340 | piece of Clojure EDN. In classic clojure style, data is KING! 341 | 342 | Once the data is received by your tech stack on the server side, you 343 | pump it through om-next server. In our example we make use of a 344 | reader function and the om-next parser to handle this data from the 345 | client. In a full example you'd also have mutators too most likely. 346 | 347 | So lets switch gears and head over and build up an om-next server. 348 | 349 | # Om Next Server Basics 350 | 351 | So continuing on with our example, by some mechanism, the piece of 352 | data: 353 | 354 | ```clojure 355 | [(:user/authenticated 356 | {:user/name "fenton" 357 | :user/password "passwErd"})] 358 | 359 | ``` 360 | 361 | is going to arrive. We will fill in the plumbing between the client 362 | and server later. Remember that is not the focus of this tutorial, so 363 | it will not be explored in detail. 364 | 365 | ## Om-next Server Parts 366 | 367 | In om-next, it is the job of the *Parser*, to figure out what to do 368 | with both queries and mutations. Checkout the following github 369 | project if you haven't already done so: 370 | 371 | *Github Repository*: 372 | 373 | *Git Branch*: `step1-backend` 374 | 375 | Checkout the project and branch and launch your REPL. 376 | 377 | ```clojure 378 | lein repl 379 | 380 | ``` 381 | 382 | Now try some tests in the REPL: 383 | 384 | ```clojure 385 | omn1be.core> (parser {:state users} 386 | '[(:user/authenticated 387 | {:user/name "fenton" 388 | :user/password "passwerd"})]) 389 | #:user{:authenticated false} 390 | 391 | omn1be.core> (parser {:state users} 392 | '[(:user/authenticated 393 | {:user/name "fenton" 394 | :user/password "passwErd"})]) 395 | #:user{:authenticated true} 396 | 397 | ``` 398 | 399 | Lets quickly look at our reader function, even though it doesn't 400 | present any new ideas. 401 | 402 | ```clojure 403 | 1 (defn reader 404 | 2 [env kee params] 405 | 3 (let [userz (:state env) 406 | 4 username (:user/name params) 407 | 5 password (:user/password params)] 408 | 6 {:value (valid-user userz username password)} 409 | 7 )) 410 | 411 | ``` 412 | 413 | The input params are the same as on the client. 414 | 415 | **Line 6:** just like the client we simply return a map with 416 | the answer attached to the `:value` key. 417 | 418 | And our parser is dead simple: 419 | 420 | ```clojure 421 | (def parser (om/parser {:read reader})) 422 | 423 | ``` 424 | 425 | Thats all there is to a basic om-next server. 426 | 427 | # Full example 428 | 429 | For the full working sample checkout the master branches of the two 430 | projects, `omn1` and `omn1be`. 431 | 432 | ## Start the backend 433 | 434 | Start the backend at the command prompt: 435 | 436 | ```clojure 437 | cd omn1be; lein repl 438 | (load "websocket") 439 | (in-ns 'omn1be.websocket) 440 | (start) 441 | (in-ns 'omn1be.router) 442 | 443 | ``` 444 | 445 | ## Start the frontend 446 | 447 | ```clojure 448 | cd omn1; lein figwheel 449 | 450 | ``` 451 | 452 | Navigate to: 453 | 454 | 455 | 456 | Of course you'll need to have datomic installed for this complete 457 | example to work. 458 | 459 | # Additional and More in Depth Information 460 | 461 | ## Om Next Lifecycle Stages 462 | 463 | Our code has one root UI component. This component has a query for 464 | one field, `:user/authenticated`. The query for this field accept two 465 | parameters, `:user/name` and `:user/password`. 466 | 467 | The basic idea is that we send this query for the 468 | `:user/authenticated` value, passing along the username and password 469 | of the user. This gets looked up in the database and if the pair is 470 | valid, then `:user/authenticated` gets set to the value `true` 471 | otherwise it is set `false`. 472 | 473 | ### Load Root Component 474 | 475 | The first stage to an om next application is to load the Root 476 | component. This is dictated by the following line: 477 | 478 | ```clojure 479 | (om/add-root! reconciler Login (gdom/getElement "app")) 480 | 481 | ``` 482 | 483 | Here the second param, root-class, is set to the `Login` component. 484 | The third param, `target`, is the div in the `index.html` where to 485 | mount or locate this component. Finally the first argument is the 486 | reconciler to use for this application. The reconciler hold together 487 | all the function and state required to handle data flows in the 488 | application. 489 | 490 | 1. Our Query 491 | 492 | Our root component, `Login`, has a query of the form: 493 | 494 | ```clojure 495 | static om/IQuery 496 | (query 497 | [_] 498 | '[(:user/authenticated 499 | {:user/name ?name 500 | :user/password ?password})]) 501 | 502 | ``` 503 | 504 | Basically this says, get the value of `:user/authenticated` supplying 505 | as parameters to the query the values for the `:user/name` and 506 | `:user/password` fields. 507 | 508 | 2. Query Parameters 509 | 510 | `?name` and `?password` are query parameter variables that hold the 511 | values for the username and password that this query will eventually 512 | use in its query for `:user/authenticated`. We initially set their 513 | value to be the empty string: 514 | 515 | ```clojure 516 | static om/IQueryParams 517 | (params [this] 518 | {:name "" :password ""}) 519 | 520 | ``` 521 | 522 | 3. Component State 523 | 524 | In react we can have local state variables. The code: 525 | 526 | ```clojure 527 | (initLocalState 528 | [this] 529 | {:username "fenton" 530 | :password "passwErd"}) 531 | 532 | ``` 533 | 534 | creates two parameters: `:username:` and `:password` and sets their 535 | initial values. 536 | 537 | In the `:onChange` handlers for our two input elements we set the 538 | values of these two react state variables to be whatever the user 539 | types into the name and password input boxes. 540 | 541 | ```clojure 542 | (input 543 | #js 544 | {:name "uname" 545 | :type "text" 546 | :placeholder "Enter Username" 547 | :required true :value username 548 | :onChange 549 | (fn [ev] 550 | (let [value (.. ev -target -value)] 551 | (om/update-state! this assoc :username value)))}) 552 | 553 | ``` 554 | 555 | 4. Submitting username/password to backend 556 | 557 | Finally when the user clicks the submit button to send the username 558 | and password to the backend we take the values from the react 559 | component state, and use those values to update the values of the 560 | query parameters. Updating a query's parameter values causes the 561 | query to be rerun. 562 | 563 | Next we'll see how this state all runs by logging out to the console 564 | each time the reader is run. The reader is the function that is run 565 | to handle processing the queries. 566 | 567 | ### lifecycle logged to console 568 | 569 | We can see everytime a query is run by putting a log statement into 570 | our reader function. 571 | 572 | ```clojure 573 | (defmethod reader :default 574 | [{st :state :as env} key _] 575 | (log "default reader" key "env:target" (:target env)) 576 | {:value (key (om/db->tree [key] @st @st)) 577 | ;; :remote true 578 | :remote false 579 | }) 580 | 581 | ``` 582 | 583 | Here we see a log statement at the top of the reader function. Lets 584 | see what a dump of the browser console looks like and try to 585 | understand it. 586 | 587 | ```clojure 588 | 1 [default reader]: :user/authenticated env:target null 589 | 2 [props]: {:user/authenticated false} 590 | 3 [default reader]: :user/authenticated env:target :remote 591 | 592 | ``` 593 | 594 | In line nil: the query of the component is run before the 595 | component is first loaded. 596 | 597 | In line 2: as the component is rendered we dump the react 598 | properties that have been passed into the component, in this case it 599 | is simply the `@app-state`. 600 | 601 | This is done with line: 602 | 603 | ```clojure 604 | (log "props" (om/props this)) 605 | 606 | ``` 607 | 608 | In the component rendering. 609 | 610 | The line: 3, comes again from our `:default` reader, but this 611 | time it is passed for the remote called `:remote`. By default out of 612 | the box in om-next we get a remote named `:remote`. So the reader 613 | will get called once for a local call, and once for each remote we 614 | have defined. 615 | 616 | So we have traced a basic flow of a simple component. Now lets see 617 | how to trigger a remote read. When our reader is getting called with 618 | the `:target` a remote, if we then also return `:remote true` in our 619 | returned map from the reader, then our remote functions will also be 620 | called. 621 | 622 | ### Adding in a fake remote 623 | 624 | *Git Repository*: 625 | 626 | *Git Branch*: `simple-remote` 627 | 628 | So we want to send our stuff to a backend server. Om next creates a 629 | default hook for this. So basically what happens again, is that our 630 | reader will get called twice, once for trying to satisfy our query 631 | from our local state, and once for trying to get the information from 632 | the backend. 633 | 634 | If we return `:remote true` in our reader response map, the remote 635 | hooks will get triggered. So lets see this in action. First lets 636 | wire up some basic 'remotes'. 637 | 638 | First we must write a function that will be our remote query hook: 639 | 640 | ```clojure 641 | (defn my-remoter 642 | [qry cb] 643 | (log "remote query" (str qry)) 644 | (cb {:some-param "some value"})) 645 | 646 | ``` 647 | 648 | And lets wire this into the reconciler. 649 | 650 | ```clojure 651 | (def reconciler 652 | (om/reconciler 653 | {:state app-state 654 | :parser parser 655 | :send my-remoter})) 656 | 657 | ``` 658 | 659 | And finally our reader needs to return `:remote true` for the remote 660 | to run: 661 | 662 | ```clojure 663 | (defmethod reader :default 664 | [{st :state :as env} key _] 665 | (log "default reader" key "env:target" (:target env)) 666 | {:value (key (om/db->tree [key] @st @st)) 667 | :remote true}) 668 | 669 | ``` 670 | 671 | Now lets see what happens as we trace the programs execution with some 672 | logging statements 673 | 674 | ```clojure 675 | 1 [default reader]: :some-param env:target null 676 | 2 [props]: {:some-param "not much"}meta 677 | 3 [default reader]: :some-param env:target :remote 678 | 4 [remote query]: {:remote [:some-param]} 679 | 5 [app state]: {:some-param "not much"} 680 | 6 [default reader]: :some-param env:target null 681 | 7 [props]: {:some-param "value gotten from remote!"}meta 682 | 8 [app state]: {:some-param "value gotten from remote!"} 683 | 9 [default reader]: :some-param env:target null 684 | 685 | ``` 686 | 687 | The first three lines remain unchanged. 688 | 689 | **Line 4:** we see we've entered into the hook for the 690 | remote function. We dump the `@app-state` 691 | 692 | **Line 5:** before we call the callback, `cb`, with our 693 | new data, which should merge the data into our `@app-state` map. The 694 | callback is called and we can see that the `@app-state` is updated and 695 | the component is re-rendered. 696 | 697 | I'm not quite sure why the reader is called at the end…but maybe 698 | someone who knows om-next better can explain that. 699 | 700 | ### A real remote 701 | 702 | At this point we aren't hooking into any backend, we are just stubbing 703 | out the call to the backend. To have a real call to a backend 704 | involves taking our request and sending via `http`, `json`, 705 | `websockets`, `edn`, or some other way to our backend. Receiving the 706 | data, doing something with it and creating a response and sending it 707 | back, then getting it back on the client, and updating the local 708 | client data and therefore updating the client webpage. 709 | 710 | So that is a lot of stuff. Don't dispair, I will demonstrate real 711 | code that does this, but the scope of this tutorial is to demonstrate 712 | how to use om-next with a remote. How exactly data is exchanged with 713 | a remote is actually a separate concern. This is actually a wonderful 714 | thing. As clojuristas we dont like monolithic frameworks that package 715 | the entire world into an opinionated whole. Perhaps like a rails 716 | project. We would rather pick the pieces that best suit our needs, 717 | and data transport between client and server is not something that om 718 | next has an opinion on and it lets you fill in that blank however you 719 | would like. 720 | 721 | What we need to be clear on is the boundaries between the transport 722 | segment and om next. So lets reiterate that now to be absolutely 723 | clear. 724 | 725 | This boundary or responsibility handoff occurs in our `my-remoter` 726 | function. Om next hands us the data of the query that we've put into 727 | the `qry` parameter, then it expects us to call the callback, `cb`, 728 | with the results of our remote query. We'll look into detail of what 729 | the shape of the data is that om next expects us to return the result 730 | in. 731 | 732 | Here is a sample of data in and data out that om next would be happy 733 | with: 734 | 735 | IN: 736 | 737 | ```clojure 738 | [:some-param] 739 | 740 | ``` 741 | 742 | OUT: 743 | 744 | ```clojure 745 | {:some-param "Some New Value"} 746 | 747 | ``` 748 | 749 | ## My choice of transport 750 | 751 | I have written simple websocket client and server libraries that I 752 | use. They are located at: 753 | 754 | 755 | 756 | and 757 | 758 | 759 | 760 | I have chosen to send EDN over this websocket connection. 761 | 762 | Another perhaps better choice would be to send JSON over Transit. 763 | Perhaps using a Ring server or some other type of web server. My 764 | websocket server uses http-kit to act as the websocket server. 765 | 766 | Again, what you use is really beyond the scope of this tutorial, and I 767 | dont want this tutorial to get bogged down in those details, since it 768 | would detract from this tutorials purpose which is solely to educate a 769 | user on how to create a typical client server app using om-next. 770 | 771 | Truely this tutorial is about how to use om-next in a client/server 772 | setup, somewhat agnostic to whatever the backend database of choice 773 | is. 774 | 775 | So with those caveats declared lets look into what an om-next backend 776 | might look like. 777 | 778 | ## Om Next Backend 779 | 780 | The project for the om next backend is a git project located here, go 781 | ahead and clone it: 782 | 783 | 784 | 785 | The project name, omn1be, is the abbreviation of Om Next version 1 786 | Back End. 787 | 788 | In our example we are asking if a user has supplied the correct 789 | username password combination, and if so, to set the flag 790 | `:user/authenticated` to `true`, otherwise set it to `false`. 791 | 792 | Our complete example contains more pieces than what this tutorial is 793 | aiming to teach about. Here is a word diagram about the flow and 794 | architecture of the system: 795 | 796 | ```clojure 797 | [(:user/authenticated 798 | {:user/name "fenton" 799 | :user/password "passwErd"})] 800 | 801 | ``` 802 | 803 | Again here we need to be clear of where the handoff occurs from the 804 | choice of wire or transport architecture occurs and where we enter the 805 | land of om-next for the backend. Lets inspect the file layout for the 806 | project first: 807 | 808 | ```clojure 809 | ╭─fenton@ss9 ~/projects ‹system› ‹master*› 810 | ╰─➤ cd omn1be 811 | ╭─fenton@ss9 ~/projects/omn1be ‹system› ‹upper-case› 812 | ╰─➤ tree src 813 | src 814 | `-- omn1be 815 | |-- core.clj 816 | |-- router.clj 817 | `-- websocket.clj 818 | 819 | ``` 820 | 821 | The `core.clj` file has all the information about the datomic 822 | database. It has the schema, the testdata, etc. If you need more 823 | help understanding how datomic works, please checkout my tutorial at: 824 | 825 | [Beginner Datomic Tutorial](https://github.com/ftravers/missing-links/blob/master/datomic-tutorial.md) 826 | 827 | Again, I will highlight the boundaries of the durability layer 828 | (i.e. the database), and om-next server side. 829 | 830 | The file: `websocket.clj`, is the servers side of the transport 831 | layer. Again you could sub this out with whatever type of transport 832 | you wanted to do. 833 | 834 | Finally, the file: `router.clj` is truely the om-next server side. If 835 | you want to do om-next on the server side then this file will be the 836 | most interesting for you. 837 | 838 | ### The transport to om-next server boundary 839 | 840 | Lets point out where the boundary of the server end of the transport 841 | layer to the om-next server is. 842 | 843 | Have a look at the 844 | 845 | *Git Branch*: `full-working-basic-backend` 846 | 847 | To fire up the backend you could do: 848 | 849 | ```clojure 850 | $ cd omn1be; lein repl 851 | (load "websocket") 852 | (in-ns 'omn1be.websocket) 853 | (start) 854 | (in-ns 'omn1be.router) 855 | 856 | ``` 857 | 858 | Then to test it without our front end, we could use the "Simple 859 | Websocket Client" chrome extension. 860 | 861 | The websocket URL end point is: `ws://localhost:7890` 862 | 863 | Then we can send the following data in it: 864 | 865 | ```clojure 866 | [(:user/authenticated 867 | {:user/name "fenton" 868 | :user/password "passwErd"})] 869 | 870 | ``` 871 | 872 | Here is a log of some sent requests and their response from the 873 | server: 874 | 875 | ```clojure 876 | [(:user/authenticated 877 | {:user/name "fenton" 878 | :user/password "passwErd"})] 879 | {:user/authenticated true} 880 | 881 | [(:user/authenticated 882 | {:user/name "fenton" 883 | :user/password "password"})] 884 | {:user/authenticated false} 885 | 886 | ``` 887 | 888 | ### Backend Parser 889 | 890 | So we can see that all we are sending over the wire is an om next 891 | parameterized query. 892 | 893 | ```clojure 894 | [(:user/authenticated 895 | {:user/name "fenton" 896 | :user/password "passwErd"})] 897 | 898 | ``` 899 | 900 | A good reference for the different types of queries can be found at: 901 | [Query Syntax Explained](https://anmonteiro.com/2016/01/om-next-query-syntax/). 902 | 903 | If we create a server side reader and parser, we can pass this query 904 | to it and it will act almost the same as the front end. 905 | 906 | When we develop an om next backend there is a symmetry to the front 907 | end. Again we will create a reader function and create a parser with 908 | this reader function. So we pass from the transport layer, into the 909 | om-next server layer in this code: 910 | 911 | ```clojure 912 | (defn process-data [data] 913 | (->> data 914 | read-string 915 | (router/parser {:database (be/db)}) 916 | prn-str)) 917 | 918 | ``` 919 | 920 | Particularly when we call the `parser` with the data we recieved. The 921 | result of calling the parser is passed back into the transport layer. 922 | -------------------------------------------------------------------------------- /om-next-client-server-tutorial.org: -------------------------------------------------------------------------------- 1 | #+TITLE: Om Next on Client & Server Tutorial 2 | * Overview 3 | 4 | This tutorial will try to explain how to make the simplest 5 | client/server app using om-next and a simple backend. 6 | 7 | We'll start the tutorial with building a simple login page. The user 8 | is prompted for a username and a password, and when they supply the 9 | correct one, as verified by the backend database, the login form is 10 | replaced by a success message. 11 | 12 | We'll break down each step of this simple om-next app as it proceeds. 13 | 14 | * Get the Code 15 | 16 | The code is contained in two git repositories: 17 | 18 | The client code: 19 | https://github.com/ftravers/omn1 20 | 21 | ~omn1~ stands for: "Om Next, 1st tutorial" 22 | 23 | The server code: https://github.com/ftravers/omn1be 24 | 25 | ~omn1be~ stands for: "Om Next Back End, 1st tutorial" 26 | 27 | Throughout the tutorial we'll reference branches in these 28 | repositories. Check that branch out to follow along. 29 | 30 | * In the Beginning 31 | 32 | /Git Repository/: ~omn1~ 33 | 34 | /Git Branch/: ~in-the-beginning~ 35 | 36 | Here is a super simple om-next web-app. All it does is print "Hello 37 | World". 38 | 39 | #+BEGIN_SRC clojure -r -n 40 | (ns omn1.webpage 41 | (:require 42 | [om.next :as om :refer-macros [defui]] 43 | [om.dom :as dom :refer [div]] 44 | [goog.dom :as gdom])) 45 | 46 | (defui SimpleUI (ref:ss-comp) 47 | Object 48 | (render 49 | [this] 50 | (div nil "Hello World"))) (ref:ss-hard-code) 51 | 52 | (om/add-root! 53 | (om/reconciler {}) (ref:ss-recon) 54 | SimpleUI 55 | (gdom/getElement "app")) 56 | #+END_SRC 57 | 58 | *Line [[(ss-recon)]]:* is a bit redundant in this example, but its a required argument to the 59 | ~add-root!~ function, so we include it. It doesn't do anything at 60 | this point, but later on we'll see what it does. 61 | 62 | *Line [[(ss-comp)]]:* we refer to the user interface created by the ~defui~ macro as a 63 | /Component/. 64 | 65 | *Line [[(ss-hard-code)]]:* we've hard-coded "Hello World" into the UI component, which is bad 66 | form, lets extract it now. 67 | 68 | * Remove State from Component 69 | 70 | /Git Branch/: ~remove-state~ 71 | 72 | Now we move the data from being hard coded inside the component to an 73 | external light weight database. Parts that are redundant from 74 | previous examples are elided. 75 | 76 | #+BEGIN_SRC clojure -r -n 77 | ;; ... 78 | (defui SimpleUI 79 | Object 80 | (render 81 | [this] 82 | (div nil (:greeting (om/props this))))) (ref:rs-access) 83 | 84 | (def app-state (atom {:greeting "Hello World"})) (ref:rs-state) 85 | 86 | (def reconciler 87 | (om/reconciler 88 | {:state app-state})) (ref:rs-wire-state) 89 | ;; ... 90 | #+END_SRC 91 | 92 | *Line [[(rs-state)]]:* here we create a state atom that holds the state 93 | for the application. 94 | 95 | *Line [[(rs-wire-state)]]:* here we add the state atom into the 96 | application by wiring it into the ~reconciler~. 97 | 98 | *Line [[(rs-access)]]:* finally, we access the state in the ~root~ component. 99 | 100 | * Add Query, Parser, Reader 101 | 102 | /Git Branch/: ~add-reader-query-parser~ 103 | 104 | #+BEGIN_SRC clojure -r -n 105 | ;; ... 106 | (defui SimpleUI 107 | static om/IQuery 108 | (query [_] [:greeting]) 109 | ;; ... 110 | ) 111 | ;; ... 112 | (defn my-reader 113 | [env kee parms] 114 | (.log js/console (:target env)) (ref:rqp-logging) 115 | {:value "abc"}) 116 | 117 | (def parser 118 | (om/parser {:read my-reader})) 119 | 120 | (def reconciler 121 | (om/reconciler 122 | {:state app-state 123 | :parser parser})) 124 | ;; ... 125 | #+END_SRC 126 | 127 | Now the application looks quite a bit more complicated. We've added a 128 | query to the component, a reader function and a parser. 129 | 130 | Run the program and inspect the console. The code: 131 | 132 | *Line [[(rqp-logging)]]*: causes the following output: 133 | 134 | #+BEGIN_SRC config 135 | null 136 | :remote 137 | #+END_SRC 138 | 139 | Om-next will run the reader function once for a local query, and once 140 | for any remotes that are defined. We haven't define any remote end 141 | points, but om-next out of the box provides one remote called: 142 | ~:remote~. A remote is a mechanism to wire in calls to a backend 143 | server. 144 | 145 | Our reader function ~my-reader~, has the function parameter ~kee~, set 146 | to the keyword ~:greeting~. Then the reader result is a map with a 147 | key ~:value~ set to the string ~abc~. 148 | 149 | Reader functions should always return a map with a ~:value~ key, that 150 | is set to whatever the value for the passed in ~kee~ is. 151 | 152 | As you can see ~{:greeting "abc"}~ gets printed out on the webpage. 153 | 154 | So we have a lot of ceremony already, and it is a bit hard to percieve 155 | the benefits of this approach at this point. Unfortunately, we'll 156 | just need to chug through this and hopefully in the end you can start 157 | to appreciate the benefits. 158 | 159 | * A Parameterized Query 160 | 161 | Our eventual goal is to create a login page that passes a username and 162 | password to a backend database, and if the username/password pair 163 | matches what is in the database, then we display a "login successful" 164 | page. 165 | 166 | Our query is going to be: ~:user/authenticated~. This value will 167 | initially be ~false~, but eventually, when the correct 168 | username/password pair is supplied, be changed to be ~true~. 169 | 170 | /Git Branch/: ~parameterize-query~ 171 | 172 | #+BEGIN_SRC clojure -n -r 173 | (defui SimpleUI 174 | static om/IQuery 175 | (query [_] 176 | '[(:user/authenticated 177 | {:user/name ?name 178 | :user/password ?pword})]) 179 | 180 | static om/IQueryParams 181 | (params [this] 182 | {:name "" :pword ""}) 183 | ;; ... 184 | ) 185 | 186 | (defn my-reader 187 | [env kee parms] 188 | (.log js/console parms) (ref:pq-logging) 189 | ;; ... 190 | ) 191 | #+END_SRC 192 | 193 | The ~IQueryParams~ indicate which parameters are available to this 194 | component and query. Our ~IQuery~ section has been updated to make 195 | use of these parameters. 196 | 197 | *Line [[(pq-logging)]]:* We are dumping the ~parms~ parameter of the reader 198 | function to the console. Go inspect the console to see the shape of 199 | the data. 200 | 201 | * Adding in a remote 202 | 203 | /Git Branch/: ~add-remote~ 204 | 205 | #+BEGIN_SRC clojure -r -n 206 | ;; ... 207 | (defui SimpleUI 208 | static om/IQuery 209 | (query [_] '[(:user/authenticated {:user/name ?name :user/password ?pword})]) 210 | 211 | static om/IQueryParams 212 | (params [this] 213 | {:name "fenton" :pword "passwErd"}) (ref:ar-hard-code) 214 | ;; ... 215 | ) 216 | 217 | (defn my-reader 218 | [env kee parms] 219 | (let [st (:state env)] 220 | {:value (get @st kee) 221 | :remote true (ref:ar-reader-remote) 222 | })) 223 | 224 | (defn remote-connection 225 | [qry cb] 226 | (.log js/console (str (:remote qry))) 227 | (cb {:user/authenticated true})) 228 | 229 | (def reconciler 230 | (om/reconciler 231 | {:state app-state 232 | :parser parser 233 | :send remote-connection (ref:ar-wire-recon) 234 | })) 235 | ;; ... 236 | #+END_SRC 237 | 238 | *Line [[(ar-reader-remote)]]:* Here we return ~true~ from our reader 239 | function to trigger the remote call. Here we return the name of the 240 | remote as the key, ~:remote~, and set it's value to ~true~. Om-next 241 | gives us this remote by default. We could add other remotes if we 242 | wanted to. 243 | 244 | *Line [[(ar-wire-recon)]]:* We must wire up our remote function in the 245 | ~reconciler~ with the ~:send~ keyword parameter. 246 | 247 | Now we have added a function that is stubbing out what will eventually 248 | be an actual call to a remote server. Our ~remote-connection~ 249 | function responds with the key ~:user/authenticate~ to ~true~. 250 | 251 | *Line [[(ar-hard-code)]]:* Finally lets hardcode in a username password 252 | pair. If you look at the console of the browser then, you'll see the 253 | following data spit out: 254 | 255 | #+BEGIN_SRC clojure 256 | [(:user/authenticated 257 | {:user/name "fenton" 258 | :user/password "passwErd"})] 259 | #+END_SRC 260 | 261 | So this is the data that our client will send to our server. This is 262 | EDN. 263 | 264 | * The Architecture 265 | 266 | Om-next has nothing to say about how you would communicate with a 267 | backend server. So you can use any of the methods available to a 268 | browser to do this. Some examples of technologies you could use: 269 | http, REST, json, websockets, EDN, transit, blah, blah, blah. 270 | 271 | The key to understand is that the client has a piece of Clojure EDN 272 | data that it will give to you, and you have to send that back to the 273 | server somehow. This example happens to use EDN over websockets. 274 | Transit with REST might be another good way. 275 | 276 | In our example we are using this data: 277 | 278 | #+BEGIN_SRC clojure 279 | [(:user/authenticated 280 | {:user/name "fenton" 281 | :user/password "passwErd"})] 282 | #+END_SRC 283 | 284 | Please keep this front and center in your mind. Any good integration 285 | is going to be all about data and only data. Here we have a classic 286 | piece of Clojure EDN. In classic clojure style, data is KING! 287 | 288 | Once the data is received by your tech stack on the server side, you 289 | pump it through om-next server. In our example we make use of a 290 | reader function and the om-next parser to handle this data from the 291 | client. In a full example you'd also have mutators too most likely. 292 | 293 | So lets switch gears and head over and build up an om-next server. 294 | 295 | * Om Next Server Basics 296 | 297 | So continuing on with our example, by some mechanism, the piece of 298 | data: 299 | 300 | #+BEGIN_SRC clojure 301 | [(:user/authenticated 302 | {:user/name "fenton" 303 | :user/password "passwErd"})] 304 | #+END_SRC 305 | 306 | is going to arrive. We will fill in the plumbing between the client 307 | and server later. Remember that is not the focus of this tutorial, so 308 | it will not be explored in detail. 309 | 310 | ** Om-next Server Parts 311 | 312 | In om-next, it is the job of the /Parser/, to figure out what to do 313 | with both queries and mutations. Checkout the following github 314 | project if you haven't already done so: 315 | 316 | /Github Repository/: https://github.com/ftravers/omn1be 317 | 318 | /Git Branch/: ~step1-backend~ 319 | 320 | Checkout the project and branch and launch your REPL. 321 | 322 | #+BEGIN_SRC config 323 | lein repl 324 | #+END_SRC 325 | 326 | Now try some tests in the REPL: 327 | 328 | #+BEGIN_SRC clojure 329 | omn1be.core> (parser {:state users} 330 | '[(:user/authenticated 331 | {:user/name "fenton" 332 | :user/password "passwerd"})]) 333 | #:user{:authenticated false} 334 | 335 | omn1be.core> (parser {:state users} 336 | '[(:user/authenticated 337 | {:user/name "fenton" 338 | :user/password "passwErd"})]) 339 | #:user{:authenticated true} 340 | #+END_SRC 341 | 342 | Lets quickly look at our reader function, even though it doesn't 343 | present any new ideas. 344 | 345 | #+BEGIN_SRC clojure -r -n 346 | (defn reader 347 | [env kee params] 348 | (let [userz (:state env) 349 | username (:user/name params) 350 | password (:user/password params)] 351 | {:value (valid-user userz username password)} (ref:be-reader) 352 | )) 353 | #+END_SRC 354 | 355 | The input params are the same as on the client. 356 | 357 | *Line [[(be-reader)]]:* just like the client we simply return a map with 358 | the answer attached to the ~:value~ key. 359 | 360 | And our parser is dead simple: 361 | 362 | #+BEGIN_SRC clojure 363 | (def parser (om/parser {:read reader})) 364 | #+END_SRC 365 | 366 | Thats all there is to a basic om-next server. 367 | 368 | * Full example 369 | 370 | For the full working sample checkout the master branches of the two 371 | projects, ~omn1~ and ~omn1be~. 372 | 373 | ** Start the backend 374 | 375 | Start the backend at the command prompt: 376 | 377 | #+BEGIN_SRC clojure 378 | cd omn1be; lein repl 379 | (load "websocket") 380 | (in-ns 'omn1be.websocket) 381 | (start) 382 | (in-ns 'omn1be.router) 383 | #+END_SRC 384 | 385 | ** Start the frontend 386 | 387 | #+BEGIN_SRC 388 | cd omn1; lein figwheel 389 | #+END_SRC 390 | 391 | Navigate to: 392 | 393 | http://localhost:3449/ 394 | 395 | Of course you'll need to have datomic installed for this complete 396 | example to work. 397 | 398 | * Additional and More in Depth Information 399 | ** Om Next Lifecycle Stages 400 | 401 | Our code has one root UI component. This component has a query for 402 | one field, ~:user/authenticated~. The query for this field accept two 403 | parameters, ~:user/name~ and ~:user/password~. 404 | 405 | The basic idea is that we send this query for the 406 | ~:user/authenticated~ value, passing along the username and password 407 | of the user. This gets looked up in the database and if the pair is 408 | valid, then ~:user/authenticated~ gets set to the value ~true~ 409 | otherwise it is set ~false~. 410 | 411 | *** Load Root Component 412 | 413 | The first stage to an om next application is to load the Root 414 | component. This is dictated by the following line: 415 | 416 | #+BEGIN_SRC clojure 417 | (om/add-root! reconciler Login (gdom/getElement "app")) 418 | #+END_SRC 419 | 420 | Here the second param, root-class, is set to the ~Login~ component. 421 | The third param, ~target~, is the div in the ~index.html~ where to 422 | mount or locate this component. Finally the first argument is the 423 | reconciler to use for this application. The reconciler hold together 424 | all the function and state required to handle data flows in the 425 | application. 426 | 427 | **** Our Query 428 | 429 | Our root component, ~Login~, has a query of the form: 430 | 431 | #+BEGIN_SRC clojure 432 | static om/IQuery 433 | (query 434 | [_] 435 | '[(:user/authenticated 436 | {:user/name ?name 437 | :user/password ?password})]) 438 | #+END_SRC 439 | 440 | Basically this says, get the value of ~:user/authenticated~ supplying 441 | as parameters to the query the values for the ~:user/name~ and 442 | ~:user/password~ fields. 443 | 444 | **** Query Parameters 445 | 446 | ~?name~ and ~?password~ are query parameter variables that hold the 447 | values for the username and password that this query will eventually 448 | use in its query for ~:user/authenticated~. We initially set their 449 | value to be the empty string: 450 | 451 | #+BEGIN_SRC clojure 452 | static om/IQueryParams 453 | (params [this] 454 | {:name "" :password ""}) 455 | #+END_SRC 456 | 457 | **** Component State 458 | 459 | In react we can have local state variables. The code: 460 | 461 | #+BEGIN_SRC clojure 462 | (initLocalState 463 | [this] 464 | {:username "fenton" 465 | :password "passwErd"}) 466 | #+END_SRC 467 | 468 | creates two parameters: ~:username:~ and ~:password~ and sets their 469 | initial values. 470 | 471 | In the ~:onChange~ handlers for our two input elements we set the 472 | values of these two react state variables to be whatever the user 473 | types into the name and password input boxes. 474 | 475 | #+BEGIN_SRC clojure 476 | (input 477 | #js 478 | {:name "uname" 479 | :type "text" 480 | :placeholder "Enter Username" 481 | :required true :value username 482 | :onChange 483 | (fn [ev] 484 | (let [value (.. ev -target -value)] 485 | (om/update-state! this assoc :username value)))}) 486 | #+END_SRC 487 | 488 | **** Submitting username/password to backend 489 | 490 | Finally when the user clicks the submit button to send the username 491 | and password to the backend we take the values from the react 492 | component state, and use those values to update the values of the 493 | query parameters. Updating a query's parameter values causes the 494 | query to be rerun. 495 | 496 | Next we'll see how this state all runs by logging out to the console 497 | each time the reader is run. The reader is the function that is run 498 | to handle processing the queries. 499 | 500 | *** lifecycle logged to console 501 | 502 | We can see everytime a query is run by putting a log statement into 503 | our reader function. 504 | 505 | #+BEGIN_SRC clojure 506 | (defmethod reader :default 507 | [{st :state :as env} key _] 508 | (log "default reader" key "env:target" (:target env)) 509 | {:value (key (om/db->tree [key] @st @st)) 510 | ;; :remote true 511 | :remote false 512 | }) 513 | #+END_SRC 514 | 515 | Here we see a log statement at the top of the reader function. Lets 516 | see what a dump of the browser console looks like and try to 517 | understand it. 518 | 519 | #+BEGIN_SRC config -n -r 520 | [default reader]: :user/authenticated env:target null(ref:load-comp1) 521 | [props]: {:user/authenticated false} (ref:load-comp2) 522 | [default reader]: :user/authenticated env:target :remote (ref:remote) 523 | #+END_SRC 524 | 525 | In line [[(load-comp)]]: the query of the component is run before the 526 | component is first loaded. 527 | 528 | In line [[(load-comp2)]]: as the component is rendered we dump the react 529 | properties that have been passed into the component, in this case it 530 | is simply the ~@app-state~. 531 | 532 | This is done with line: 533 | 534 | #+BEGIN_SRC clojure 535 | (log "props" (om/props this)) 536 | #+END_SRC 537 | 538 | In the component rendering. 539 | 540 | The line: [[(remote)]], comes again from our ~:default~ reader, but this 541 | time it is passed for the remote called ~:remote~. By default out of 542 | the box in om-next we get a remote named ~:remote~. So the reader 543 | will get called once for a local call, and once for each remote we 544 | have defined. 545 | 546 | So we have traced a basic flow of a simple component. Now lets see 547 | how to trigger a remote read. When our reader is getting called with 548 | the ~:target~ a remote, if we then also return ~:remote true~ in our 549 | returned map from the reader, then our remote functions will also be 550 | called. 551 | 552 | *** Adding in a fake remote 553 | /Git Repository/: https://github.com/ftravers/omn1 554 | 555 | /Git Branch/: ~simple-remote~ 556 | 557 | So we want to send our stuff to a backend server. Om next creates a 558 | default hook for this. So basically what happens again, is that our 559 | reader will get called twice, once for trying to satisfy our query 560 | from our local state, and once for trying to get the information from 561 | the backend. 562 | 563 | If we return ~:remote true~ in our reader response map, the remote 564 | hooks will get triggered. So lets see this in action. First lets 565 | wire up some basic 'remotes'. 566 | 567 | First we must write a function that will be our remote query hook: 568 | 569 | #+BEGIN_SRC clojure 570 | (defn my-remoter 571 | [qry cb] 572 | (log "remote query" (str qry)) 573 | (cb {:some-param "some value"})) 574 | #+END_SRC 575 | 576 | And lets wire this into the reconciler. 577 | 578 | #+BEGIN_SRC clojure 579 | (def reconciler 580 | (om/reconciler 581 | {:state app-state 582 | :parser parser 583 | :send my-remoter})) 584 | #+END_SRC 585 | 586 | And finally our reader needs to return ~:remote true~ for the remote 587 | to run: 588 | 589 | #+BEGIN_SRC clojure 590 | (defmethod reader :default 591 | [{st :state :as env} key _] 592 | (log "default reader" key "env:target" (:target env)) 593 | {:value (key (om/db->tree [key] @st @st)) 594 | :remote true}) 595 | #+END_SRC 596 | 597 | Now lets see what happens as we trace the programs execution with some 598 | logging statements 599 | 600 | #+BEGIN_SRC config -n -r 601 | [default reader]: :some-param env:target null 602 | [props]: {:some-param "not much"}meta 603 | [default reader]: :some-param env:target :remote 604 | [remote query]: {:remote [:some-param]} (ref:remote-query) 605 | [app state]: {:some-param "not much"} (ref:app-state-before-remote) 606 | [default reader]: :some-param env:target null 607 | [props]: {:some-param "value gotten from remote!"}meta 608 | [app state]: {:some-param "value gotten from remote!"} 609 | [default reader]: :some-param env:target null 610 | #+END_SRC 611 | 612 | The first three lines remain unchanged. 613 | 614 | *Line [[(remote-query)]]:* we see we've entered into the hook for the 615 | remote function. We dump the ~@app-state~ 616 | 617 | *Line [[(app-state-before-remote)]]:* before we call the callback, ~cb~, with our 618 | new data, which should merge the data into our ~@app-state~ map. The 619 | callback is called and we can see that the ~@app-state~ is updated and 620 | the component is re-rendered. 621 | 622 | I'm not quite sure why the reader is called at the end...but maybe 623 | someone who knows om-next better can explain that. 624 | 625 | *** A real remote 626 | 627 | At this point we aren't hooking into any backend, we are just stubbing 628 | out the call to the backend. To have a real call to a backend 629 | involves taking our request and sending via ~http~, ~json~, 630 | ~websockets~, ~edn~, or some other way to our backend. Receiving the 631 | data, doing something with it and creating a response and sending it 632 | back, then getting it back on the client, and updating the local 633 | client data and therefore updating the client webpage. 634 | 635 | So that is a lot of stuff. Don't dispair, I will demonstrate real 636 | code that does this, but the scope of this tutorial is to demonstrate 637 | how to use om-next with a remote. How exactly data is exchanged with 638 | a remote is actually a separate concern. This is actually a wonderful 639 | thing. As clojuristas we dont like monolithic frameworks that package 640 | the entire world into an opinionated whole. Perhaps like a rails 641 | project. We would rather pick the pieces that best suit our needs, 642 | and data transport between client and server is not something that om 643 | next has an opinion on and it lets you fill in that blank however you 644 | would like. 645 | 646 | What we need to be clear on is the boundaries between the transport 647 | segment and om next. So lets reiterate that now to be absolutely 648 | clear. 649 | 650 | This boundary or responsibility handoff occurs in our ~my-remoter~ 651 | function. Om next hands us the data of the query that we've put into 652 | the ~qry~ parameter, then it expects us to call the callback, ~cb~, 653 | with the results of our remote query. We'll look into detail of what 654 | the shape of the data is that om next expects us to return the result 655 | in. 656 | 657 | Here is a sample of data in and data out that om next would be happy 658 | with: 659 | 660 | IN: 661 | 662 | #+BEGIN_SRC clojure 663 | [:some-param] 664 | #+END_SRC 665 | 666 | OUT: 667 | 668 | #+BEGIN_SRC clojure 669 | {:some-param "Some New Value"} 670 | #+END_SRC 671 | 672 | ** My choice of transport 673 | 674 | I have written simple websocket client and server libraries that I 675 | use. They are located at: 676 | 677 | https://github.com/ftravers/websocket-client 678 | 679 | and 680 | 681 | https://github.com/ftravers/websocket-server 682 | 683 | I have chosen to send EDN over this websocket connection. 684 | 685 | Another perhaps better choice would be to send JSON over Transit. 686 | Perhaps using a Ring server or some other type of web server. My 687 | websocket server uses http-kit to act as the websocket server. 688 | 689 | Again, what you use is really beyond the scope of this tutorial, and I 690 | dont want this tutorial to get bogged down in those details, since it 691 | would detract from this tutorials purpose which is solely to educate a 692 | user on how to create a typical client server app using om-next. 693 | 694 | Truely this tutorial is about how to use om-next in a client/server 695 | setup, somewhat agnostic to whatever the backend database of choice 696 | is. 697 | 698 | So with those caveats declared lets look into what an om-next backend 699 | might look like. 700 | 701 | ** Om Next Backend 702 | 703 | The project for the om next backend is a git project located here, go 704 | ahead and clone it: 705 | 706 | https://github.com/ftravers/omn1be 707 | 708 | The project name, omn1be, is the abbreviation of Om Next version 1 709 | Back End. 710 | 711 | In our example we are asking if a user has supplied the correct 712 | username password combination, and if so, to set the flag 713 | ~:user/authenticated~ to ~true~, otherwise set it to ~false~. 714 | 715 | Our complete example contains more pieces than what this tutorial is 716 | aiming to teach about. Here is a word diagram about the flow and 717 | architecture of the system: 718 | 719 | #+BEGIN_SRC clojure 720 | [(:user/authenticated 721 | {:user/name "fenton" 722 | :user/password "passwErd"})] 723 | #+END_SRC 724 | 725 | Again here we need to be clear of where the handoff occurs from the 726 | choice of wire or transport architecture occurs and where we enter the 727 | land of om-next for the backend. Lets inspect the file layout for the 728 | project first: 729 | 730 | #+BEGIN_SRC config 731 | ╭─fenton@ss9 ~/projects ‹system› ‹master*› 732 | ╰─➤ cd omn1be 733 | ╭─fenton@ss9 ~/projects/omn1be ‹system› ‹upper-case› 734 | ╰─➤ tree src 735 | src 736 | `-- omn1be 737 | |-- core.clj 738 | |-- router.clj 739 | `-- websocket.clj 740 | #+END_SRC 741 | 742 | The ~core.clj~ file has all the information about the datomic 743 | database. It has the schema, the testdata, etc. If you need more 744 | help understanding how datomic works, please checkout my tutorial at: 745 | 746 | [[https://github.com/ftravers/missing-links/blob/master/datomic-tutorial.md][Beginner Datomic Tutorial]] 747 | 748 | Again, I will highlight the boundaries of the durability layer 749 | (i.e. the database), and om-next server side. 750 | 751 | The file: ~websocket.clj~, is the servers side of the transport 752 | layer. Again you could sub this out with whatever type of transport 753 | you wanted to do. 754 | 755 | Finally, the file: ~router.clj~ is truely the om-next server side. If 756 | you want to do om-next on the server side then this file will be the 757 | most interesting for you. 758 | 759 | *** The transport to om-next server boundary 760 | 761 | Lets point out where the boundary of the server end of the transport 762 | layer to the om-next server is. 763 | 764 | Have a look at the 765 | 766 | /Git Branch/: ~full-working-basic-backend~ 767 | 768 | To fire up the backend you could do: 769 | 770 | #+BEGIN_SRC clojure 771 | $ cd omn1be; lein repl 772 | (load "websocket") 773 | (in-ns 'omn1be.websocket) 774 | (start) 775 | (in-ns 'omn1be.router) 776 | #+END_SRC 777 | 778 | Then to test it without our front end, we could use the "Simple 779 | Websocket Client" chrome extension. 780 | 781 | The websocket URL end point is: ~ws://localhost:7890~ 782 | 783 | Then we can send the following data in it: 784 | 785 | #+BEGIN_SRC clojure 786 | [(:user/authenticated 787 | {:user/name "fenton" 788 | :user/password "passwErd"})] 789 | #+END_SRC 790 | 791 | Here is a log of some sent requests and their response from the 792 | server: 793 | 794 | #+BEGIN_SRC clojure 795 | [(:user/authenticated 796 | {:user/name "fenton" 797 | :user/password "passwErd"})] 798 | {:user/authenticated true} 799 | 800 | [(:user/authenticated 801 | {:user/name "fenton" 802 | :user/password "password"})] 803 | {:user/authenticated false} 804 | #+END_SRC 805 | 806 | *** Backend Parser 807 | 808 | So we can see that all we are sending over the wire is an om next 809 | parameterized query. 810 | 811 | #+BEGIN_SRC clojure 812 | [(:user/authenticated 813 | {:user/name "fenton" 814 | :user/password "passwErd"})] 815 | #+END_SRC 816 | 817 | A good reference for the different types of queries can be found at: 818 | [[https://anmonteiro.com/2016/01/om-next-query-syntax/][Query Syntax Explained]]. 819 | 820 | If we create a server side reader and parser, we can pass this query 821 | to it and it will act almost the same as the front end. 822 | 823 | When we develop an om next backend there is a symmetry to the front 824 | end. Again we will create a reader function and create a parser with 825 | this reader function. So we pass from the transport layer, into the 826 | om-next server layer in this code: 827 | 828 | #+BEGIN_SRC clojure 829 | (defn process-data [data] 830 | (->> data 831 | read-string 832 | (router/parser {:database (be/db)}) 833 | prn-str)) 834 | #+END_SRC 835 | 836 | Particularly when we call the ~parser~ with the data we recieved. The 837 | result of calling the parser is passed back into the transport layer. 838 | 839 | 840 | 841 | -------------------------------------------------------------------------------- /writing-specs.org: -------------------------------------------------------------------------------- 1 | * About Missing Link Tutorials 2 | 3 | I find a lot of technology doesn't have documentation that I can 4 | relate to. The style of Missing Link Tutorials is to be clear, 5 | succint, easy, and to cover concepts and essential practical aspects 6 | of the topic. Without further ado... 7 | 8 | 9 | * A function special 10 | 11 | Lets say we have a function that expects a map that could be called in 12 | one of two ways: 13 | 14 | #+BEGIN_SRC clojure 15 | (authenticated {:username "abc" :password "123"}) 16 | 17 | ;; -- OR -- 18 | 19 | (authenticated {:token "1a2b3c"}) 20 | #+END_SRC 21 | 22 | First lets define a basic spec 23 | 24 | #+BEGIN_SRC clojure 25 | (s/def ::date inst?) 26 | #+END_SRC 27 | 28 | This creates a spec called ~date~ that we could use on a piece of data 29 | like so: 30 | 31 | #+BEGIN_SRC clojure 32 | 33 | #+END_SRC 34 | --------------------------------------------------------------------------------