├── DESIGN-0.11.rst └── GUIDE.rst /DESIGN-0.11.rst: -------------------------------------------------------------------------------- 1 | Duct 0.11 Design Decisions 2 | ========================== 3 | 4 | Preface 5 | ~~~~~~~ 6 | 7 | Duct 0.11 will introduce breaking changes to the project. This 8 | document covers these changes in detail, and explains the reasons they 9 | were implemented. 10 | 11 | These changes have already been implemented in the alpha version, 12 | which can be tested with:: 13 | 14 | lein new duct-alpha 15 | 16 | 17 | Modules 18 | ~~~~~~~ 19 | 20 | In 0.10 21 | """"""" 22 | 23 | In version 0.10 and below, modules were written: 24 | 25 | .. code-block:: clojure 26 | 27 | (defmulti ig/init-key ::module [_ options] 28 | {:req [::precondition] 29 | :fn (fn [config] config)}) 30 | 31 | Initiating the module key returns a map. The ``:fn`` key contains a 32 | pure function that is used to transform the configuration. The 33 | ``:req`` key contains a collection of keys the module requires to be 34 | present in the configuration. The required keys are used by Duct to 35 | apply the modules in order of their dependencies. 36 | 37 | The problem with this approach is that there is already a more 38 | sophisticated mechanism for managing dependencies in Integrant. Rather 39 | than having two separate systems for dependency ordering, it makes 40 | more sense to use the same system for both modules and normal keys. 41 | 42 | In 0.11 43 | """"""" 44 | 45 | In previous versions of Integrant, there was no way to add refs to a 46 | key beyond changing the configuration. If we want to make a module 47 | depend on another key, a way of adding refs automatically is required. 48 | 49 | Integrant 0.7 introduces the ``prep-key`` multimethod for this 50 | purpose, which is called by ``prep`` for each key before a 51 | configuration is initiated. This allows refs to be added after the 52 | configuration has been written, producing a dependency ordering. 53 | 54 | From Duct version 0.11 onwards, modules are therefore written: 55 | 56 | .. code-block:: clojure 57 | 58 | (defmulti ig/prep-key ::module [_ options] 59 | (assoc options ::requires (ig/ref ::precondition))) 60 | 61 | (defmulti ig/init-key ::module [_ options] 62 | (fn [config] config)) 63 | 64 | In the prep stage, a ref is added under a private ``::requires`` 65 | key. When the modules are initiated, this ensures that the 66 | ``::module`` key is initiated after the ``::precondition`` key. 67 | 68 | 69 | Profiles 70 | ~~~~~~~~ 71 | 72 | In 0.10 73 | """"""" 74 | 75 | In previous versions of Duct, modules and normal Integrant keys 76 | existed within the same configuration: 77 | 78 | .. code-block:: clojure 79 | 80 | {:duct.core/project-ns foo 81 | :duct.module/example {}} 82 | 83 | However, this meant that the configuration had to be split into module 84 | keys and non-module keys before it could be initiated. It also lead to 85 | problems if module keys and non-module keys shared refs. 86 | 87 | While it's convenient to have module and non-module keys in the same 88 | configuration map, it also produces a leaky abstraction. 89 | 90 | 91 | In 0.11 92 | """"""" 93 | 94 | The solution to this is to explicitly separate module keys from 95 | non-module keys. The way Duct handles this in version 0.11 is to make 96 | every key in the configuration a module. Non-module keys are placed 97 | into profiles, like so: 98 | 99 | .. code-block:: clojure 100 | 101 | {:duct.profile/base {:duct.core/project-ns foo} 102 | :duct.module/example {}} 103 | 104 | Profiles are just modules that meta-merge their value into the 105 | configuration. 106 | 107 | This results in a tiered structure with a clear separation between 108 | tiers: a Duct configuration produces an Integrant configuration, which 109 | is then used to create a running system of dependent components. 110 | 111 | To ensure that profiles are run before any other module, we need a way 112 | of defining a set of dependencies. Integrant 0.7 introduces refsets to 113 | solve this problem. Refsets act like refs, except they produce a set 114 | of all matching keys. 115 | 116 | We can use refsets to ensure that any key derived from 117 | ``:duct/module`` must be applied after a key deriving from 118 | ``:duct/profile``. In ``duct.core`` there is the following definition 119 | that does exactly that: 120 | 121 | .. code-block:: clojure 122 | 123 | (defmethod ig/prep-key :duct/module [_ profile] 124 | (assoc profile ::requires (ig/refset :duct/profile))) 125 | 126 | Between refs, refsets and keyword inheritance, we can set up 127 | sophisticated but predictable dependency graphs. 128 | 129 | 130 | Includes 131 | ~~~~~~~~ 132 | 133 | In 0.10 134 | """"""" 135 | 136 | In version 0.10 and below, includes were handled by a special key, 137 | ``:duct.core/include``: 138 | 139 | .. code-block:: clojure 140 | 141 | {:duct.core/include ["example"]} 142 | 143 | This will look for a resource named ``example.edn`` and meta-merge it 144 | into the configuration. 145 | 146 | There are two problems with this approach. 147 | 148 | The first and most obvious issue is that it requires one key to have a 149 | special function, one that isn't defined by a standard multimethod. It 150 | cannot be a module because it's side-effectful. 151 | 152 | The second issue is that it introduces new side-effects after the 153 | configuration has been read. Ideally we want reading the configuration 154 | to happen at the same step. 155 | 156 | In 0.11 157 | """"""" 158 | 159 | In version 0.11 the ``:duct.core/include`` key is replaced with the 160 | ``#duct/include`` reader tag. The tag is replaced by the contents of 161 | the referenced resource. If we want to merge it into the 162 | configuration, we place it in a profile: 163 | 164 | .. code-block:: clojure 165 | 166 | {:duct.profile/example #duct/include "example"]} 167 | 168 | This ensures all the included configurations are read together by 169 | ``read-config``, and moves the complexities of merging into the 170 | profiles. 171 | 172 | This approach also allows smaller chunks of data to be included from 173 | external files, rather than full configurations. 174 | 175 | 176 | Summary 177 | ~~~~~~~ 178 | 179 | The changes represent an overall simplification of the module and 180 | include system: 181 | 182 | - Modules and normal components are separated. 183 | - Modules no longer use their own dependency management. 184 | - Merging is separated out into profiles. 185 | - Including other configurations happens at read time. 186 | - No keys with 'special' functionality. 187 | 188 | In addition to the simplification, extra functionality has been added: 189 | 190 | - ``prep-key`` removes the need for modules in simple cases 191 | - ``refset`` allows for more sophisticated dependencies 192 | -------------------------------------------------------------------------------- /GUIDE.rst: -------------------------------------------------------------------------------- 1 | Guide to the Duct Framework 2 | =========================== 3 | 4 | Preface 5 | ~~~~~~~ 6 | 7 | Duct is a data-driven framework for writing server-side applications 8 | in the Clojure_ programming language. This guide is intended as an 9 | in-depth explanation of how to use Duct, and will focus on web 10 | applications in particular. 11 | 12 | This guide assumes you have a working knowledge of Clojure code, and 13 | that you have Leiningen_ installed. A basic understanding of Ring_ 14 | will also be useful, but not required. 15 | 16 | .. _Clojure: https://clojure.org/ 17 | .. _Leiningen: https://leiningen.org/ 18 | .. _Ring: https://github.com/ring-clojure/ring 19 | 20 | 21 | Getting Started 22 | ~~~~~~~~~~~~~~~ 23 | 24 | The most straightforward way of getting started is to use the Duct 25 | Leiningen template. Duct can be used to build many types of 26 | server-side applications, but for the purposes of this guide we'll be 27 | building a web service backed by a SQLite_ database. 28 | 29 | Creating the Project 30 | """""""""""""""""""" 31 | 32 | At the shell, run:: 33 | 34 | $ lein new duct todo +api +ataraxy +sqlite 35 | 36 | This produces the output:: 37 | 38 | Generating a new Duct project named todo... 39 | Run 'lein duct setup' in the project directory to create local config files. 40 | 41 | The parameters prefixed by ``+`` are profile hints, which are used to 42 | tell the template we want to build a web service (``+api``), using the 43 | Ataraxy_ routing library (``+ataraxy``), against a SQLite database 44 | (``+sqlite``). 45 | 46 | If you want to see what profile hints there are available, you can 47 | run:: 48 | 49 | $ lein new :show duct 50 | 51 | For now, let's change directory into the ``todo`` project that has 52 | been created:: 53 | 54 | $ cd todo 55 | 56 | Then run the local setup:: 57 | 58 | $ lein duct setup 59 | 60 | This creates four files that should be kept out of source control:: 61 | 62 | Created profiles.clj 63 | Created .dir-locals.el 64 | Created dev/resources/local.edn 65 | Created dev/src/local.clj 66 | 67 | If you're using Git_, then these files are already added to your 68 | ``.gitignore`` file. If you're using another version control system, 69 | then you'll need to manually update your ignore files. 70 | 71 | .. _SQLite: https://sqlite.org/ 72 | .. _Ataraxy: https://github.com/weavejester/ataraxy 73 | .. _Git: https://git-scm.com/ 74 | 75 | 76 | Starting the REPL 77 | """"""""""""""""" 78 | 79 | Duct development is orientated around the REPL. It's recommended that 80 | you use an editor with REPL integration, such as Cursive_, Emacs_ with 81 | CIDER_, Vim_ with `fireplace.vim`_, or Atom_ with `Proto REPL`_. 82 | However, this guide doesn't require editor integration, and the 83 | instructions will assume you're working directly from the command 84 | line. 85 | 86 | So start the REPL with:: 87 | 88 | $ lein repl 89 | 90 | At the prompt, we'll first load the development environment: 91 | 92 | .. code-block:: clojure 93 | 94 | user=> (dev) 95 | :loaded 96 | dev=> 97 | 98 | This isn't loaded automatically, as errors in the development could 99 | cause the REPL not to start. 100 | 101 | Once we're in the ``dev`` namespace we can start the application: 102 | 103 | .. code-block:: clojure 104 | 105 | dev=> (go) 106 | :duct.server.http.jetty/starting-server {:port 3000} 107 | :initiated 108 | 109 | The web server has been started on port 3000. Lets check it's running 110 | by sending it a HTTP request. This can be done from the command line 111 | with the standard curl_ or wget_ tools, but I prefer HTTPie_ for 112 | testing web services:: 113 | 114 | $ http :3000 115 | HTTP/1.1 404 Not Found 116 | Content-Length: 21 117 | Content-Type: application/json; charset=utf-8 118 | Date: Wed, 06 Dec 2017 11:27:22 GMT 119 | Server: Jetty(9.2.21.v20170120) 120 | 121 | { 122 | "error": "not-found" 123 | } 124 | 125 | We get a "not found" response, but this is expected as we've yet to 126 | add any routes to the application. 127 | 128 | .. _Cursive: https://cursive-ide.com/ 129 | .. _Emacs: https://www.gnu.org/software/emacs/ 130 | .. _CIDER: https://github.com/clojure-emacs/cider 131 | .. _Vim: http://www.vim.org/ 132 | .. _fireplace.vim: https://github.com/tpope/vim-fireplace 133 | .. _Atom: https://atom.io/ 134 | .. _Proto Repl: https://atom.io/packages/proto-repl 135 | .. _curl: https://curl.haxx.se/ 136 | .. _wget: https://www.gnu.org/software/wget/ 137 | .. _HTTPie: https://httpie.org/ 138 | 139 | 140 | Configuration 141 | ~~~~~~~~~~~~~ 142 | 143 | Duct applications are built around an edn_ configuration file. This 144 | defines the structure and dependencies of the application. In the 145 | project we're writing in this guide, the configuration is located at: 146 | ``resources/todo/config.edn``. 147 | 148 | 149 | Adding a Static Route 150 | """"""""""""""""""""" 151 | 152 | Let's take a look at the configuration file: 153 | 154 | .. code-block:: edn 155 | 156 | {:duct.profile/base 157 | {:duct.core/project-ns todo 158 | 159 | :duct.router/ataraxy 160 | {:routes {}}} 161 | 162 | :duct.profile/dev #duct/include "dev" 163 | :duct.profile/local #duct/include "local" 164 | :duct.profile/prod {} 165 | 166 | :duct.module/logging {} 167 | :duct.module.web/api {} 168 | :duct.module/sql {}} 169 | 170 | The configuration is divided into *profile* and *module* 171 | components. Profiles are where we'll store the majority of our 172 | configuration, and the base profile, ``:duct.profile/base``, is where 173 | we're going to add the majority of our configuration. 174 | 175 | 176 | We're going to begin by adding in a static index route, and to do that 177 | we're going to add to the ``:duct.router/ataraxy`` key: 178 | 179 | .. code-block:: edn 180 | 181 | :duct.router/ataraxy 182 | {:routes {[:get "/"] [:todo.handler/index]}} 183 | 184 | This connects a route ``[:get "/"]`` with a result 185 | ``[:todo.handler/index]``. As a shortcut, the Ataraxy router 186 | automatically looks for a Ring handler in the configuration with a 187 | matching name to pair with the result. We could also have written: 188 | 189 | .. code-block:: edn 190 | 191 | :duct.router/ataraxy 192 | {:routes {[:get "/"] [:todo.handler/index]} 193 | :handlers {:todo.handler/index #ig/ref :todo.handler/index}} 194 | 195 | This means the same thing. The ``:duct.router/ataraxy`` component is smart 196 | enough to connect a routing result to a handler automatically, if one 197 | is not specified. 198 | 199 | Next we need to actually add a Ring handler. This will handle an 200 | incoming HTTP request and return a HTTP response. Under the 201 | ``:duct.profile/base`` key, add in another configuration entry: 202 | 203 | .. code-block:: edn 204 | 205 | [:duct.handler.static/ok :todo.handler/index] 206 | {:body {:entries "/entries"}} 207 | 208 | Your base profile should now look like: 209 | 210 | .. code-block:: edn 211 | 212 | :duct.profile/base 213 | {:duct.core/project-ns todo 214 | 215 | :duct.router/ataraxy 216 | {:routes {[:get "/"] [:todo.handler/index]}} 217 | 218 | [:duct.handler.static/ok :todo.handler/index] 219 | {:body {:entries "/entries"}}} 220 | 221 | Notice that this time we're using a vector of two keywords as a key. 222 | In Duct parlance, this is known as a *composite key*. Composite keys 223 | inherit the properties of all the keywords contained in them; because 224 | the vector contains the key ``:duct.handler.static/ok``, it inherits 225 | the properties of a static handler. 226 | 227 | Let's apply this change to the application. Go to back to the REPL and 228 | run: 229 | 230 | .. code-block:: clojure 231 | 232 | dev=> (reset) 233 | :reloading (todo.main dev user) 234 | :resumed 235 | 236 | This reloads the configuration and any changed files. When we send a 237 | HTTP request to the web server, we now get the expected response:: 238 | 239 | $ http :3000 240 | HTTP/1.1 200 OK 241 | Content-Length: 22 242 | Content-Type: application/json; charset=utf-8 243 | Date: Wed, 06 Dec 2017 13:28:52 GMT 244 | Server: Jetty(9.2.21.v20170120) 245 | 246 | { 247 | "entries": "/entries" 248 | } 249 | 250 | .. _edn: https://github.com/edn-format/edn 251 | 252 | Adding a Database Migration 253 | """"""""""""""""""""""""""" 254 | 255 | We want to begin adding more dynamic routes, but before we can we need 256 | to create our database schema. Duct uses Ragtime_ for migrations, and 257 | each migration is defined in the configuration. 258 | 259 | Add two more keys to the base profile: 260 | 261 | .. code-block:: edn 262 | 263 | :duct.migrator/ragtime 264 | {:migrations [#ig/ref :todo.migration/create-entries]} 265 | 266 | [:duct.migrator.ragtime/sql :todo.migration/create-entries] 267 | {:up ["CREATE TABLE entries (id INTEGER PRIMARY KEY, content TEXT)"] 268 | :down ["DROP TABLE entries"]} 269 | 270 | The ``:duct.migrator/ragtime`` key contains an ordered list of 271 | migrations. Individual migrations can be defined by including 272 | ``:duct.migrator.ragtime/sql`` in a composite key. The ``:up`` and 273 | ``:down`` options contains vectors of SQL to execute; the former to 274 | apply the migration, the latter to roll it back. 275 | 276 | To apply the migration we run ``reset`` again at the REPL: 277 | 278 | .. code-block:: clojure 279 | 280 | dev=> (reset) 281 | :reloading () 282 | :duct.migrator.ragtime/applying :todo.migration/create-entries#b34248fc 283 | :resumed 284 | 285 | Suppose after applying the migration we change our mind about the 286 | schema. We could write another migration, but if we haven't committed 287 | the code or deployed it to production it's often more convenient to 288 | edit the migration we have. 289 | 290 | Let's change the migration and rename the ``content`` column to 291 | ``description``: 292 | 293 | .. code-block:: edn 294 | 295 | [:duct.migrator.ragtime/sql :todo.migration/create-entries] 296 | {:up ["CREATE TABLE entries (id INTEGER PRIMARY KEY, description TEXT)"] 297 | :down ["DROP TABLE entries"]} 298 | 299 | Then ``reset``: 300 | 301 | .. code-block:: clojure 302 | 303 | dev=> (reset) 304 | :reloading () 305 | :duct.migrator.ragtime/rolling-back :todo.migration/create-entries#b34248fc 306 | :duct.migrator.ragtime/applying :todo.migration/create-entries#5c2bb12a 307 | :resumed 308 | 309 | The old version of the migration is automatically rolled back, and the 310 | new version of the migration applied in its place. 311 | 312 | .. _Ragtime: https://github.com/weavejester/ragtime 313 | 314 | Running Database Migrations in Production 315 | """"""""""""""""""""""""""""""""""""""""" 316 | 317 | We can easily run migrations in production:: 318 | 319 | $ lein run :duct/migrator 320 | 321 | If you are using Heroku for deployment, this can be added to the 322 | release phase via your Procfile:: 323 | 324 | web: java -jar target/sstandalone.jar 325 | release: lein run :duct/migrator 326 | 327 | Adding a Query Route 328 | """""""""""""""""""" 329 | 330 | Now that we have a database table, it's time to write some routes to 331 | query it. To do this, we're going to use a library called 332 | ``duct/handler.sql``, which should be added to the ``:dependencies`` 333 | key in your ``project.clj`` file: 334 | 335 | .. code-block:: clojure 336 | 337 | [duct/handler.sql "0.4.0"] 338 | 339 | Your dependencies should now look something like: 340 | 341 | .. code-block:: clojure 342 | 343 | :dependencies [[org.clojure/clojure "1.10.0"] 344 | [duct/core "0.7.0"] 345 | [duct/handler.sql "0.4.0"] 346 | [duct/module.logging "0.4.0"] 347 | [duct/module.web "0.7.0"] 348 | [duct/module.ataraxy "0.3.0"] 349 | [duct/module.sql "0.5.0"] 350 | [org.xerial/sqlite-jdbc "3.25.2"]] 351 | 352 | Adding dependencies is one of the few times we have to restart the 353 | REPL. So first we exit: 354 | 355 | .. code-block:: clojure 356 | 357 | dev=> (exit) 358 | Bye for now! 359 | 360 | Then we restart:: 361 | 362 | $ lein repl 363 | 364 | And start the application running again: 365 | 366 | .. code-block:: clojure 367 | 368 | user=> (dev) 369 | :loaded 370 | dev=> (go) 371 | :duct.server.http.jetty/starting-server {:port 3000} 372 | :initiated 373 | 374 | We can now turn back to the project configuration. Let's start by 375 | adding a new Ataraxy route: 376 | 377 | .. code-block:: edn 378 | 379 | :duct.router/ataraxy 380 | {:routes 381 | {[:get "/"] [:todo.handler/index] 382 | [:get "/entries"] [:todo.handler.entries/list]}} 383 | 384 | As before, we'll need to create a handler for this route, which should 385 | have the key: ``:todo.handler.entries/list``. Rather than deriving 386 | from a static handler, this time we'll derive from the 387 | ``:duct.handler.sql/query`` key. 388 | 389 | Place the following in your base profile: 390 | 391 | .. code-block:: edn 392 | 393 | [:duct.handler.sql/query :todo.handler.entries/list] 394 | {:sql ["SELECT * FROM entries"]} 395 | 396 | Once the handler is defined in the configuration, we can ``reset``: 397 | 398 | .. code-block:: clojure 399 | 400 | dev=> (reset) 401 | :reloading (todo.main dev user) 402 | :resumed 403 | 404 | Then we check the route by sending a HTTP request to it:: 405 | 406 | $ http :3000/entries 407 | HTTP/1.1 200 OK 408 | Content-Length: 2 409 | Content-Type: application/json; charset=utf-8 410 | Date: Thu, 07 Dec 2017 10:13:34 GMT 411 | Server: Jetty(9.2.21.v20170120) 412 | 413 | [] 414 | 415 | We get a valid, though empty response. This makes sense, as we've yet 416 | to populate the ``entries`` table with any data. 417 | 418 | 419 | Adding an Update Route 420 | """""""""""""""""""""" 421 | 422 | Next we'd like to add a route that updates the database. Again we're 423 | going to be making use of the ``duct/handler.sql`` library, but both 424 | the route and handler are going to be more complex. 425 | 426 | First, add a new POST route: 427 | 428 | .. code-block:: edn 429 | 430 | :duct.router/ataraxy 431 | {:routes 432 | {[:get "/"] [:todo.handler/index] 433 | [:get "/entries"] [:todo.handler.entries/list] 434 | 435 | [:post "/entries" {{:keys [description]} :body-params}] 436 | [:todo.handler.entries/create description]}} 437 | 438 | The new Ataraxy route not only matches the method and URI of the 439 | request, it also destructures the request body and places the 440 | description of the todo entry into the result. 441 | 442 | When we come to write the associated handler, we need some way of 443 | getting the information from the result. Ataraxy places the result 444 | into the ``:ataraxy/result`` key on the request map, so we can 445 | destructure the request to find the description of the new entry: 446 | 447 | .. code-block:: edn 448 | 449 | [:duct.handler.sql/insert :todo.handler.entries/create] 450 | {:request {[_ description] :ataraxy/result} 451 | :sql ["INSERT INTO entries (description) VALUES (?)" description]} 452 | 453 | Next we ``reset``: 454 | 455 | .. code-block:: clojure 456 | 457 | dev=> (reset) 458 | :reloading (todo.main dev user) 459 | :resumed 460 | 461 | And test:: 462 | 463 | $ http post :3000/entries description="Write Duct guide" 464 | HTTP/1.1 201 Created 465 | Content-Length: 0 466 | Content-Type: application/octet-stream 467 | Date: Thu, 07 Dec 2017 11:29:46 GMT 468 | Server: Jetty(9.2.21.v20170120) 469 | 470 | 471 | $ http get :3000/entries 472 | HTTP/1.1 200 OK 473 | Content-Length: 43 474 | Content-Type: application/json; charset=utf-8 475 | Date: Thu, 07 Dec 2017 11:29:51 GMT 476 | Server: Jetty(9.2.21.v20170120) 477 | 478 | [ 479 | { 480 | "description": "Write Duct guide", 481 | "id": 1 482 | } 483 | ] 484 | 485 | We can now have the bare bones of a useful application. 486 | 487 | 488 | Becoming More RESTful 489 | """"""""""""""""""""" 490 | 491 | We can now GET and POST to lists of entries for our Todo application, 492 | but ideally we'd also like to DELETE particular entries as well. In 493 | order to do that, each entry needs to have a distinct URI. 494 | 495 | Let's start by adding some hypertext references to our list handler: 496 | 497 | .. code-block:: edn 498 | 499 | [:duct.handler.sql/query :todo.handler.entries/list] 500 | {:sql ["SELECT * FROM entries"] 501 | :hrefs {:href "/entries/{id}"}} 502 | 503 | The ``:hrefs`` option allows hypertext references to be added to the 504 | response using `URI templates`_. If we ``reset``: 505 | 506 | .. code-block:: clojure 507 | 508 | dev=> (reset) 509 | :reloading (todo.main dev user) 510 | :resumed 511 | 512 | And test:: 513 | 514 | $ http :3000/entries 515 | HTTP/1.1 200 OK 516 | Content-Length: 63 517 | Content-Type: application/json; charset=utf-8 518 | Date: Thu, 07 Dec 2017 21:13:20 GMT 519 | Server: Jetty(9.2.21.v20170120) 520 | 521 | [ 522 | { 523 | "description": "Write Duct guide", 524 | "href": "/entries/1", 525 | "id": 1 526 | } 527 | ] 528 | 529 | We can see that each list entry now has a new key. Let's write two new 530 | Ataraxy routes: 531 | 532 | .. code-block:: edn 533 | 534 | :duct.router/ataraxy 535 | {:routes 536 | {[:get "/"] [:todo.handler/index] 537 | [:get "/entries"] [:todo.handler.entries/list] 538 | 539 | [:post "/entries" {{:keys [description]} :body-params}] 540 | [:todo.handler.entries/create description] 541 | 542 | [:get "/entries/" id] [:todo.handler.entries/find ^int id] 543 | [:delete "/entries/" id] [:todo.handler.entries/destroy ^int id]}} 544 | 545 | These routes show how we can pull data out of the URI, and coerce it 546 | into a new type. 547 | 548 | The routes require associated handlers. As before, we'll make use of 549 | the `duct/handler.sql` library, using the `query-one` and `execute` 550 | handler types: 551 | 552 | .. code-block:: edn 553 | 554 | [:duct.handler.sql/query-one :todo.handler.entries/find] 555 | {:request {[_ id] :ataraxy/result} 556 | :sql ["SELECT * FROM entries WHERE id = ?" id] 557 | :hrefs {:href "/entries/{id}"}} 558 | 559 | [:duct.handler.sql/execute :todo.handler.entries/destroy] 560 | {:request {[_ id] :ataraxy/result} 561 | :sql ["DELETE FROM entries WHERE id = ?" id]} 562 | 563 | 564 | We also want to improve the entry creation route and give it a 565 | `Location` header to the resource it creates: 566 | 567 | .. code-block:: edn 568 | 569 | [:duct.handler.sql/insert :todo.handler.entries/create] 570 | {:request {[_ description] :ataraxy/result} 571 | :sql ["INSERT INTO entries (description) VALUES (?)" description] 572 | :location "/entries/{last_insert_rowid}"} 573 | 574 | The `last_insert_rowid` is a resultset column specific to 575 | SQLite. Other databases will return the generated row ID in different 576 | ways. 577 | 578 | With all that done we `reset`: 579 | 580 | .. code-block:: clojure 581 | 582 | dev=> (reset) 583 | :reloading () 584 | :resumed 585 | 586 | And test:: 587 | 588 | $ http :3000/entries/1 589 | HTTP/1.1 200 OK 590 | Content-Length: 61 591 | Content-Type: application/json; charset=utf-8 592 | Date: Sat, 09 Dec 2017 12:59:05 GMT 593 | Server: Jetty(9.2.21.v20170120) 594 | 595 | { 596 | "description": "Write Duct guide", 597 | "href": "/entries/1", 598 | "id": 1 599 | } 600 | 601 | $ http delete :3000/entries/1 602 | HTTP/1.1 204 No Content 603 | Content-Type: application/octet-stream 604 | Date: Sat, 09 Dec 2017 12:59:12 GMT 605 | Server: Jetty(9.2.21.v20170120) 606 | 607 | 608 | $ http :3000/entries/1 609 | HTTP/1.1 404 Not Found 610 | Content-Length: 21 611 | Content-Type: application/json; charset=utf-8 612 | Date: Sat, 09 Dec 2017 12:59:18 GMT 613 | Server: Jetty(9.2.21.v20170120) 614 | 615 | { 616 | "error": "not-found" 617 | } 618 | 619 | $ http post :3000/entries description="Continue Duct guide" 620 | HTTP/1.1 201 Created 621 | Content-Length: 0 622 | Content-Type: application/octet-stream 623 | Date: Sat, 09 Dec 2017 13:18:46 GMT 624 | Location: http://localhost:3000/entries/1 625 | Server: Jetty(9.2.21.v20170120) 626 | 627 | .. _URI templates: https://tools.ietf.org/html/rfc6570 628 | 629 | 630 | Code 631 | ~~~~ 632 | 633 | So far we've seen how the configuration can be leveraged to produce 634 | applications in Duct. This works well when our needs are modest, but 635 | for most applications we're going to have to knuckle down and write 636 | some code. 637 | 638 | While defining handlers using data has advantages, it's important not 639 | to take this too far. Treat the configuration as the skeleton of your 640 | application, and the code as the muscles and organs that drive it. 641 | 642 | 643 | Adding Users 644 | """""""""""" 645 | 646 | So far our application has been the single-user variety. Let's change 647 | that by adding a ``users`` table. First we'll add a reference to a new 648 | migration in the configuration: 649 | 650 | .. code-block:: edn 651 | 652 | :duct.migrator/ragtime 653 | {:migrations [#ig/ref :todo.migration/create-entries 654 | #ig/ref :todo.migration/create-users]} 655 | 656 | Then create the migration and add it to the base profile: 657 | 658 | .. code-block:: edn 659 | 660 | [:duct.migrator.ragtime/sql :todo.migration/create-users] 661 | {:up ["CREATE TABLE users (id INTEGER PRIMARY KEY, email TEXT UNIQUE, password TEXT)"] 662 | :down ["DROP TABLE users"]} 663 | 664 | And ``reset`` to apply the new migration: 665 | 666 | .. code-block:: clojure 667 | 668 | dev=> (reset) 669 | :reloading () 670 | :duct.migrator.ragtime/applying :todo.migration/create-users#66d6b1f8 671 | :resumed 672 | 673 | Now that we have a table to hold our users, we next need to provide a 674 | way for people to sign up to our web service. We could write a handler 675 | for this with the ``duct/handler.sql`` library, but good security 676 | practice tells us that we should avoid writing passwords directly to 677 | the database. 678 | 679 | Instead, we'll be writing our own handler function, one that secures 680 | the password with a `key derivation function`_ or KDF. To do this, we 681 | first need to introduce a new dependency to the project file: 682 | 683 | .. code-block:: clojure 684 | 685 | [buddy/buddy-hashers "1.3.0"] 686 | 687 | This is the library that we'll use to supply our KDF. Once the 688 | dependency is in place, exit the REPL: 689 | 690 | .. code-block:: clojure 691 | 692 | dev=> (exit) 693 | Bye for now! 694 | 695 | Then restart:: 696 | 697 | $ lein repl 698 | 699 | And start the application: 700 | 701 | .. code-block:: clojure 702 | 703 | user=> (dev) 704 | :loaded 705 | dev=> (go) 706 | :duct.server.http.jetty/starting-server {:port 3000} 707 | :initiated 708 | 709 | Next we want to add in an additional Ataraxy route that allows users 710 | to be created: 711 | 712 | .. code-block:: edn 713 | 714 | :duct.router/ataraxy 715 | {:routes 716 | {[:get "/"] [:todo.handler/index] 717 | [:get "/entries"] [:todo.handler.entries/list] 718 | 719 | [:post "/entries" {{:keys [description]} :body-params}] 720 | [:todo.handler.entries/create description] 721 | 722 | [:get "/entries/" id] [:todo.handler.entries/find ^int id] 723 | [:delete "/entries/" id] [:todo.handler.entries/destroy ^int id] 724 | 725 | [:post "/users" {{:keys [email password]} :body-params}] 726 | [:todo.handler.users/create email password]}} 727 | 728 | And we next write the handler configuration, adding to the base 729 | profile as before: 730 | 731 | .. code-block:: edn 732 | 733 | :todo.handler.users/create 734 | {:db #ig/ref :duct.database/sql} 735 | 736 | You'll notice that this isn't a composite key; we're not using 737 | existing functionality, but instead we're going to write our own 738 | method. 739 | 740 | You might also notice that we're also including a reference to the 741 | database. All SQL database keys in Duct inherit from 742 | ``:duct.database/sql``, so by using that key in the reference we're 743 | telling Duct to find the first available SQL database. 744 | 745 | You may wonder why the ``duct.handler.sql`` keys didn't include a 746 | database key. This is because they all inherit from the 747 | ``:duct.module.sql/requires-db`` keyword, which is a indicator to the 748 | ``:duct.module/sql`` module to automatically insert the reference. We 749 | could also do this, but for now we'll keep the reference explicit. 750 | 751 | It's now finally time to write the handler. The namespace of the 752 | keyword is ``todo.handler.users``, so we'll use that as the namespace 753 | for the code. Create a new file ``src/todo/handler/users.clj`` and add 754 | a namespace declaration: 755 | 756 | .. code-block:: clojure 757 | 758 | (ns todo.handler.users 759 | (:require [ataraxy.response :as response] 760 | [buddy.hashers :as hashers] 761 | [clojure.java.jdbc :as jdbc] 762 | duct.database.sql 763 | [integrant.core :as ig])) 764 | 765 | Naturally we need ``buddy.hashers`` for our KDF, and we need 766 | ``clojure.java.jdbc`` because we're accessing the database. The 767 | ``integrant.core`` namespace is necessary because we're writing an 768 | Integrant multimethod, but the purpose of ``ataraxy.response`` and 769 | ``duct.database.sql`` might be less obvious. 770 | 771 | Let's create the function to insert the new user into the database, 772 | and return the ID of the newly created row: 773 | 774 | .. code-block:: clojure 775 | 776 | (defprotocol Users 777 | (create-user [db email password])) 778 | 779 | (extend-protocol Users 780 | duct.database.sql.Boundary 781 | (create-user [{db :spec} email password] 782 | (let [pw-hash (hashers/derive password) 783 | results (jdbc/insert! db :users {:email email, :password pw-hash})] 784 | (-> results ffirst val)))) 785 | 786 | If you're new to Duct, you might be surprised that we're using a 787 | protocol here. Why not just write a function? Why are we writing a 788 | protocol, then implementing it against this mysterious 789 | ``duct.database.sql.Boundary`` type? 790 | 791 | The answer is that we *could* use a function, and it would certainly 792 | save us a few lines, but by using a protocol we gain the capability to 793 | mock out the database for testing or development. Duct provides an 794 | empty 'boundary' record, ``duct.database.sql.Boundary``, for this 795 | purpose. This is why we need to require the ``duct.database.sql`` 796 | namespace, or the record will not be loaded. 797 | 798 | Finally, we write the ``init-key`` method for our keyword: 799 | 800 | .. code-block:: clojure 801 | 802 | (defmethod ig/init-key ::create [_ {:keys [db]}] 803 | (fn [{[_ email password] :ataraxy/result}] 804 | (let [id (create-user db email password)] 805 | [::response/created (str "/users/" id)]))) 806 | 807 | Ataraxy allows a vector to be returned instead of the usual Ring 808 | response map. This is both a convenience, and an abstraction. Ataraxy 809 | will turn this into a ``201 Created`` response map for you. 810 | 811 | Let's ``reset``: 812 | 813 | .. code-block:: clojure 814 | 815 | dev=> (reset) 816 | :reloading (todo.main todo.handler.users dev user) 817 | :resumed 818 | 819 | Then test it out:: 820 | 821 | $ http post :3000/users email=bob@example.com password=hunter2 822 | HTTP/1.1 201 Created 823 | Content-Length: 0 824 | Content-Type: application/octet-stream 825 | Date: Mon, 11 Dec 2017 14:10:31 GMT 826 | Location: http://localhost:3000/users/1 827 | Server: Jetty(9.2.21.v20170120) 828 | 829 | We don't have any way of visualizing this information yet, so we need 830 | to take a look at the database. 831 | 832 | .. _key derivation function: https://en.wikipedia.org/wiki/Key_derivation_function 833 | 834 | 835 | Querying the Database 836 | """"""""""""""""""""" 837 | 838 | During development we likely want to query the database to ensure that 839 | the code we write is inserting the correct data. To make this process 840 | easier, we'll be adding to the ``dev`` namespace in 841 | ``dev/src/dev.clj``. 842 | 843 | First, we want to add a new require for the the ``clojure.java.jdbc`` 844 | namespace: 845 | 846 | .. code-block:: clojure 847 | 848 | [clojure.java.jdbc :as jdbc] 849 | 850 | Next we want a way of getting a database connection. Duct stores the 851 | running system in the ``system`` var during development. This allows 852 | us to write a simple function to retrieve a JDBC database spec: 853 | 854 | .. code-block:: clojure 855 | 856 | (defn db [] 857 | (-> system (ig/find-derived-1 :duct.database/sql) val :spec)) 858 | 859 | Now that we can get the database, we can add a small function to help 860 | us query it: 861 | 862 | .. code-block:: clojure 863 | 864 | (defn q [sql] 865 | (jdbc/query (db) sql)) 866 | 867 | Once these changes are made, we ``reset``: 868 | 869 | .. code-block:: clojure 870 | 871 | dev=> (reset) 872 | :reloading (dev) 873 | :resumed 874 | 875 | Then try querying our ``users`` table: 876 | 877 | .. code-block:: clojure 878 | 879 | dev=> (q "SELECT * FROM users") 880 | ({:id 1, 881 | :email "bob@example.com", 882 | :password 883 | "bcrypt+sha512$f4c1bc592ecd1869d0bf802f7c8f6e36$12$19a9ae3ed9118cb6cbfcd8c4a31aadb6b00162288b1fce50"}) 884 | 885 | That certainly looks correct. We have an ID, email and an hashed password. 886 | --------------------------------------------------------------------------------