├── dummy ├── .gitignore ├── Setup.hs ├── ChangeLog.md ├── test │ └── Spec.hs ├── app │ └── Main.hs ├── src │ └── Lib.hs ├── README.md ├── stack.yaml.lock ├── package.yaml ├── LICENSE └── stack.yaml └── README.md /dummy/.gitignore: -------------------------------------------------------------------------------- 1 | .stack-work/ 2 | dummy.cabal 3 | *~ -------------------------------------------------------------------------------- /dummy/Setup.hs: -------------------------------------------------------------------------------- 1 | import Distribution.Simple 2 | main = defaultMain 3 | -------------------------------------------------------------------------------- /dummy/ChangeLog.md: -------------------------------------------------------------------------------- 1 | # Changelog for dummy 2 | 3 | ## Unreleased changes 4 | -------------------------------------------------------------------------------- /dummy/test/Spec.hs: -------------------------------------------------------------------------------- 1 | main :: IO () 2 | main = putStrLn "Test suite not yet implemented" 3 | -------------------------------------------------------------------------------- /dummy/app/Main.hs: -------------------------------------------------------------------------------- 1 | module Main where 2 | 3 | import Lib 4 | 5 | main :: IO () 6 | main = someFunc 7 | -------------------------------------------------------------------------------- /dummy/src/Lib.hs: -------------------------------------------------------------------------------- 1 | module Lib 2 | ( someFunc 3 | ) where 4 | 5 | someFunc :: IO () 6 | someFunc = putStrLn "someFunc" 7 | -------------------------------------------------------------------------------- /dummy/README.md: -------------------------------------------------------------------------------- 1 | # dummy 2 | 3 | This is a dummy project. It does nothing. It's just for GitHub to recognize this project as Haskell-related. 4 | -------------------------------------------------------------------------------- /dummy/stack.yaml.lock: -------------------------------------------------------------------------------- 1 | # This file was autogenerated by Stack. 2 | # You should not edit this file by hand. 3 | # For more information, please see the documentation at: 4 | # https://docs.haskellstack.org/en/stable/lock_files 5 | 6 | packages: [] 7 | snapshots: 8 | - completed: 9 | size: 524789 10 | url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/14/18.yaml 11 | sha256: 646be71223e08234131c6989912e6011e01b9767bc447b6d466a35e14360bdf2 12 | original: lts-14.18 13 | -------------------------------------------------------------------------------- /dummy/package.yaml: -------------------------------------------------------------------------------- 1 | name: dummy 2 | version: 0.1.0.0 3 | github: "githubuser/dummy" 4 | license: BSD3 5 | author: "Author name here" 6 | maintainer: "example@example.com" 7 | copyright: "2019 Author name here" 8 | 9 | extra-source-files: 10 | - README.md 11 | - ChangeLog.md 12 | 13 | # Metadata used when publishing your package 14 | # synopsis: Short description of your package 15 | # category: Web 16 | 17 | # To avoid duplicated efforts in documentation and dealing with the 18 | # complications of embedding Haddock markup inside cabal files, it is 19 | # common to point users to the README.md file. 20 | description: Please see the README on GitHub at 21 | 22 | dependencies: 23 | - base >= 4.7 && < 5 24 | 25 | library: 26 | source-dirs: src 27 | 28 | executables: 29 | dummy-exe: 30 | main: Main.hs 31 | source-dirs: app 32 | ghc-options: 33 | - -threaded 34 | - -rtsopts 35 | - -with-rtsopts=-N 36 | dependencies: 37 | - dummy 38 | 39 | tests: 40 | dummy-test: 41 | main: Spec.hs 42 | source-dirs: test 43 | ghc-options: 44 | - -threaded 45 | - -rtsopts 46 | - -with-rtsopts=-N 47 | dependencies: 48 | - dummy 49 | -------------------------------------------------------------------------------- /dummy/LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Author name here (c) 2019 2 | 3 | All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are met: 7 | 8 | * Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 11 | * Redistributions in binary form must reproduce the above 12 | copyright notice, this list of conditions and the following 13 | disclaimer in the documentation and/or other materials provided 14 | with the distribution. 15 | 16 | * Neither the name of Author name here nor the names of other 17 | contributors may be used to endorse or promote products derived 18 | from this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 21 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 22 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 23 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 24 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 25 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 26 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 27 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 28 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 30 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 31 | -------------------------------------------------------------------------------- /dummy/stack.yaml: -------------------------------------------------------------------------------- 1 | # This file was automatically generated by 'stack init' 2 | # 3 | # Some commonly used options have been documented as comments in this file. 4 | # For advanced use and comprehensive documentation of the format, please see: 5 | # https://docs.haskellstack.org/en/stable/yaml_configuration/ 6 | 7 | # Resolver to choose a 'specific' stackage snapshot or a compiler version. 8 | # A snapshot resolver dictates the compiler version and the set of packages 9 | # to be used for project dependencies. For example: 10 | # 11 | # resolver: lts-3.5 12 | # resolver: nightly-2015-09-21 13 | # resolver: ghc-7.10.2 14 | # 15 | # The location of a snapshot can be provided as a file or url. Stack assumes 16 | # a snapshot provided as a file might change, whereas a url resource does not. 17 | # 18 | # resolver: ./custom-snapshot.yaml 19 | # resolver: https://example.com/snapshots/2018-01-01.yaml 20 | resolver: lts-14.18 21 | 22 | # User packages to be built. 23 | # Various formats can be used as shown in the example below. 24 | # 25 | # packages: 26 | # - some-directory 27 | # - https://example.com/foo/bar/baz-0.0.2.tar.gz 28 | # subdirs: 29 | # - auto-update 30 | # - wai 31 | packages: 32 | - . 33 | # Dependency packages to be pulled from upstream that are not in the resolver. 34 | # These entries can reference officially published versions as well as 35 | # forks / in-progress versions pinned to a git hash. For example: 36 | # 37 | # extra-deps: 38 | # - acme-missiles-0.3 39 | # - git: https://github.com/commercialhaskell/stack.git 40 | # commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a 41 | # 42 | # extra-deps: [] 43 | 44 | # Override default flag values for local packages and extra-deps 45 | # flags: {} 46 | 47 | # Extra package databases containing global packages 48 | # extra-package-dbs: [] 49 | 50 | # Control whether we use the GHC we find on the path 51 | # system-ghc: true 52 | # 53 | # Require a specific version of stack, using version ranges 54 | # require-stack-version: -any # Default 55 | # require-stack-version: ">=2.1" 56 | # 57 | # Override the architecture used by stack, especially useful on Windows 58 | # arch: i386 59 | # arch: x86_64 60 | # 61 | # Extra directories used by stack for building 62 | # extra-include-dirs: [/path/to/dir] 63 | # extra-lib-dirs: [/path/to/dir] 64 | # 65 | # Allow a newer minor version of GHC than the snapshot specifies 66 | # compiler-check: newer-minor 67 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hierarchical Free Monads: The Most Developed Approach In Haskell 2 | # _**(And The Death Of Final Tagless)**_ 3 | 4 | ### Abstract 5 | 6 | This article is for those who are searching for approaches in Haskell to create real software. The article describes the approach known as Hierarchical Free Monads (HFM), shows why it's better than other approaches, and how the approach was used to drive several businesses. The article also provides a technical perspective on HFM in comparison to Final Tagless / mtl and demonstrates how to solve typical tasks with it. Additionally, the article debunks several widespread myths about Free Monads. 7 | 8 | There are also philosophical digressions in which I'm trying to explain why Haskell is not yet as popular as it can be, and why we need all that knowledge known as "Software Engineering" to make it more spread. 9 | 10 | This article **is not for seasoned haskellers** with a strong opinion and not for those people who consider Software Engineering to be something essentially bad. **They may find I'm exaggerating and overbolizing too much** but this is just the best illustration of how I feel. 11 | 12 | ### Table of Contents 13 | 14 | * [Essay: My Long Path To Real World Haskell](#Essay-My-Long-Path-To-Real-World-Haskell) 15 | * [Philosophy: The State Of Software Design In Haskell](#Philosophy-The-State-Of-Software-Design-In-Haskell) 16 | * [Exposition: Hierarchical Free Monads For Real World](#Exposition-Hierarchical-Free-Monads-For-Real-World) 17 | * [Dive-in: Hierarchical Free Monads](#Dive-in-Hierarchical-Free-Monads) 18 | - [Nesting Free Monadic Languages](#Nesting-Free-Monadic-Languages) 19 | - [Boilerplate](#Boilerplate) 20 | - [A Unified Design](#A-Unified-Design) 21 | - [Composition And Explicit Effects](#Composition-And-Explicit-Effects) 22 | - [Extensibility](#Extensibility) 23 | - [Expression Problem](#Expression-Problem) 24 | - [Declarative Semantics](#Declarative-Semantics) 25 | - [Exception Handling](#Exception-Handling) 26 | - [Resource Management](#Resource-Management) 27 | - [Testability](#Testability) 28 | - [Performance](#Performance) 29 | * [Conclusion](#Conclusion) 30 | * [Farewell word](#Farewell-word) 31 | 32 | # Essay: My Long Path To Real World Haskell 33 | 34 | My professional path started with C++. It was a beautiful time. I was young, C++ was simple, the future seemed to be bright. I felt like I was on the crest of the wave with the most useful and powerful language out there. I was a wizard of C++, I could create any possible application with it, and I thought there was nothing I could learn because I knew everything. 35 | 36 | And then Haskell has come. 37 | 38 | I met Haskell when reading some article, and I immediately remembered I've seen this name earlier in the discussions. People talked about Haskell like something unusual, beautiful, even transcendental. I haven't paid that much attention to it until this article. And when I read it, - I didn't understand it, not a single word. But I was trapped already. 39 | 40 | Short story long; the first week with Haskell turned me into a Haskell addict. I fell in love, and the bright future with C++ became now broken. Unfortunately, it took a long eight years before I started writing in Haskell for money. 41 | 42 | But before this finally happened, during all those dark stone ages with C++, I was playing with Haskell and discovered a great drama in it. 43 | 44 | I was learning Haskell, I was advancing towards the steep curve of new concepts, and I was very curious how people write real programs in Haskell. I started asking questions, but no one seemed to be understanding me and what I'm asking about. 45 | 46 | To that moment, I've built a strong experience in Software Engineering with OOP languages, such as C++ and C#, I've learned the need of the software design principles, I had a nice understanding of how to create testable software, and even all those patterns and approaches could lead me to the final goal of real world development: a software that solves the business' problems. 47 | 48 | But it seemed haskellers weren't interested in things like this. Haskellers looked more intelligent and more smart than me because they were able to juggle crazy types, categories, lemmas and coyonedas, and they did everything to get me thinking I'm too amateurish here. Like while having those crazy types and Math, I don't need to care about that ugly mainstream nonsense anymore. What I actually need is to learn this new world of superior concepts just to have a chance to get into Heaven. I believed them, - if I'd only had a choice! 49 | 50 | It was my biggest mistake. 51 | 52 | Unfortunately, this delayed my Haskell career for several years. When I realized my mistake I began working secretly on my hobby projects in Haskell. I was unsure but it started looking like I do need the approaches to build real software in Haskell. I do need to make it testable, and the property based testing does not satisfy all my needs. I do want to make my code highly decoupled, and I do want to make it layered, with all the details hidden behind some interfaces. I really need a way to do Inversion of Control and Dependency Injection in Haskell. Am I asking too much? 53 | 54 | It became clear to me that there was the Boötes Void, The Great Nothing, the knowingly unlearnt lesson in the Haskell world. A terra incognita the community was ignoring completely. I'm not sure if it was a willful blindness or just a high focus on the more interesting academic stuff, but there was a stone, there was a mountain, and there was no person who could call himself Sisyphus. 55 | 56 | And I rolled up my sleeves. It was 2015 when I started my research of Software Engineering in Haskell, and in 2016 I wrote the first line for my future book [Functional Design and Architecture](https://graninas.com/functional-design-and-architecture-book/). A long path of reinventing engineery things in the FP setting was waiting for me ahead... To be honest, this wasn't an easy path because I placed myself into the opposition to the main dogmas of the community. It's not only the idea of having practices, patterns, methodologies and approaches was ["too simple"](https://www.simplehaskell.org/), ["juniorish"](https://www.parsonsmatt.org/2019/12/26/write_junior_code.html) and ["boring"](https://www.snoyman.com/blog/2019/11/boring-haskell-manifesto), but the very tool I found to achieve this - Free monads - has been already diminished and dogmated. I had to fight with the myths spread by influential articles from respected folks. But it was a pure luck for me because I managed to see what all others couldn't. 57 | 58 | This is how I developed a methodology I call FDD, "Functional Declarative Design". I took the raw idea of Free Monads and showed how it's suitable for Software Design in Haskell. I found a way to work with Free monads easily without going too deep into theoretical concepts. I invented an approach I call Hierarchical Free Monads which is much better than other approaches in Haskell. I wrote a whole [book](https://graninas.com/functional-design-and-architecture-book/) about it, gave many [talks](https://graninas.com/talks-eng/) and proved Free Monads are fast, reliable, simple and expressive. I created several useful frameworks and technologies with Hierarchical Free Monads. I successfully tested this approach in production, many times. 59 | 60 | And today I can say: Hierarchical Free Monads is the most developed, documented and rewarding approach in Haskell nowadays. 61 | 62 | # Philosophy: The State Of Software Design In Haskell 63 | 64 | You might have heard that the biggest thing haskellers value is Math: Category Theory, Abstract Algebra, Lambda Calculus, advanced type systems. You might have even heard the opinion that mainstream developers just aren't proficient in Math and this is why they have "Software Engineering" which is no more than just a rough substitution of it. So the discipline called "Software Engineering" is not applicable, is dirty and unwanted, they say. Why should we put our effort into methodologies and other mind flaws the outside industry has? 65 | 66 | And I'm not exaggerating. Haskellers consider their language as something special, unique, blessed and therefore standing out of those strange agreements the outer world managed to build. This is clearly not true. Although Haskell is kinda a unique language (what language is not?), it's no way different in the design space. Software Design is about how to create a working software with low risks. How to make it reliable, maintainable, satisfying requirements. And certainly, how to achieve industry's main goals: a ready product that solves real problems and can produce money. Software Engineering helps the industry to not invent things again and again, to not start from scratch, to not spend more time for curiosity than it's actually needed. There should be several well established ways to do all the usual things: web apps, backends, frontends, command line apps, machine learning apps. This makes the development much cheaper and allows for less proficient developers who can be unfamiliar with all the high-level concepts but still will be able to solve business problems. 67 | 68 | In this regard, Haskell was lacking high-level approaches and ideas composed into a complete methodology of Software Design. At least until recent years. Yes, sure, we have the mtl approach for a decade or so. We also call it Final Tagless, and it's the most popular approach to write code... But from the design point of view, it doesn't really satisfy the requirements the industry has. For example, its testability is very low, and its complexity is very high, especially if we're trying to incorporate some advanced library with massive type level tricks involved. The problem with mtl/FT is not in its idea but in how it's done. It doesn't allow you to separate layers completely, it forces you to use advanced type level features (like type classes, type equality, type families), and doesn't provide any good way to express the semantics other than just "immediate evaluation". It has many other flaws, and it's a leaky abstraction. I wouldn't recommend using FT for big codebases. 69 | 70 | Maybe, dissatisfaction by Final Tagless was the main reason why different effect systems were born. Haskellers were searching for a way to describe effects in a more explicit and more algebraic way, so they started putting effects into lists of types and then interpret this list somehow. Roughly, this is how all the extensible effects work. However, it turns out that working with such a code is no pleasure, and the bigger your codebase, the more boilerplate you have to deal with. Explicit lists of effects are maybe very granular, but this is no way an advantage because it doesn't bring any extra safety but represents a major obstacle to refactoring. This approach is unmerciful. If you made a mistake in your list of effects, or you forgot some effect, you'll end up with a huge unreadable compile error about something completely unrelated. Something about type level magic rather than about your business logic. Effect systems bring too much accidental complexity in the code; perhaps this was a reason why PureScript developers removed a similar system from the language. It turned out that [effect tracking is commercially worthless](https://degoes.net/articles/no-effect-tracking). 71 | 72 | Effect systems may look cool and interesting on the first sight. Complete correctness! Explicit declarations of effects! Mathematical foundations! Smart type level magic to play with!.. But Software Engineering is not about cool things, and we should not follow the Cool Thing Driven Development methodology if we want to keep the risks low. To our benefit, different other approaches have been discovered and investigated, and the whole idea of Software Engineering in Haskell became more respected. 73 | 74 | You might ask, what else you can use for software design in Haskell if not Final Tagless or effect systems? Well, there are different approaches out there: 75 | 76 | * Final Tagless (FT) / mtl 77 | * Effect Systems and Freer Monads 78 | * ReaderT pattern 79 | * Service Handle pattern 80 | * Free Monads 81 | * Hierarchical Free Monads (HFM) 82 | 83 | To help with that, I collected a list of materials about Software Design in Haskell and composed a comparison table on these approaches. You can also check out my talks and presentations to get a better understanding on what I'll be talking about in the next sections. And for sure, you can read my book to ensure that the complete methodology of building software in Haskell now exists. 84 | 85 | * [Software Design in Haskell (list of materials)](https://github.com/graninas/software-design-in-haskell) 86 | * [Opinionated comparison table of different approaches](https://gist.github.com/graninas/1b7961ccaedf7b5cb92417a1599fdc99) 87 | * [Functional Design and Architecture (book)](https://graninas.com/functional-design-and-architecture-book/) 88 | * [Hierarchical Free Monads and Software Design in Functional Programming (Talk)](https://youtu.be/3GKQ4ni2pS0) | [Slides](https://docs.google.com/presentation/d/1SYMIZ-LOI8Ylykz0PTxwiPuHN_02gIWh9AjJDO6xbvM/edit?usp=sharing) 89 | * [Final Tagless vs Free Monad (Talk, Rus)](https://youtu.be/u1GGqDQyGfc) | [Slides (Eng)](https://drive.google.com/open?id=1VhS8ySgk2w5RoN_l_Ar_axcE4Dzf97zLw1uuzUJQbCo) 90 | 91 | And now let me prove to you that Hierarchical Free Monads is the best approach to build real software in Haskell. 92 | 93 | # Exposition: Hierarchical Free Monads For Real World 94 | 95 | So what is the approach I call "Hierarchical Free Monads" and how it's different from regular Free Monads? Well, it's just an idea that Free Monadic languages can be nested. It's a simple idea, but it allows you to go very far in the designing of your applications. 96 | 97 | I proposed a design for several frameworks using HFM as a core technology. Usign this design, we implemented these projects: 98 | 99 | - [Juspay's PureScript Presto](https://github.com/juspay/purescript-presto) - a framework for building mobile apps using a handy eDSL. 100 | - [Juspay's PureScript Presto.Backend](https://github.com/juspay/purescript-presto-backend) - a framework for web RESTful backends. This framework was initially created by my colleagues using the design of Presto. It has the same design as Presto but also is empowered by some additional features like logging, HTTP APIs integration, KV DB, SQL DB subsystems, state handling. I participated in its further development and added a technology called [automatic whitebox testing](https://github.com/graninas/automatic-whitebox-testing-showcase). 101 | - [Juspay's EulerHS](https://github.com/juspay/euler-hs). A Haskell framework for building web services and RESTful backends. A Haskell counterpart to PureScript Presto.Backend. It has own interesting properties though, for example, support of SQLite, MySQL and Postgres out of the box. 102 | 103 | All these frameworks drive Indian financial company [Juspay](juspay.in). Its financial services are written on top of Presto, Presto.Backend and EulerHS. There are new projects completely based on EulerHS. This technology helped the company to grow up and enabled a wide adoption of PureScript and Haskell in the future. Essentially, Juspay was the first company I used my ideas in. We found that HFMs hide all the complexity of implementation details behind convenient interfaces so that the code is understandable by not only developers but by managers as well. This helped us to easily reason about the domain, the logic, the business goals and requirements. My approach also brought confidence that Haskell and PureScript are not only academic languages. 104 | 105 | - [Enecuum's Node](https://github.com/graninas/Node) - a full-fledged framework for building distributed, concurrent, multithreading apps, blockchains initially, but not only. I designed this framework for [Enecuum](https://enecuum.com/) and then we successfully created our own blockchain using it. What's important, we managed to achieve our goals less than in 4 months with a team of 4 haskellers. The framework allowed us to write fast, complicated blockchain logic easily, quickly, without bugs and with maximum confidence due to a great testability of the HFM approach. You can read more about the framework in this my article: [Building network actors with Node Framework](https://gist.github.com/graninas/9beb8df5d88dda5fa21c47ce9bcb0e16). 106 | 107 | - [Hydra](https://github.com/graninas/Hydra). This is a special story here. Hydra is not only a full-fledged Free Monadic framework to build web and CLI apps, it's also a showcase project for my book. This project is aimed to show the differences between approaches, and it currently provides 3 engines: Free Monads, Church Encoded Free Monads and Final Tagless. The framework allows you to easily write RESTful apps (with [servant](https://hackage.haskell.org/package/servant)) having SQL DB (with [beam](https://hackage.haskell.org/package/beam)), KV DB support, multithreading, with logging, concurrency etc. It also provides a testing framework for your apps. Hydra is the further development of the HFMs approach and highly resembles the design of the Node framework. 108 | 109 | When I started my book, I couldn't even imagine the ideas I developed would give a light to such an interesting project. Unfortunately, Free Monads suffer from a bad reputation nowadays. In the next section I'll explain many technical aspects and destroy several myths lurking around the approach. 110 | 111 | # Dive In: Hierarchical Free Monads 112 | 113 | ### Nesting Free Monadic Languages 114 | 115 | Consider the following two Free monadic languages: a language for logging `Logger`: 116 | 117 | ```haskell 118 | -- Algebra (interface) for the LoggerL Free monadic language with only 1 method 119 | data LoggerF next where 120 | LogMessage :: LogLevel -> Message -> (() -> next) -> LoggerF next 121 | 122 | -- Functor instance needed for the Free machinery 123 | instance Functor LoggerF where 124 | fmap f (LogMessage lvl msg next) = LogMessage lvl msg (f . next) 125 | 126 | -- Free monadic language 127 | type Logger a = Free LoggerF a 128 | ``` 129 | 130 | And a top language `App` which contains `Logger` as a sublanguage: 131 | 132 | ```haskell 133 | data AppF next where 134 | EvalLogger :: Logger () -> (() -> next) -> AppF next 135 | 136 | instance Functor AppF where 137 | fmap f (EvalLogger logAct next) = EvalLogger logAct (f . next) 138 | 139 | type App a = Free AppF a 140 | ``` 141 | 142 | This is the whole idea. You put one language into a method of another language making a hierarchy of languages. The interpreters will also follow this hierarchy structure. Let's do a short inspection. One for the top language: 143 | 144 | ```haskell 145 | -- Interpreting function 146 | interpretAppF :: AppF a -> IO a 147 | interpretAppF (EvalLogger loggerAct next) = do 148 | runLogger loggerAct -- nested interpreter call 149 | pure $ next () 150 | 151 | -- Interpreter entry point 152 | runApp :: App a -> IO a 153 | runApp = foldFree interpretAppF 154 | ``` 155 | 156 | And one for the nested language: 157 | 158 | ```haskell 159 | -- Simple console logger 160 | interpretLoggerF :: LoggerF a -> IO a 161 | interpretLoggerF (LogMessage lvl msg next) = do 162 | putStrLn msg 163 | pure $ next () 164 | 165 | runLogger :: Logger a -> IO a 166 | runLogger = foldFree interpretLoggerF 167 | ``` 168 | 169 | For an easier usage, it's better to define a convenient interface for the `App` language. You can do this with several smart constructors: 170 | 171 | ```haskell 172 | -- Log message with Info level. 173 | logInfo :: Message -> App () 174 | logInfo msg = evalLogger (logMessage Info msg) id 175 | 176 | -- Helper function to wrap LoggerF method 177 | logMessage :: Level -> Message -> Logger () 178 | logMessage lvl msg = liftF $ LogMessage lvl msg id 179 | 180 | -- Helper function to wrap AppF method 181 | evalLogger :: Logger () -> App () 182 | evalLogger logger = liftF $ EvalLogger logger id 183 | ``` 184 | 185 | Suppose we want to add a new feature - getting a random value - into this small HFM framework. It's easy. The updated language will look like this: 186 | 187 | ```haskell 188 | data AppF next where 189 | GetRandomInt :: (Int, Int) -> (Int -> next) -> AppF next 190 | EvalLogger :: Logger () -> (() -> next) -> AppF next 191 | 192 | instance Functor AppF where 193 | fmap f (GetRandomInt range next) = GetRandomInt range (f . next) 194 | fmap f (EvalLogger logAct next) = EvalLogger logAct (f . next) 195 | 196 | getRandomInt :: (Int, Int) -> App Int 197 | getRandomInt range = liftF $ GetRandomInt range id 198 | 199 | -- Updated interpreter: 200 | interpretAppF :: AppF a -> IO a 201 | interpretAppF (EvalLogger loggerAct next) = next <$> runLogger loggerAct 202 | interpretAppF (GetRandomInt range next) = next <$> randomRIO range 203 | ``` 204 | 205 | Just a line here, a line there... 206 | 207 | Stop. Wrapper functions, Functor instance, smart constructors, interpreters? So much boilerplate, huh? What's the point then? Whether this is just a waste of time and effort? Well, yes but actually no. It's not boilerplate. 208 | 209 | ### Boilerplate 210 | 211 | Boilerplate is not a code that you have a possibility to automate somehow. A code once written and no longer touched, - is not boilerplate. Boilerplate is a code you have to write many times in your day-to-day practice, or a repeating code which pursuits imaginary goals. Boilerplate makes it hard to achieve more real and more important goals and becomes a big obstacle for refactoring. This is why Functor instances, interpreters and smart constructors are not boilerplate. You don't really need to automate it (although it's possible to generate Functor instances with Template Haskell). If you are about to introduce a new method then adding a new line into the Functor instance is a matter of seconds. It's about 0.01% of your time. Instead of automating this it's better to focus on something more valuable. Tests or business logic for example. 212 | 213 | Although Free Monads require some work on the language definition and implementation levels, this approach doesn't bring extra boilerplate into the business logic layer. This is good because boilerplate in the business logic costs much more. Let's compare this for FT and HFM. 214 | 215 | Simple scenario in this small HFM framework: 216 | 217 | ```haskell 218 | printRandomFactorial :: App () 219 | printRandomFactorial = do 220 | n <- getRandomInt (1, 100) 221 | logInfo $ show $ fact n 222 | ``` 223 | 224 | The same scenario with FT: 225 | 226 | ```haskell 227 | printRandomFactorial :: (Random m, Logger m) => m () 228 | printRandomFactorial = do 229 | n <- getRandomInt (1, 100) 230 | logInfo $ show $ fact n 231 | ``` 232 | 233 | While the bodies of the two functions are pretty much the same there is a significant difference in the function definitions. FT requires you to specify a list of constraints. The more effects you have the more constraints will be there. Normally, business logic of a regular web service consists of dozens if not hundreds functions, and typing this kind of boilerplate makes coding extremely annoying. It doesn't buy anything useful. Absolute correctness is not required, and moreover, cannot be achieved. Documenting the effects doesn't make the code clearer. Fine structuring of the project has more impact on the code clearness. Good project organization will allow you to know what effects are used there by just looking into the namespace (like, `app/Product/Storage/Queries` or `app/Server/API`). List of effects/constraints is not a tool for layering, effects can contradict to the namespaces, and in general it is a redundant boilerplate for no real purpose, "just in case". 234 | 235 | It's worth to note that there is an approach for FT to overcome this (described [here](https://serokell.io/blog/tagless-final)). To avoid explicit effects list you can wrap it into a single type, adopt the `AppM` pattern and live happily... Or not. There are other problems with FT which make it very hard to live happily. Let's discuss how we can design our programs with FT and HFM. 236 | 237 | ### A Unified Design 238 | 239 | *"With FT you have freedom. You can incorporate an effect whenever you want to do it. You're not limited by someone's opinionated design. It's so easy, and so cool to compose effects. The HFM approach prohibits that. It's too rigid. All the effects should be specified in the algebra. Developer of this framework makes decisions instead of you. He limits your creativity. Are you really suggesting this? Are you really suggesting to restrict the freedom?"* 240 | 241 | Yes, I'm really suggesting this. There are several reasons. 242 | 243 | * Freedom is not free. Allowing business logic developers to incorporate arbitrary effects into arbitrary places will lead to a mess. Trusting to your developers is a good idea, but providing a unified design for the whole code is the practice one cannot refuse. Freedom of FT is too risky. Being opinionated is not bad, it's a way to decrease risks and to make the development cheaper. This is what HFM does and this is why it's very practical. 244 | * HFM provides a unified design for all the applications by default. This design is also known as [3-layered cake](https://www.parsonsmatt.org/2018/03/22/three_layer_haskell_cake.html). HFM helps to separate concerns much better than FT. Language definition, interpreters and business logic represent own layers with clear responsibilities and precise boundaries. This is naturally a way to follow many Software Design principles: Interface Segregation, Single Responsibility, low coupling / high cohesion etc. 245 | 246 | Violation of these principles seems to be embedded into FT. For example, in FT effects are usually peppered by implementation details too much. These details are coming from the native libraries and leak into the business logic. No good abstraction, no separation of concerns. Guts are exhibited and accessible for all curious people. 247 | 248 | Let's investigate a simple case. The following code contains too much info about the logger implementation: 249 | 250 | ```haskell 251 | printRandomFactorial :: (Random m, WithLog SomeLogEnvironment String m) => m () 252 | ``` 253 | 254 | This makes you think about the details a lot, but why should you? With HFM, you only know you have a logging interface (i.e, this method: `logInfo :: Message -> App ()` ), and you don't need to care about how it works under the hood. FT doesn't allow easy abstractions. HFM does it graciously. You can provide a good interface over some effect, and this interface won't expose the implementation details. Business logic will be decoupled from the raw libraries. All the native libraries will go to the interpreters. You can even substitute your implementations on the fly, - which is not easily possible with FT. 255 | 256 | There are even more reasons why FT violates core design principles. It also does a bad job in keeping complexity low. In other words, it fails the main task of Software Design. 257 | 258 | ### Composition And Explicit Effects 259 | 260 | _"FT enables simple composition of effects. Such a cool idea - composing the effects! And what about HFM? It forces you to update the framework once some new effect becomes needed. Is this even viable? Whether the composition is the essence of Functional Programming and you just threw it to the trash?"_ 261 | 262 | Another point of FT proponents is about composability. However there is no value in composition itself. As well as there is no value in lambdas, high order functions, types, type classes and other features of the language. We're here not to admire the language. We're here to solve real problems and achieve business goals, and we should be very careful in choosing the tools. 263 | 264 | There is no value composing effects. There is value in controlling effects. Specifications in the FT are very like when you place a mark `BUG: fix me!` near a bug. Do you really control it? Nope. The bug is still there. Explicit lists of effects is just a needless dancing around a landing strip in order to summon the complete correctness. 265 | 266 | ### Extensibility 267 | 268 | *"But FT is extensible. Your HFM is not. Huh?.."* 269 | 270 | True, FT is easy to extend, but by what cost? Let's see. This is how we can add an effect in FT: 271 | 272 | ```haskell 273 | -- Then: 274 | -- printRandomFactorial :: (Random m, WithLog SomeLogEnvironment String m) => m () 275 | 276 | -- Now: 277 | printRandomFactorial :: (Random m, Database m, WithLog SomeLogEnvironment String m) => m () 278 | ``` 279 | 280 | However once it's done, a lot of code should be updated. If the outer function had a call to `printRandomFactorial`, it's affected now. And all other functions up to the top of the call stack. 281 | 282 | ```haskell 283 | -- Then: 284 | -- printFactAndFib :: (Random m, WithLog SomeLogEnvironment String m) => m () 285 | 286 | -- Now: 287 | printFactAndFib :: (Random m, Database m, WithLog SomeLogEnvironment String m) => m () 288 | printFactAndFib = do 289 | printRandomFactorial 290 | printRandomFibonacci 291 | ``` 292 | 293 | Let's try to see how this will look with the `AppM` approach (pseudocode; see full description [here](https://serokell.io/blog/tagless-final)): 294 | 295 | ```haskell 296 | newtype AppM a = AppM { runAppM :: ReaderT Env IO a } 297 | deriving (Functor, Applicative, Monad, Random, WithLog SomeLogEnvironment String) 298 | 299 | class (WithLog SomeLogEnvironment String m, Random m) => Lang m 300 | instance Lang AppM 301 | 302 | printFactAndFib :: (Lang m) => m () 303 | ``` 304 | 305 | Solution works but costs too much. It brings a lot of accidental complexity and involves some extra features from the language. Besides that, throwing in new effects is still possible: 306 | 307 | ```haskell 308 | printFactAndFib :: (Lang m, MonadIO m) => m () 309 | ``` 310 | 311 | Compare these movements with a simple `App` from the Free monadic language: 312 | 313 | ```haskell 314 | printRandomFactorial :: App () 315 | printRandomFibonacci :: App () 316 | printFactAndFib :: App () 317 | ``` 318 | 319 | This code is very simple. Adding new ~~effects~~ subsystems into the language itself won't break any code. There is no overengineering in the form of smart tricks on the type level. This is why the HFM approach is better for designing software: it keeps accidental complexity low which is orders of magnitude more important then extensibility. 320 | 321 | And in case you truly need some extensibility without changing the Free monadic framework, you can add a method that makes introducing new effects outside the framework possible: 322 | 323 | ```haskell 324 | data AppF next where 325 | RunIO :: IO a -> (a -> next) -> LangF next 326 | 327 | runIO :: IO a -> Lang a 328 | runIO ioAct = liftF $ RunIO ioAct id 329 | ``` 330 | 331 | Now you can implement some additional subsystem using this method: 332 | 333 | ```haskell 334 | import qualified SQLite as SQLite 335 | 336 | runSQLiteQuery :: String -> Lang (Either Error SQLite.Rows) 337 | runSQLiteQuery query = runIO $ SQLite.runQuery query 338 | ``` 339 | 340 | True that this method is somewhat dangerous, but it's still under control. You can disable it by a config for your framework, you can add a trace message into the interpreter, you can even handle exceptions from the `ioAct`. 341 | 342 | ```haskell 343 | interpretAppF :: AppF a -> IO a 344 | interpretAppF (RunIO ioAct next) = do 345 | eResult <- try ioAct 346 | case eResult of 347 | Left (err :: SomeException) -> ... -- do something with error 348 | Right res -> pure res 349 | ``` 350 | 351 | And you know what? It's much more safe than the `MonadIO` effect (or `MonadUnliftIO`) that is barely avoided in the FT codebases. 352 | 353 | ### Expression Problem 354 | 355 | *"But you didn't solve the Expression Problem!"* 356 | 357 | Well, you're right. With FT the Expression Problem is kinda solved. But we're not paid for solving expression problems, we're paid for solving business problems. Sometimes extensibility is a requirement, and sometimes it's not. My experience shows that the number of core subsystems rarely exceeds 10. Maybe 15. Some of these subsystems can be implemented on the start, some of them are fine to add lately. You always know what you do and why you do this. You design an interface for a subsystem, you think about its usage, you plan the abstraction over impure calls, you investigate the behaviour of the native library, you test your integration, you deliver a new feature that can be used in the business logic now. This is how Software Development works. 358 | 359 | ...And BTW, the outer world is not aware about the term "Expression Problem", at all. Although haskellers love it and love to solve the Expression Problem, it's not a primary goal for industrial development. 360 | 361 | ### Declarative Semantics 362 | 363 | Interestingly, Hierarchical Free Monads allow to express a way bigger range of behavioral patterns. Besides a "traditional" imperative style (like methods `logInfo` and `getRandomInt`) it's possible to have a declarative style with syntax best suitable for a specific case. Declarativity and introspection of Free Monads opens doors into a whole new world of various semantics. 364 | 365 | To clarify what I mean, let's imagine there is a requirement to create a CLI interactive application. In Haskell, you can take an interesting library [haskeline](https://hackage.haskell.org/package/haskeline) for this, but how to incorporate it into your code without breaking the whole design? With HFM, there is a nice solution. Consider the following business logic code (this code is a part of the [Labyrinth game](https://github.com/graninas/Hydra/tree/master/app/labyrinth) from the Hydra framework): 366 | 367 | ```haskell 368 | app :: GameState -> AppL () 369 | app st = do 370 | scenario $ putStrLn "Labyrinth (aka Terra Incognita) game" 371 | 372 | cliToken <- cli (onStep st) onUnknownCommand $ do -- declaring a CLI interactive interface 373 | cmd "go up" $ makeMove st DirUp -- supported commands with handlers 374 | cmd "go down" $ makeMove st DirDown 375 | cmd "go left" $ makeMove st DirLeft 376 | cmd "go right" $ makeMove st DirRight 377 | 378 | cmd "quit" $ quit st 379 | 380 | awaitCliFinished cliToken 381 | ``` 382 | 383 | What do you see here? There is a declarative `cli` method from the framework. It states there should be an interactive subsystem with such commands. Each command is tied to a handler to be called. When the user starts this program, he will see the prompt: 384 | 385 | ``` 386 | $ stack exec labyrinth 387 | 388 | Labyrinth (aka Terra Incognita) game 389 | > go right 390 | step executed. 391 | > quit 392 | Bye-bye 393 | ``` 394 | 395 | The Hydra framework has a bit more complex hierarchical structure than the HFM language in this article. There are several layers of Free monadic languages: 396 | 397 | `AppL` <- `LangL` <- `various core effects` 398 | 399 | In the sample above, the `app` function is an entry point into the game, and the `cli` method works within the `AppL` monad. In turn, the `cli` method takes handlers operating one level down, within the `LangL` monad. Take a look at the definition: 400 | 401 | ```haskell 402 | makeMove :: GameState -> Direction -> LangL () 403 | 404 | quit :: GameState -> LangL () 405 | ``` 406 | 407 | These handlers can do all the core effects: logging, state handling, working with SQL and KV DB and so on. But it's impossible to run another `cli` subsystem because it's a responsibility of the `AppL` monad only. 408 | 409 | Another declarative subsystem in Hydra allows it to work with processes (forked flows). Again, the only `AppL` layer has a right to spawn processes (see [Meteor Counter](https://github.com/graninas/Hydra/blob/25b820af45e289a1fe5bb66e9137f0e88215b00d/app/MeteorCounter/Free.hs#L79) app): 410 | 411 | ```haskell 412 | -- Process ofr counting meteors 413 | meteorCounter :: AppState -> LangL () 414 | meteorCounter = ... 415 | 416 | -- Application definition 417 | meteorsMonitoring :: AppConfig -> AppL () 418 | meteorsMonitoring cfg = do 419 | st <- atomically $ initState cfg 420 | 421 | process $ forever $ meteorCounter st 422 | process $ forever $ withRandomDelay st $ meteorShower st NorthEast 423 | process $ forever $ withRandomDelay st $ meteorShower st NorthWest 424 | process $ forever $ withRandomDelay st $ meteorShower st SouthEast 425 | process $ forever $ withRandomDelay st $ meteorShower st SouthWest 426 | 427 | ... 428 | ``` 429 | 430 | This specific separation might be debatable, because you can't spawn threads in the `LangL` methods even if you want to. Still it's kinda possible by organizing your app so that the `AppL` layer will be waiting for signals from the `LangL` layer to spawn more processes. This makes the program even more reactive and declarative. 431 | 432 | Let me present to you one more sample of a semantics different than "just imperative evaluation". In my another hobby project, [hinteractive](https://github.com/graninas/hinteractive), an engine for interactive fiction games, Free Monads are used to achieve a kind of reactive syntax for game transitions. Check out this sample of a Zork-like game: 433 | 434 | ```haskell 435 | -- | West of House location. 436 | westOfHouse :: AGGraph () 437 | westOfHouse = graph $ 438 | with (westOfHouse' >> getInput) 439 | ~> on "open mailbox" openMailbox 440 | ~> on "read leaflet" reading 441 | /> leaf nop 442 | ``` 443 | 444 | It's hard to believe but the transition operators `~>`, `/>` (and more others: `<~>`, ``...) are just symbolic aliases for methods of a specific Free language designed for this task. The game engine is based on several Free Monadic languages interacting to each other so you could express your domain logic in a more illustrative and self-explaining way. 445 | 446 | I'm really not sure how to do similar things with FT. Proponents of FT often note that Final Tagless and Free Monads are equivalent in the Math sense. Like, you can easily convert between them and what's the deal then. But while being absolutely correct this argument misses the point completely. The difference is how we use either of the approaches in real tasks. And this difference is dramatic. 447 | 448 | ### Exception Handling 449 | 450 | *"In real tasks you said? But it's impossible to have exceptions with Free Monads! How would you even use Free Monads in production while it lacks such an important feature?!"* 451 | 452 | Well, let me share a secret with you. It's a huge and deeply rooted myth. There is no problem to work with exceptions in Free Monads. Moreover, HFM offers a much better approach that eliminates all that complexity around error handling, sync and async exceptions in Haskell. 453 | 454 | Firstly, let me show you how to incorporate exception throwing and catching into a Free Monad framework. It's very simple actually. There will be two methods, let's call them `throwException` and `runSafely`: 455 | 456 | ```haskell 457 | data LangF next where 458 | ThrowException :: forall a e next. Exception e => e -> (a -> next) -> LangF next 459 | RunSafely :: Lang a -> (Either Text a -> next) -> LangF next 460 | 461 | type Lang a = Free LangF a 462 | 463 | instance Functor AppF where 464 | fmap f (ThrowException exc next) = ThrowException exc (f . next) 465 | fmap f (RunSafely act next) = RunSafely act (f . next) 466 | 467 | throwException :: forall a e. Exception e => e -> Lang a 468 | throwException ex = liftF $ ThrowException ex id 469 | 470 | runSafely :: Lang a -> Lang (Either Text a) 471 | runSafely act = liftF $ RunSafely act id 472 | ``` 473 | 474 | Notice how we're nesting the `Lang` scenario recursively into the `RunSafely` method. And now you can throw exceptions in your scenarios as well as catching them: 475 | 476 | ```haskell 477 | data AppException = InvalidOperation Text 478 | deriving (Eq, Ord, Show, Generic, Exception) 479 | 480 | unsafeScenario :: Lang Int 481 | unsafeScenario = do 482 | val <- getRandomInt (1, 90) 483 | case () of 484 | _ | val <= 30 -> pure 0 485 | | val <= 60 -> pure val 486 | | otherwise -> throwException $ InvalidOperation "Failed with 1/3 chance" 487 | 488 | safeScenario :: Lang () 489 | safeScenario = do 490 | eVal <- runSafely unsafeScenario 491 | case eVal of 492 | Left err -> logError $ "Exception got: " <> err 493 | Right val -> logInfo $ "Value got: " <> show val 494 | ``` 495 | 496 | The main work is done by the interpreter. As there is a nested `Lang` scenario, we can run the `Lang` interpreter recursively and use `try` around it: 497 | 498 | ```haskell 499 | interpretLangF :: LangF a -> IO a 500 | interpretLangF (ThrowException exc next) = throwIO exc 501 | interpretLangF (RunSafely act next) = do 502 | eResult <- try $ runLang coreRt act 503 | pure $ next $ case eResult of 504 | Left (err :: SomeException) -> Left $ show err 505 | Right r -> Right r 506 | 507 | runLang :: Lang a -> IO a 508 | runLang = foldFree interpretLangF 509 | ``` 510 | 511 | If the nested scenario throws an exception, the latter will be caught here. It can be an exception produced with the `throwException` method, or an exception made by the standard `error` function, or even an exception coming from the interpreters of other subsystems. For example, the following code is trying to connect to a DB. It will be safe irrespective whether the implementation of the `runIO` method catches the exceptions or not: 512 | 513 | ```haskell 514 | unsafeIOScenario :: Lang () 515 | unsafeIOScenario = runIO $ do 516 | conn <- SQLite.connect sqliteCfg -- can throw 517 | SQLite.query "INSERT INTO students VALUES ('John Doe')" -- can throw 518 | error "Oops" -- throws 519 | 520 | safeScenario :: Lang () 521 | safeScenario = void $ runSafely unsafeIOScenario 522 | ``` 523 | 524 | The `runSafely` can catch all normal sync exceptions. Thanks to the great abstracting power of the Free Monad, we can wrap all the dependencies like raw DB libraries, logging libraries, networking libraries into our own languages. Making an abstracted interface to a subsystem simplifies the interaction with the subsystem but also it allows to safely handle its exceptions and turning them into values. See this sample of an HTTP API interaction language from the Hydra framework: 525 | 526 | ```haskell 527 | interpretLangF coreRt (L.CallServantAPI bUrl clientAct next) 528 | = next <$> catchAny 529 | (S.runClientM clientAct (S.mkClientEnv (coreRt ^. RLens.httpClientManager) bUrl)) 530 | (pure . Left . S.ConnectionError) 531 | ``` 532 | 533 | In your business logic scenarios, you don't see these implementation details, you just use a handy safe method that returns `Left err` on case of something went wrong: 534 | 535 | ```haskell 536 | callServantAPI :: BaseUrl -> ClientM a -> LangL (Either ClientError a) 537 | callServantAPI url cl = liftF $ CallServantAPI url cl id 538 | ``` 539 | 540 | With HFM, you get a nice separation of concerns. The code is now divided into error domains, and each part of it deals with its own type of problems. Interpreters handle native exceptions. Languages convert unsafe code into safe one. Scenarios can work with business-specific errors and exceptions. Exception and error handling is a very hard theme, and there is no reason to increase accidental complexity of the code beyond this. I am horrified by the thought that I'm one to one with this problem if I decide to use FT. The practices we elaborated (see [1](https://markkarpov.com/tutorial/exceptions.html), [2](https://www.fpcomplete.com/blog/2016/11/exceptions-best-practices-haskell)) don't seem to be simple, convenient and handy. You know, all those `MonadThrow`, `MonadMask`, `MonadCatch`, `Exception` and other clunky monsters. There is literally no good way to deal with errors in FT. All the approaches are broken by design and cannot be fixed. 541 | 542 | ### Resource Management 543 | 544 | Have you heard about a practice from mainstream development which is called **RAII**? RAII stands for _Resource Acquisition Is Initialization_. This practice is very important as it simplifies resource management significantly. In such languages as C++ or C# (or even in Python), RAII can be used to control lifetimes of file handles, memory, threads and other resources. Instead of manually managing a resource, you bind its lifetime to, for example, a lifetime of a class, and the resource will be released when the object of this class is destroyed. Another popular idea for RAII is to have a scope that is the only place where a resource can be accessed. Once the control flow leaves this scope, the resource is destroyed. 545 | 546 | The Haskell's `bracket` pattern is a form of RAII as well. I've heard that with Free Monads, it's impossible to implement such semantics. Either at all, or in a safe manner. But actually this function can be directly encoded as a method of a framework: 547 | 548 | ```haskell 549 | data LangF next where 550 | IOBracket :: IO a -> (a -> IO b) -> (a -> IO c) -> (c -> next) -> LangF next 551 | 552 | ioBracket 553 | :: IO a -- computation to run first ("acquire resource") 554 | -> (a -> IO b) -- computation to run last ("release resource") 555 | -> (a -> IO c) -- computation to run in-between 556 | -> LangL c 557 | ioBracket acq rel act = liftF $ IOBracket acq rel act id 558 | 559 | -- Interpreter 560 | interpretLangF (IOBracket acq rel act next) = next <$> bracket acq rel act 561 | ``` 562 | 563 | As it's implemented via `bracket`, it behaves similarly. Notice, the IO actions are used there. However it might be needed to run your `Lang` scenarios to work with the resource. This is finely doable: 564 | 565 | ```haskell 566 | data LangF next where 567 | WithResource :: IO a -> (a -> IO b) -> (a -> Lang c) -> (c -> next) -> LangF next 568 | 569 | interpretLangF (WithResource acq rel act next) = next <$> bracket acq rel (\r -> runLang $ act r) 570 | ``` 571 | 572 | No changes are needed if you need a kind of `bracket` within the `Lang` environment. Just encode it like this: 573 | 574 | ```haskell 575 | langBracket :: LangL a -> (a -> LangL b) -> (a -> LangL c) -> LangL c 576 | langBracket acq rel act = do 577 | r <- acq 578 | a <- act r 579 | rel r 580 | pure a 581 | ``` 582 | 583 | The code of `langBracket` isn't exactly like the `bracket` function and it's not completely safe in the event of a presence of `throwException` and `runSafely` methods. But with using these functions it's for sure possible to express all the resource handling combinators from the [Control.Exception](https://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception.html) module. 584 | 585 | One more interesting aspect of resource handling with Free Monads is closely corresponding to how we do it in the mainstream languages. The architecture based on Free Monads is divided into three layers: business logic, interface eDSLs and implementation. The latter layer includes not only interpreters but also runtime structures, resources and data hidden from the two other layers. Free Monadic scenarios should be run on top of this runtime, and it's possible to track different resources there. For example, the Hydra framework has the following runtime structure for keeping DB connections, threads, raw variables and other resources: 586 | 587 | ```haskell 588 | -- | Runtime data for core subsystems. 589 | data CoreRuntime = CoreRuntime 590 | { _rocksDBs :: R.RocksDBHandles 591 | , _loggerRuntime :: LoggerRuntime 592 | , _stateRuntime :: StateRuntime 593 | , _processRuntime :: ProcessRuntime 594 | , _sqlConns :: MVar (Map D.ConnTag D.NativeSqlConn) 595 | } 596 | 597 | -- Interpreters can use CoreRuntime to store different things. 598 | interpretLangF :: R.CoreRuntime -> L.LangF a -> IO a 599 | interpretLangF coreRt (L.EvalLogger loggerAct next) = ... 600 | 601 | runLangL :: R.CoreRuntime -> L.LangL a -> IO a 602 | runLangL coreRt = foldFree (interpretLangF coreRt) 603 | ``` 604 | 605 | Some of the runtime values drive the framework and do not appear in the business logic somehow (like the logger handler of a specific logging library), some other values have an 'avatar'-like representation in the business logic. For example, the STM subsystem in the Hydra framework is wrapped into own abstraction eDSL, the `StateL` Free Monadic language (see [here](https://github.com/graninas/Hydra/blob/master/src/Hydra/Core/State/Language.hs)). The business logic code isn't allowed to interact with the native `STM` subsystem directly, but it can use `StateVar` and the `StateL` abstraction to do this safely and under close control. With this approach, you can easily introspect your STM variables, you can print your variables at any moment of application run. You can even establish some limits for your system by tracking how many STM variables have been created. 606 | 607 | All these and several other techniques the HFM approach provides help to manage resources easily, without adding any extra accidental complexity into the business logic code. This is drastically important for big code bases within the industrial setting. 608 | 609 | Now let me ask you a philosophical question. Why do all the articles about Free Monads say that handling exceptions and managing resources is not possible or quite limited compared to FT? The cause maybe is that a theoretical reasoning about Free Monads as a Math object has nothing to do with an engineery way of solving problems. Sometimes hacky, sometimes dirty, - engineery solutions help us to achieve business' goals, and we don't really need to chase a complete Math-like correctness all the time. 610 | 611 | ### Testability 612 | 613 | Historically, Haskell developers tended to idolize property-based testing. Although this approach is good it follows the idea that there are some immanent properties you could test. This might be true for pure algorithms and small programs but once you step to the ground of usual, IO-bound applications, the property-based testing becomes less useful. It's rarely a set of algorithms. More often applications like web-services are a bunch of interactions with external services: databases, HTTP services, filesystems. Extracting some internal properties (better to say invariants) from these scenarios is not an easy task. This is why other testing approaches have been invented. Integration testing is such. 614 | 615 | It's indeed important to test applications with integration tests. These tests can spot many problems with the code and its behavior. We can even say integration tests are the most useful from others. This is true however setuping integration tests is not easy, and they can be very slow and fragile. And here is the question: how to still test the behavior and not suffer from problems with integration tests? Software Engineering has an answer: mocking. Yes, you've heard it right - mocking, - in our lovely Haskell. Even in Haskell we need the practices the mainstream development has. 616 | 617 | Here, a problem ocurred. The Final Tagless architecture seems to be very inconvenient for this. When a subsystem (or effect if you wish) is represented as a type class, mocking then effectively means there should be a special mocking type instead of a real one. Let's say there is a type class for logging and an interpreter for an actual working monad like this: 618 | 619 | ```haskell 620 | class Logger m where 621 | logInfo :: Level -> Message -> m () 622 | 623 | newtype AppM a = AppM { runAppM :: ReaderT Env IO a } 624 | deriving (Functor, Applicative, Monad, MonadIO) 625 | 626 | instance Logger AppM where 627 | logInfo _ msg = liftIO $ putStrLn msg 628 | ``` 629 | 630 | This means you should create another "base monad" class for your testing environment, which is no longer the same as the live one: 631 | 632 | ```haskell 633 | newtype TestM a = TestM (IO a) 634 | deriving (Functor, Applicative, Monad, MonadIO) 635 | 636 | instance Logger TestM where 637 | logInfo _ _ = pure () 638 | ``` 639 | 640 | It's certainly possible to build additional mechanisms for mocking and more or less fine functional testing, but the classy essence of FT doesn't allow to do this with the same level of convenience as Free Monads do. I would even say Free Monads here outperform all other approaches. 641 | 642 | Let's investigate a short sample on how to organize a small mocking framework for the `App` language. To mock `GetRandomInt` calls, we will need a container (let it be list), and a special mocking interpreter will use these values during the interpretation instead of evaluating the actual effect: 643 | 644 | ```haskell 645 | data RandomValueMocks = RandomValueMocks 646 | { curVal :: IORef Int 647 | , vals :: [Int] 648 | } 649 | 650 | interpretAppF' :: RandomValueMocks -> AppF a -> IO a 651 | interpretAppF' mocks (EvalLogger loggerAct next) = pure $ next () -- No mocking 652 | interpretAppF' mocks (GetRandomInt range next) = do 653 | idx <- readIORef (curVal mocks) -- getting the next mock 654 | writeIORef (curVal mocks) (idx + 1) 655 | pure $ next $ (vals mocks) !! idx 656 | 657 | runApp' :: RandomValueMocks -> App a -> IO a 658 | runApp' mocks = foldFree (interpretAppF' mocks) 659 | ``` 660 | 661 | Now it's possible to setup mocks in the tests: 662 | 663 | ```haskell 664 | -- Program to test 665 | getRandomFib :: App Int 666 | getRandomFib = getRandomInt (1, 100) >>= pure . fib 667 | 668 | spec :: Spec 669 | spec = describe "Functional tests" $ do 670 | it "getRandomFib should return 5 for 6th member" $ do 671 | 672 | curVal <- newIORef 0 -- creating mocks 673 | let mocks = RandomValueMocks curVal [6] 674 | 675 | result <- runApp' mocks getRandomFib -- running the test interpreter 676 | result `shouldBe` 5 677 | ``` 678 | 679 | Now you see how it's easy to substitute systems by mocking interpreters; even those methods which return something generic can be mocked. Let's say we need to mock the `runSafely` method: 680 | 681 | `runSafely :: Lang a -> Lang (Either Text a)` 682 | 683 | How would we do this when every occurrence of this method can have its own type and we literally can't place all the mocks into a single structure? Well, there is nothing bad in passing those mocks as GHC.Any's: 684 | 685 | ```haskell 686 | data RunSafelyMocks = RunSafelyMocks 687 | { curVal :: IORef Int 688 | , vals :: [GHC.Any] 689 | } 690 | 691 | interpretAppF' :: RunSafelyMocks -> AppF a -> IO a 692 | interpretAppF' mocks (RunSafely _ next) = do 693 | idx <- readIORef (curVal mocks) 694 | writeIORef (curVal mocks) (idx + 1) 695 | let any = (vals mocks) !! idx 696 | 697 | let val = unsafeCoerce any -- turning an untyped mock into an appropriate type from GHC.Any 698 | pure $ next val 699 | ``` 700 | 701 | Here, we used a kind of "dirty hack" to store mocks of different types in the same structure. While `unsafeCoerce` looks too dangerous, this code will be fine if the mocks are formed correctly. This trick is truly an engineering solution: strange, hacky but working. 702 | 703 | But you might say this is not that impressive because you can do the same with FT. True. However the fact that Free Monadic languages and interpreters are just values helps to keep the code simple, finely separated and decoupled. You can easily build a design in which you will be able to substitute any subsystem by providing another implementation (interpreter). You can even do Dependency Injection on the fly, - in contrast to Final Tagless. Free Monads do not add any extra complexity into this. 704 | 705 | Still not convinced? Then I have a killer feature, ace up my sleeve. I call it ["Automatic White-Box Testing Approach"](https://github.com/graninas/automatic-whitebox-testing-showcase). 706 | 707 | What is this? It's a way to record your Free Monadic scenario, its steps and effects, into a single list of entries. Why? Because you can replay this recording against your scenario, and the player will immediately spot what parts of the scenario have changed. Does it sound like magic? Consider the following recording you could obtain from a simple scenario: 708 | 709 | ```json 710 | // recording.json: 711 | { 712 | "entries": [ 713 | [ 714 | 0, "RunDBQueryEntry", 715 | {"jsonResult":[], "query":"SELECT * FROM students"} 716 | ], 717 | [ 718 | 1, "LogInfoEntry", 719 | {"message":"No records found."} 720 | ] 721 | ] 722 | } 723 | ``` 724 | 725 | This is just a json file you can read to see what happens in your business logic. It's also replayable. True that this approach is a kind of unit testing because it knows everything about the details, but still this approach might be helpful if you want to have a golden set of tests. Automatic tests. You don't have to write a line of code for your tests; once you have a business logic written on top of your HFM framework, - you can obtain recordings immediately. 726 | 727 | I recommend you to read my article for more info about this approach. It's closely tied with Free Monads. To be honest I don't see any good way to port this to Final Tagless. Maybe there is; but still I'm very sure Free Monads are much more testable than other design approaches. 728 | 729 | ### Performance 730 | 731 | _**"Stop it. Stop spreading Free Monads. Don't bash our anti-Free-Monad-FUD. Nothing of these benefits outweighs the fact that Free Monads are inherently slow. Very slow, extremely slow. Quadratic complexity of binding? How could this have been even considered?"**_ 732 | 733 | Performance. The last resort argument of all developers who has a focus on the details rather than on the big picture. This argument usually implies that all the cases require the best possible performance right here, right now. Developers like to argue about the absolute need of performance, but once they finish arguing, they return to work and continue writing the slow code. 734 | 735 | Fortunately, the myth about slow Free Monads is just a myth. Let me state several things: 736 | * Normal Free Monads (the `Free` type from [here](http://hackage.haskell.org/package/free-5.1.3/docs/Control-Monad-Free.html)) are indeed slow. They have O(n^2) binding of monadic chains. 737 | * Church Encoded Free Monads (the `F` type from [here](http://hackage.haskell.org/package/free-5.1.3/docs/Control-Monad-Free-Church.html)) are as fast as Final Tagless. 738 | * There are even more different Free Monads out there. For example, in PureScript, a Free Monad from the ["Reflection with No Remorse"](http://okmij.org/ftp/Haskell/zseq.pdf) paper has been implemented. This Free Monad is also fast enough to be used in production. 739 | * Normal Free Monads and Church Encoded Free Monads have a different implementation, but the interface is exactly the same. You can even change the monad without affecting your business logic. This means, even if you started from the normal Free Monad, you can move to the Church Encoded one whenever you want. 740 | * In HFM, scenarios are never a single flow of actions. Due to the hierarchical structure of a HFM framework, scenarios are more like trees of separate Free Monadic chains. This means these scenarios have own counter on how many operations in them. This smoothes the binding problem, if any. 741 | * Even normal Free Monads can be used for short scenarios. 742 | 743 | When we deal with software solving real problems, performance shouldn't be an object for theoretical reasoning. Theorethizing can't provide you the actual picture on how your code behaves. Abstract performance that is unrelated to a specific task, to a specific code, doesn't make any sense. The only measurement of truth here is the experiment. I knew this and I did some experiments for you in my Hydra framework. 744 | 745 | The following table shows comparison of the four approaches for a simple scenario. It was obtained by running `time` on a program compiled without any specific tweaks and with default GHC options. Don't mind the absolute numbers but compare the difference. 746 | 747 | |Ops cnt | FT | FreeM | ChurchM | IO 748 | ----------| ------|-------|---------|-------- 749 | |10 | 0.265 | 0.222 | 0.223 | 0.227 750 | |100 | 0.221 | 0.226 | 0.228 | 0.222 751 | |1000 | 0.227 | 0.245 | 0.223 | 0.226 752 | |10000 | 0.229 | 4.106 | 0.227 | 0.224 753 | |100000 | 0.289 | inf | 0.31 | 0.309 754 | |1000000 | 0.859 | inf | 1.134 | 0.857 755 | |10000000 | 6.384 | inf | 9.507 | 6.413 756 | |20000000 | 13.734| inf | 18.997 | 12.588 757 | |30000000 | 18.16 | inf | 28.568 | 17.76 758 | 759 | Scenario itself: 760 | 761 | ```haskell 762 | flow :: IORef Int -> L.AppL () 763 | flow ref = L.scenario $ do 764 | val' <- L.evalIO $ readIORef ref 765 | val <- L.getRandomInt (1, 100) 766 | L.evalIO $ writeIORef ref $ val' + val 767 | 768 | 769 | scenario :: Int -> R.AppRuntime -> IO () 770 | scenario ops appRt = do 771 | ref <- newIORef 0 772 | void $ R.startApp appRt (replicateM_ ops $ flow ref) 773 | val <- readIORef ref 774 | print val 775 | ``` 776 | 777 | Here, the Church Encoded Free Monad engine is a bit slower than Final Tagless, but don't mind the difference, - it starts to be significant from 1 million of operations. Are you sure your scenarios will be that long? This table says you can even use a normal Free Monad with monadic chains containing up to 10K actions. (If you're still unsure, try to measure performance yourself using the possibilities the Hydra framework provides). 778 | 779 | # Conclusion 780 | 781 | Now this is it. Hierarchical Free Monads is the most developed approach in Haskell because: 782 | 783 | * There is a whole book about this approach: ["Functional Design and Architecture"](https://graninas.com/functional-design-and-architecture-book/). 784 | * There are articles describing it in detail: 785 | - This one; 786 | - [Automatic White-Box Testing With Free Monads](https://github.com/graninas/automatic-whitebox-testing-showcase) 787 | - [Building network actors with Node Framework](https://gist.github.com/graninas/9beb8df5d88dda5fa21c47ce9bcb0e16) 788 | * There are talks on it: 789 | - [Hierarchical Free Monads and Software Design in Functional Programming (Eng)](https://www.youtube.com/watch?v=3GKQ4ni2pS0) 790 | - [Automatic Whitebox Testing with Free Monads (Eng)](https://www.youtube.com/watch?v=ciZL-adDpVQ) 791 | * There are showcase projects demonstrating all the aspects of HFM: 792 | - [Hydra framework](https://github.com/graninas/Hydra) 793 | - [hinteractive](https://github.com/graninas/hinteractive) 794 | * There are successful commercial technologies based on it: 795 | - [Node framework](https://github.com/graninas/Node) 796 | - [PureScript Presto](https://github.com/juspay/purescript-presto) 797 | - [PureScript Presto.Backend](https://github.com/juspay/purescript-presto-backend) 798 | * Finally, the most important: there are happy businesses driven by this technology to their great benefit and pleasure. 799 | 800 | This is why you should use Hierarchical Free Monads, too. 801 | 802 | ### Farewell Word 803 | 804 | The long history of programming languages shows that a language can't really survive without the industrial adoption. All the languages which were only academic toys became completely abandoned after a decade or maybe two. The languages which are refusing to interact with the mainstream are doomed to die slowly and inevitably. There is no reason for Haskell to be different here. The tasks from academia will be finished sooner or later. If we want the language to be successful and alive, adoption by the industry is the only possible way long term. 805 | 806 | But the industry is not interested in advanced Math concepts, it's not interested in cool smart things, it doesn't value curiosity as haskellers do. The only thing the industry considers important is can it have its goals achieved or not and by what cost. Unfortunately, achieving the industry's goals is where the Haskell community has a big problem. We all need to learn from the mainstream, we need to adopt methodologies, ideas and practices to show the industry that Haskell is not a toy but rather a tool able to solve real tasks and lead businesses to success. We all should be more open-minded and should not stay in our Ivory Tower. 807 | 808 | Hire me to know more, support me, subscribe to me on Twitter, ask questions. 809 | 810 | * Buy the book: [Functional Design And Architecture (Second Edition)](https://www.manning.com/books/functional-design-and-architecture) 811 | * [My consultancy work](https://graninas.com/cv-contacts/) 812 | * GitHub: [graninas](https://github.com/graninas) 813 | * Twitter: [@graninas](https://twitter.com/graninas) 814 | * [LinkedIn](https://www.linkedin.com/in/alexander-granin-46889236/) 815 | * Telegram: [@graninas](https://web.telegram.org/#/im?p=@graninas) 816 | * E-mail: [graninas@gmail.com](mailto:graninas@gmail.com) 817 | --------------------------------------------------------------------------------