├── .gitignore └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # Created by https://www.gitignore.io/api/vim,linux 3 | 4 | ### Linux ### 5 | *~ 6 | 7 | # temporary files which can be created if a process still has a handle open of a deleted file 8 | .fuse_hidden* 9 | 10 | # KDE directory preferences 11 | .directory 12 | 13 | # Linux trash folder which might appear on any partition or disk 14 | .Trash-* 15 | 16 | # .nfs files are created when an open file is removed but is still being accessed 17 | .nfs* 18 | 19 | ### Vim ### 20 | # swap 21 | .sw[a-p] 22 | .*.sw[a-p] 23 | # session 24 | Session.vim 25 | # temporary 26 | .netrwhist 27 | # auto-generated tag files 28 | tags 29 | 30 | 31 | # End of https://www.gitignore.io/api/vim,linux 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Running Clojure on the Raspberry Pi 2 | 3 | [![Create Commons 4 | License](https://i.creativecommons.org/l/by-sa/4.0/88x31.png)](http://creativecommons.org/licenses/by-sa/4.0/) 5 | 6 | This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 7 | International License](http://creativecommons.org/licenses/by-sa/4.0/). 8 | 9 | ## 1. Rationale 10 | 11 | Dynamic languages have gradually become prominent for building more or less 12 | smart devices. The Raspberry Pi is a perfect example of cheap but powerful 13 | hardware used for building such of devices. While Python and NodeJS are popular 14 | dynamic languages, this documentation explains the advantages of using Clojure 15 | and provides various how-to's. It aims to be accessible to people familiar with 16 | Clojure and the Raspberry Pi ecosystem while remaining beginner friendly. 17 | 18 | ### 1.1. Asynchronous programming 19 | 20 | The smarter the device and the more asynchronous its behavior will be. Clojure 21 | provides strong concurrency primitives as well as the excellent 22 | [core.async](https://clojure.org/reference/compilation) library, an 23 | implementation of [communicating sequential 24 | processes](https://en.wikipedia.org/wiki/Communicating_sequential_processes) 25 | akin to the go programming language. 26 | 27 | Frameworks such as the [Robot Operating 28 | System](https://en.wikipedia.org/wiki/Robot_Operating_System) describe the 29 | importance of loosely coupled modules talking via 30 | [pub/sub](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern). 31 | Using Clojure, arbitrarly complex programs can be written by composing modules 32 | exchanging data asynchronously. 33 | 34 | ### 1.2. Unified development 35 | 36 | Clojure targets Java as well as Javascript and code can be shared between 37 | platforms. This result in a unique experience of trully [writting once and 38 | running anywhere](https://en.wikipedia.org/wiki/Write_once,_run_anywhere). Other 39 | languages providing such multi-plaform support are not as mature yet. 40 | 41 | The process of sharing code between devices, back-end and front-end greatly 42 | simplifies data modeling. Utilities written for describing and validating data 43 | do not need to be written again in another language resulting in perfect 44 | consistency. This alone eliminates an important class of bugs besides being a 45 | valuable time saver. 46 | 47 | Clojure now ships [Spec](https://clojure.org/about/spec), a core library for 48 | describing, validading and generating data amongst other things. It provides a 49 | standard framework for data modeling. Typically, the robustness of field devices 50 | and the system they belong to is assessed by feeding test data for [fuzzy 51 | testing](https://en.wikipedia.org/wiki/Fuzzing). This is a laborious, error 52 | prone and unsatisfying process. Using Spec, data can be generated automatically, 53 | which means that not much is then needed for testing the field devices, or 54 | anything in the system, by feeding valid and unvalid data. Much bugs usually 55 | discovered in production can be uncover using this method. 56 | 57 | ## 2. Choosing and installing a JVM 58 | 59 | As of today, the three main fully-fledged Java 8 environments supported by the 60 | Raspberry Pi are : 61 | 62 | - Oracle 63 | - OpenJDK 64 | - Zulu Embedded 65 | 66 | As far as we know, OpenJDK is still not optimized and runs a lot slower than 67 | Oracle, making it unsuitable. The latter is optimized but requires a license 68 | when used for anything else than education and personal projects. Hence, our 69 | recommended choice is [Zulu 70 | Embedded](https://www.azul.com/solutions/embedded-and-the-iot/) by [Azul 71 | Systems](https://www.azul.com/). It is fully certified and offers performance 72 | parity with Oracle. More importantly, it is open source and freely downloadable. 73 | Following the open source model, the company finances the project by providing 74 | paid support. 75 | 76 | Zulu Embedded releases are available 77 | [here](https://www.azul.com/downloads/zulu-embedded/) for manual installation. 78 | 79 | The easiest way is to [add the Azul Systems debian repository and use 80 | apt](http://zulu.org/zuludocs-folder/#ZuluUserGuide/PrepareZuluPlatform/AttachAPTRepositoryUbuntuOrDebianSys.htm) 81 | : 82 | ``` 83 | $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9 84 | $ echo 'deb http://repos.azulsystems.com/debian stable main' | sudo tee /etc/apt/sources.list.d/zulu.list > /dev/null 85 | $ sudo apt update 86 | $ sudo apt install zulu-embedded-8 87 | $ java -version 88 | ``` 89 | 90 | Java 9 is net yet supported by Azul Systems but it is probably a matter of time. 91 | 92 | ## 3. Configuring IO 93 | 94 | There are no special requirements for using GPIO, I2C or SPI. Nonetheless, 95 | regardless of Clojure, the serial port behaves oddly on the Raspberry Pi 3. Many 96 | articles and posts have been written on the subject. In short, the Raspberry Pi 97 | 3 is equipped with two built-in 98 | [UART](https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter)'s 99 | for serial communication : 100 | 101 | - __/dev/ttyAMA0__ is a high performance hardware UART used by bluetooth. 102 | - __/dev/S0__ is a mini UART, much less capable, used by the serial port. 103 | 104 | Running the serial port on the mini UART will be unsuitable for many 105 | applications. The solution is to choose an alternative [device 106 | tree](https://en.wikipedia.org/wiki/Device_tree) overlay in 107 | __/boot/config.txt__. 108 | 109 | This line will swap bluetooth and the serial port, meaning bluetooth will run on 110 | mini UART, which might be enough or not at all : 111 | ``` 112 | dtoverlay=pi3-miniuart-bt 113 | ``` 114 | 115 | This line will simply switch bluetooth off and is recommended if bluetooth is 116 | not needed : 117 | ``` 118 | dtoverlay=pi3-disable-bt 119 | ``` 120 | 121 | After a reboot, the serial port will be accessible at __/dev/ttyAMA0__ just like 122 | on prior Raspberry Pi models. Read more about this matter 123 | [here](https://www.raspberrypi.org/documentation/configuration/uart.md). 124 | 125 | ## 4. Leiningen 126 | 127 | [Leiningen](https://leiningen.org/) is installed as usual. Starting a REPL is 128 | usually slow, much slower than on a desktop and the process might timeout. This 129 | is preventable by adding this option to the project file : 130 | 131 | ```clj 132 | {:repl-options {:timeout 180000}} ;; 3 minutes for example 133 | ``` 134 | 135 | Once the REPL is launched and everything is compiled, it behaves as usual. For 136 | production, projects can be compiled on a development machine rather than on the 137 | Raspberry Pi itself. 138 | 139 | ## 5. Development via SSH 140 | 141 | While it is possible to write code on the Raspberry Pi, it is probably no 142 | replacement to your desktop environment. The easiest way is to keep everything 143 | on your machine and use 144 | [SSHFS](https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh) 145 | to mount the folder where the project resides on the Raspberry Pi. This allow 146 | for launching a REPL on the device without having to keep files locally. 147 | Otherwise, it is possible to use [rsync](https://rsync.samba.org) and 148 | synchronize files everytime before starting the REPL, but it is somewhat less 149 | efficient. 150 | 151 | The REPL can be made available to the local network by typing the following 152 | (where $PORT is your favorite port) : 153 | 154 | ``` 155 | $ lein repl :start :host 0.0.0.0 :port $PORT 156 | ``` 157 | 158 | The rest is just traditional Clojure development of connecting your text 159 | editor/IDE to the REPL and doing wonders. 160 | 161 | ## 6. Optimizing uberjars 162 | 163 | For better performance and faster boot time, it is best for the user to be 164 | acquainted with the [compilation](https://clojure.org/reference/compilation) 165 | process and its various options. This is true for any environment but become 166 | more relevant for the Raspberry Pi as it is much more limited than a desktop 167 | computer or a server. 168 | 169 | ## 7. Recommended libraries 170 | 171 | ### 7.1. Miscellaneous 172 | 173 | #### 7.1.1. Logging 174 | 175 | There are a few logging libraries for Clojure but the most widely used and 176 | recommended one is [Timbre](https://github.com/ptaoussanis/timbre). It is fairly 177 | easy to write log appenders and be up and running in minutes. 178 | 179 | #### 7.1.2. Modules 180 | 181 | Complex programs are often modular and it is easier to extend modular programs. 182 | We recommend [Integrant](https://github.com/weavejester/integrant), a great 183 | micro-framework for writing modules, connecting them and managing them. 184 | 185 | ### 7.2. IO 186 | 187 | #### 7.2.1. GPIO 188 | 189 | The Raspberry Pi community often relies on either 190 | [wiringPi](http://wiringpi.com) or [PIGPIO](https://github.com/joan2937/pigpio), 191 | two common C libraries for handling GPIO lines among other things. Higher level 192 | languages often provide wrappers, specially for wiringPi. The main library for 193 | accessing wiringPi from Java is [PI4J](http://pi4j.com), although it is not 194 | actively maintained anymore. Any attempts (including our own) to provide 195 | bindings to PIGPIO resulted in an utter failure. For unknown reasons, sooner or 196 | later the JVM presents unexpected segfaults or early terminations, both when 197 | using [JNA](https://en.wikipedia.org/wiki/Java_Native_Access) and 198 | [JNI](https://en.wikipedia.org/wiki/Java_Native_Interface). 199 | 200 | In any case, those C underlying libraries directly write to /dev/mem or 201 | /dev/gpiomem. While this is fast, a number of problems arise. For instance, 202 | there is no automatic clean-up which means that if an application crashes for 203 | any reason, the state of the lines remains as is. 204 | 205 | A best approach would be to rely on a proper Linux driver. For years, driving 206 | lines has been done via the SysFS approach by exporting lines as files and then 207 | performing simple IO. Unfortunately, this was slow and presented the same kind 208 | of problems as writing directly to /dev/mem. The whole process was messy. 209 | 210 | Linux now offers a new API since version 4.8. Each GPIO chip is accessible as a 211 | char device. This new scheme is surprisingly fast, offers better support for 212 | interrupts as well as a number of other desirable features. For instance, lines 213 | need to be properly requested. A single handle can drive several lines at once 214 | and when it is released, whether on purpose or when the process terminates, 215 | lines go back to their initial state. 216 | 217 | The only java library built for this new API is 218 | [dvlopt/linux-gpio.java](https://github.com/dvlopt/linux-gpio.java). It serves 219 | as the basis for 220 | [dvlopt/linux.gpio.clj](https://github.com/dvlopt/linux.gpio.clj), a 221 | higher-level clojure wrapper. Furthermore, relying on a Linux utility means 222 | those libraries can run on any machine as they are not specific to the Raspberry 223 | Pi. 224 | 225 | #### 7.2.2. I2C 226 | 227 | [dvlopt/linux.i2c.clj](https://github.com/dvlopt/linux.i2c.clj) is the only 228 | Clojure library for talking to I2C slave devices. Here are current sub-libraries 229 | targeting specific sensors and devices : 230 | 231 | - [bme280](https://github.com/dvlopt/linux.i2c.bme280), a popular environment 232 | sensor built by Bosh. 233 | - [horter-i2hae](https://github.com/dvlopt/linux.i2c.horter-i2hae), a simple A/D 234 | converter. 235 | - [mcp342x](https://github.com/dvlopt/linux.i2c.mcp342x), a family of A/D 236 | converters. 237 | 238 | #### 7.2.3. Meter-Bus 239 | 240 | [JMbus](https://www.openmuc.org/m-bus/) is the only stable and actively 241 | maintained Java library for talking to Meter-Bus slaves, typically meters. It 242 | requires Meter-bus converters such as those provided by 243 | [Solvimus](http://www.solvimus.de/en/). 244 | 245 | [dvlopt/mbus](https://github.com/dvlopt/mbus) is a Clojure wrapper around JMbus. 246 | As of today, it supports Meter-Bus via the serial port and TCP/IP. 247 | 248 | #### 7.2.4. Serial port 249 | 250 | Talking via the serial port from the JVM has always been a bit burdensome. 251 | [Purejavacomm](https://github.com/nyholku/purejavacomm) aims to provide a 252 | multi-platform compatibility without any prior installation of native libraries. 253 | To our knowledge, [Clj-serial](https://github.com/peterschwarz/clj-serial) is 254 | the only Clojure wrapper for Purejavacomm. 255 | 256 | Historically, RXTX has been widely used for serial IO. The project is now 257 | discontinued. However, [openmuc/jrxtx](https://github.com/openmuc/jrxtx) aims to 258 | provide a replacement as well as a few improvements. The slight downside is that 259 | it requires the installation of native libraries. Because a lot of legacy 260 | projects (and a few present ones) relies on RXTX, this library might be 261 | preferable over purejavacomm. [dvlopt/rxtx](https://github.com/dvlopt/rxtx) is a 262 | simple to use clojure wrapper. 263 | 264 | #### 7.2.5. SPI 265 | 266 | Basic SPI utilities are proposed by the aforementioned PI4J library. To our 267 | knowledge, there is not any specific Java library. 268 | [dvlopt/spi](https://github.com/dvlopt/spi) is an attempt to provide bindings to 269 | the Linux API but remains a poorly documented experiment for the time being. 270 | 271 | ### 7.3. Networking 272 | 273 | #### 7.3.1. HTTP and websockets 274 | 275 | Raspberry Pies are often used for running small HTTP servers. 276 | 277 | While there are quite few libraries for HTTP servers and clients, we recommend 278 | [Aleph](https://github.com/ztellman/aleph). It is a popular choice and provides 279 | asynchronous utilities for spinning up a server or performing requests as well 280 | as for executing requests. It offers great support for 281 | [websockets](https://en.wikipedia.org/wiki/WebSocket). 282 | 283 | #### 7.3.2. MQTT 284 | 285 | [MQTT](https://en.wikipedia.org/wiki/MQTT) has become popular for internet of 286 | things projects for providing bidirectional communication organized around 287 | topics. It provides useful functionalities over something like barebone 288 | [websockets](https://en.wikipedia.org/wiki/WebSocket). 289 | 290 | The [Paho MQTT Java client](https://github.com/eclipse/paho.mqtt.java) is 291 | probably the most active and well-maintained MQTT client library in the 292 | ecosystem. 293 | 294 | [dvlopt/mqtt](https://github.com/dvlopt/mqtt) is a Clojure wrapper around the 295 | Paho library. 296 | 297 | ## 8. Recommended tools and practises 298 | 299 | ### 8.1. Writing modular programs 300 | 301 | Complex programs benefit from being modular in order to remain extensible. 302 | Networking and handling various IO's will result in plenty of asynchronicity. 303 | Core.async can provide a pub/sub event bus and modules can subscribes to the 304 | needed topics and start the necessary goroutines. The event bus would in itself 305 | be a top-level module requested by all other participating modules. 306 | 307 | However, when an integrant system is started, modules are initialized 308 | synchronous after resolving dependencies between them. If a module starts 309 | pushing values asynchronously on the event bus before other dependent modules 310 | are ready, the outcome will be data loss and inconsistency. Given this fact, 311 | modules producing values should wait a signal. In practise, besides the event 312 | bus should be a promise channel serving the purpose of signaling that the system 313 | is ready to work. This channel should deliver the signal once integrant returns, 314 | meaning that every module is ready and had the change to subscribe to the needed 315 | topics. 316 | 317 | ### 8.2. Network over 3G/4G 318 | 319 | The [Huawei e3772](https://consumer.huawei.com/en/mobile-broadband/e3372/) USB 320 | 4G dongle is recommended as it is supported by Raspbian and does not require any 321 | installation nor configuration. The only shortcoming is that it auto-disconnect 322 | after a few minutes. There is no way to disable this behaviour but it can be 323 | configured to raise the interval to two hours. A simple solution for keeping the 324 | dongle active is to setup a CRON job for pinging google or anything else within 325 | the disconnect interval. 326 | 327 | The dongle runs a webserver accessible like a typical router. Run this in the 328 | terminal and find out which address it is : 329 | 330 | ``` 331 | $ ip route 332 | ``` 333 | 334 | ### 8.3. Remote management 335 | 336 | [Ansible](https://www.ansible.com/) is an administration tool used for managing 337 | servers using declarative configuration files. It leverages SSH and hosts do not 338 | require any setup. A machine can actually manage itself and this can be used as 339 | a remote update mechanism. A Raspberry Pi can indeed be prepared for 340 | periodically fetching configuration files via FTP or by any other mean and then 341 | use Ansible on itself. An alternative similar solution would be to take 342 | advantage of 343 | [ansible-pull](http://docs.ansible.com/ansible/latest/ansible-pull.html). 344 | --------------------------------------------------------------------------------