├── .gitignore ├── README.md ├── build.gradle.kts ├── gradle.properties ├── gradle ├── libs.versions.toml └── wrapper │ ├── gradle-wrapper.jar │ └── gradle-wrapper.properties ├── gradlew ├── gradlew.bat ├── settings.gradle.kts └── src ├── commonMain └── kotlin │ └── com │ └── tap │ └── hlc │ ├── HLCDecodeError.kt │ ├── HLCError.kt │ ├── HybridLogicalClock.kt │ ├── NodeID.kt │ └── Timestamp.kt └── commonTest └── kotlin └── com └── tap └── hlc ├── HLCComparisonTest.kt ├── HLCEncodingTest.kt ├── HLCPersistTest.kt ├── HLCReceiveTest.kt ├── HLCSendTest.kt └── NodeIDTest.kt /.gitignore: -------------------------------------------------------------------------------- 1 | *.iml 2 | .gradle 3 | /local.properties 4 | /.idea/ 5 | .DS_Store 6 | /build 7 | /captures 8 | .externalNativeBuild -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # hybrid logical clock 2 | 3 | ![badge][badge-android] 4 | ![badge][badge-jvm] 5 | ![badge][badge-js] 6 | ![badge][badge-ios] 7 | ![badge][badge-linux] 8 | ![badge][badge-windows] 9 | ![badge][badge-mac] 10 | 11 | A Kotlin multiplatform implementation of a [hybrid logical clock](https://cse.buffalo.edu/tech-reports/2014-04.pdf) 12 | 13 | # Overview 14 | 15 | Technically speaking this is an implementation of a HULC [Hybrid Unique Logical Clock] 16 | 17 | It combines: 18 | 19 | - A system timestamp (Epoch millis) 20 | - A counter (Lamport Clock) 21 | - A unique identifier (First 16 chars of a uuid v4) 22 | 23 | To produce a unique monotonic timestamp which looks like the following when serialised 24 | 25 | `000943920000000:0000f:abcda554fcb2613b` 26 | 27 | 28 | As with all logical clocks, the clock "_ticks_"/"_tocks_" with events, by this I mean operations to advance the clock surround the exchange of information between nodes in the system. 29 | 30 | Hybrid Logical Clocks overcome issues that traditional system clock based timestamps suffer from in disributed environments, notably: 31 | 32 | - Duplicate timestamps, its entirely possible two nodes generate an event in the same second/millisecond 33 | - Drift, its easy for nodes system clocks to get out of sync, even in environments with protocols like NTP 34 | 35 | 36 | 37 | # Logical Clocks 38 | 39 | At the heart of a HLC is a counter / logical clock 40 | 41 | 42 | The most famous logical clock is a [Lamport Clock](https://en.wikipedia.org/wiki/Lamport_timestamp), Lamport clocks determine the causality of events amongst nodes in a distributed system by observing: 43 | 44 | - Events generated on a single node 45 | - The exchange (sending and receiving) of events between nodes 46 | 47 | For example in the diagram below, we know that event 6 happened after event 1 despite the fact they occurred on different nodes. 48 | 49 | ```mermaid 50 | sequenceDiagram 51 | autonumber 52 | participant A as Node_A 53 | participant B as Node_B 54 | participant C as Node_C 55 | A->>B: 56 | B->>B: 57 | B->>B: 58 | B->>B: 59 | B->>C: 60 | C->>C: 61 | 62 | ``` 63 | 64 | The algorithm behind lamport clocks is quite simple 65 | 66 | - A node increments its counter before each local event (e.g., message sending event); 67 | - When a node sends a message, it includes its counter value with the message after executing step 1; 68 | - On receiving a message, the counter of the recipient is updated, if necessary, to the greater of its current counter and the timestamp in the received message. The counter is then incremented by 1 before the message is considered received. 69 | 70 | # Hybrid Logical Clocks 71 | 72 | Whilst HLCs observe the same events as Lamport Clocks to determine causality, they use a composite of elements to create their timestamps. 73 | 74 | `000943920000000:0000f:abcda554fcb2613b` 75 | 76 | This provides all the causality assurances of logical clocks, with the human readability of system clocks and the uniqueness of uuids. 77 | 78 | 79 | 80 | 81 | # Usage 82 | 83 | ### Creation 84 | 85 | On each node that is a writer (i.e. can create events) create a clock. 86 | 87 | ```kotlin 88 | // You should only ever have one source of truth for your local hlc, so it's advisable to store it in a singleton. 89 | var local = HybridLogicalClock() 90 | ``` 91 | 92 | As the clock contains a uuid portion of its composite, you can consider the clock a datastore for the nodes identifier. 93 | 94 | To access the identifier simply call 95 | 96 | ```kotlin 97 | val nodeId = local.node 98 | ``` 99 | 100 | ### Persistance 101 | 102 | The clock must survive process death and continue where it left after reboot, you can easily serialise the clock to disk 103 | with the help of the encode/decode functions. 104 | 105 | ```kotlin 106 | val saveMeToDisk = HybridLogicalClock.encodeToString(local) 107 | ``` 108 | 109 | Decoding can produce errors and thus a Result monad is returned 110 | 111 | ```kotlin 112 | val result = HybridLogicalClock.decodeFromString(encoded) 113 | ``` 114 | ### Advancing the clock, "tick", "tock" 115 | 116 | When generating local events increment the clock, the result is your new clock and also the timestamp for that local event 117 | 118 | ```kotlin 119 | HybridLogicalClock.localTick(local).let { updated -> 120 | local = updated 121 | event.timestamp = updated 122 | } 123 | ``` 124 | 125 | When receiving events from external nodes update the clock 126 | 127 | ```kotlin 128 | val remote = event.hlc 129 | local = HybridLogicalClock.remoteTock(local, remote) 130 | ``` 131 | 132 | ### Comparing 133 | 134 | HybridLogicalClocks implement the Kotlin Comparable interface, and thus you can simply 135 | 136 | ```kotlin 137 | val winner = hlc1 > hlc2 138 | ``` 139 | 140 | ## Acknowledgments 141 | 142 | 143 | 144 | [badge-android]: http://img.shields.io/badge/-android-6EDB8D.svg?style=flat 145 | [badge-jvm]: http://img.shields.io/badge/-jvm-DB413D.svg?style=flat 146 | [badge-js]: http://img.shields.io/badge/-js-F8DB5D.svg?style=flat 147 | [badge-linux]: http://img.shields.io/badge/-linux-2D3F6C.svg?style=flat 148 | [badge-windows]: http://img.shields.io/badge/-windows-4D76CD.svg?style=flat 149 | [badge-ios]: http://img.shields.io/badge/-ios-CDCDCD.svg?style=flat 150 | [badge-mac]: http://img.shields.io/badge/-macos-111111.svg?style=flat -------------------------------------------------------------------------------- /build.gradle.kts: -------------------------------------------------------------------------------- 1 | @Suppress("DSL_SCOPE_VIOLATION") 2 | plugins { 3 | alias(libs.plugins.kotlin.multiplatform) 4 | alias(libs.plugins.kotlinter) 5 | id("maven-publish") 6 | } 7 | 8 | group = "com.tap.hlc" 9 | version = "1.1.0" 10 | 11 | kotlin { 12 | targets { 13 | jvm() 14 | } 15 | sourceSets { 16 | val commonMain by getting { 17 | dependencies { 18 | api(libs.okio.core) 19 | api(libs.kotlinx.datetime) 20 | api(libs.result) 21 | api(libs.uuid) 22 | } 23 | } 24 | 25 | val commonTest by getting { 26 | dependencies { 27 | implementation(kotlin("test")) 28 | implementation(libs.okio.fakefilesystem) 29 | } 30 | } 31 | } 32 | } -------------------------------------------------------------------------------- /gradle.properties: -------------------------------------------------------------------------------- 1 | org.gradle.caching=true 2 | org.gradle.parallel=true 3 | org.gradle.configuration-cache=true 4 | 5 | kotlin.code.style=official 6 | kotlin.mpp.stability.nowarn=true 7 | -------------------------------------------------------------------------------- /gradle/libs.versions.toml: -------------------------------------------------------------------------------- 1 | [versions] 2 | 3 | kotlin = "1.8.21" 4 | kotlinter = "3.14.0" 5 | 6 | kotlinx-coroutines = "1.7.0-RC" 7 | kotlinx-datetime = "0.4.0" 8 | kotlinx-serialization = "1.5.0" 9 | 10 | okio="3.3.0" 11 | result = "1.1.17" 12 | uuid = "0.7.0" 13 | 14 | [plugins] 15 | 16 | kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" } 17 | kotlin-kapt = { id = "org.jetbrains.kotlin.kapt", version.ref = "kotlin" } 18 | kotlin-multiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref = "kotlin" } 19 | kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlin" } 20 | kotlinter = { id = "org.jmailen.kotlinter", version.ref = "kotlinter" } 21 | 22 | [libraries] 23 | 24 | kotlinx-datetime = { module = "org.jetbrains.kotlinx:kotlinx-datetime", version.ref = "kotlinx-datetime"} 25 | kotlinx-serialization = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version.ref = "kotlinx-serialization"} 26 | okio-core = { module = "com.squareup.okio:okio", version.ref = "okio"} 27 | okio-fakefilesystem = { module = "com.squareup.okio:okio-fakefilesystem", version.ref = "okio"} 28 | result = { module = "com.michael-bull.kotlin-result:kotlin-result", version.ref = "result" } 29 | uuid = { module = "com.benasher44:uuid", version.ref = "uuid"} 30 | 31 | [bundles] 32 | 33 | -------------------------------------------------------------------------------- /gradle/wrapper/gradle-wrapper.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CharlieTap/hlc/33aba2ab35282437e1c08926421adafe7fff523f/gradle/wrapper/gradle-wrapper.jar -------------------------------------------------------------------------------- /gradle/wrapper/gradle-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionBase=GRADLE_USER_HOME 2 | distributionPath=wrapper/dists 3 | distributionUrl=https\://services.gradle.org/distributions/gradle-8.1.1-bin.zip 4 | zipStoreBase=GRADLE_USER_HOME 5 | zipStorePath=wrapper/dists -------------------------------------------------------------------------------- /gradlew: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # 4 | # Copyright © 2015-2021 the original authors. 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # https://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | ############################################################################## 20 | # 21 | # Gradle start up script for POSIX generated by Gradle. 22 | # 23 | # Important for running: 24 | # 25 | # (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is 26 | # noncompliant, but you have some other compliant shell such as ksh or 27 | # bash, then to run this script, type that shell name before the whole 28 | # command line, like: 29 | # 30 | # ksh Gradle 31 | # 32 | # Busybox and similar reduced shells will NOT work, because this script 33 | # requires all of these POSIX shell features: 34 | # * functions; 35 | # * expansions «$var», «${var}», «${var:-default}», «${var+SET}», 36 | # «${var#prefix}», «${var%suffix}», and «$( cmd )»; 37 | # * compound commands having a testable exit status, especially «case»; 38 | # * various built-in commands including «command», «set», and «ulimit». 39 | # 40 | # Important for patching: 41 | # 42 | # (2) This script targets any POSIX shell, so it avoids extensions provided 43 | # by Bash, Ksh, etc; in particular arrays are avoided. 44 | # 45 | # The "traditional" practice of packing multiple parameters into a 46 | # space-separated string is a well documented source of bugs and security 47 | # problems, so this is (mostly) avoided, by progressively accumulating 48 | # options in "$@", and eventually passing that to Java. 49 | # 50 | # Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, 51 | # and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; 52 | # see the in-line comments for details. 53 | # 54 | # There are tweaks for specific operating systems such as AIX, CygWin, 55 | # Darwin, MinGW, and NonStop. 56 | # 57 | # (3) This script is generated from the Groovy template 58 | # https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt 59 | # within the Gradle project. 60 | # 61 | # You can find Gradle at https://github.com/gradle/gradle/. 62 | # 63 | ############################################################################## 64 | 65 | # Attempt to set APP_HOME 66 | 67 | # Resolve links: $0 may be a link 68 | app_path=$0 69 | 70 | # Need this for daisy-chained symlinks. 71 | while 72 | APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path 73 | [ -h "$app_path" ] 74 | do 75 | ls=$( ls -ld "$app_path" ) 76 | link=${ls#*' -> '} 77 | case $link in #( 78 | /*) app_path=$link ;; #( 79 | *) app_path=$APP_HOME$link ;; 80 | esac 81 | done 82 | 83 | APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit 84 | 85 | APP_NAME="Gradle" 86 | APP_BASE_NAME=${0##*/} 87 | 88 | # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 89 | DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' 90 | 91 | # Use the maximum available, or set MAX_FD != -1 to use that value. 92 | MAX_FD=maximum 93 | 94 | warn () { 95 | echo "$*" 96 | } >&2 97 | 98 | die () { 99 | echo 100 | echo "$*" 101 | echo 102 | exit 1 103 | } >&2 104 | 105 | # OS specific support (must be 'true' or 'false'). 106 | cygwin=false 107 | msys=false 108 | darwin=false 109 | nonstop=false 110 | case "$( uname )" in #( 111 | CYGWIN* ) cygwin=true ;; #( 112 | Darwin* ) darwin=true ;; #( 113 | MSYS* | MINGW* ) msys=true ;; #( 114 | NONSTOP* ) nonstop=true ;; 115 | esac 116 | 117 | CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar 118 | 119 | 120 | # Determine the Java command to use to start the JVM. 121 | if [ -n "$JAVA_HOME" ] ; then 122 | if [ -x "$JAVA_HOME/jre/sh/java" ] ; then 123 | # IBM's JDK on AIX uses strange locations for the executables 124 | JAVACMD=$JAVA_HOME/jre/sh/java 125 | else 126 | JAVACMD=$JAVA_HOME/bin/java 127 | fi 128 | if [ ! -x "$JAVACMD" ] ; then 129 | die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME 130 | 131 | Please set the JAVA_HOME variable in your environment to match the 132 | location of your Java installation." 133 | fi 134 | else 135 | JAVACMD=java 136 | which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 137 | 138 | Please set the JAVA_HOME variable in your environment to match the 139 | location of your Java installation." 140 | fi 141 | 142 | # Increase the maximum file descriptors if we can. 143 | if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then 144 | case $MAX_FD in #( 145 | max*) 146 | MAX_FD=$( ulimit -H -n ) || 147 | warn "Could not query maximum file descriptor limit" 148 | esac 149 | case $MAX_FD in #( 150 | '' | soft) :;; #( 151 | *) 152 | ulimit -n "$MAX_FD" || 153 | warn "Could not set maximum file descriptor limit to $MAX_FD" 154 | esac 155 | fi 156 | 157 | # Collect all arguments for the java command, stacking in reverse order: 158 | # * args from the command line 159 | # * the main class name 160 | # * -classpath 161 | # * -D...appname settings 162 | # * --module-path (only if needed) 163 | # * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. 164 | 165 | # For Cygwin or MSYS, switch paths to Windows format before running java 166 | if "$cygwin" || "$msys" ; then 167 | APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) 168 | CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) 169 | 170 | JAVACMD=$( cygpath --unix "$JAVACMD" ) 171 | 172 | # Now convert the arguments - kludge to limit ourselves to /bin/sh 173 | for arg do 174 | if 175 | case $arg in #( 176 | -*) false ;; # don't mess with options #( 177 | /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath 178 | [ -e "$t" ] ;; #( 179 | *) false ;; 180 | esac 181 | then 182 | arg=$( cygpath --path --ignore --mixed "$arg" ) 183 | fi 184 | # Roll the args list around exactly as many times as the number of 185 | # args, so each arg winds up back in the position where it started, but 186 | # possibly modified. 187 | # 188 | # NB: a `for` loop captures its iteration list before it begins, so 189 | # changing the positional parameters here affects neither the number of 190 | # iterations, nor the values presented in `arg`. 191 | shift # remove old arg 192 | set -- "$@" "$arg" # push replacement arg 193 | done 194 | fi 195 | 196 | # Collect all arguments for the java command; 197 | # * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of 198 | # shell script including quotes and variable substitutions, so put them in 199 | # double quotes to make sure that they get re-expanded; and 200 | # * put everything else in single quotes, so that it's not re-expanded. 201 | 202 | set -- \ 203 | "-Dorg.gradle.appname=$APP_BASE_NAME" \ 204 | -classpath "$CLASSPATH" \ 205 | org.gradle.wrapper.GradleWrapperMain \ 206 | "$@" 207 | 208 | # Use "xargs" to parse quoted args. 209 | # 210 | # With -n1 it outputs one arg per line, with the quotes and backslashes removed. 211 | # 212 | # In Bash we could simply go: 213 | # 214 | # readarray ARGS < <( xargs -n1 <<<"$var" ) && 215 | # set -- "${ARGS[@]}" "$@" 216 | # 217 | # but POSIX shell has neither arrays nor command substitution, so instead we 218 | # post-process each arg (as a line of input to sed) to backslash-escape any 219 | # character that might be a shell metacharacter, then use eval to reverse 220 | # that process (while maintaining the separation between arguments), and wrap 221 | # the whole thing up as a single "set" statement. 222 | # 223 | # This will of course break if any of these variables contains a newline or 224 | # an unmatched quote. 225 | # 226 | 227 | eval "set -- $( 228 | printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | 229 | xargs -n1 | 230 | sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | 231 | tr '\n' ' ' 232 | )" '"$@"' 233 | 234 | exec "$JAVACMD" "$@" 235 | -------------------------------------------------------------------------------- /gradlew.bat: -------------------------------------------------------------------------------- 1 | @rem 2 | @rem Copyright 2015 the original author or authors. 3 | @rem 4 | @rem Licensed under the Apache License, Version 2.0 (the "License"); 5 | @rem you may not use this file except in compliance with the License. 6 | @rem You may obtain a copy of the License at 7 | @rem 8 | @rem https://www.apache.org/licenses/LICENSE-2.0 9 | @rem 10 | @rem Unless required by applicable law or agreed to in writing, software 11 | @rem distributed under the License is distributed on an "AS IS" BASIS, 12 | @rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | @rem See the License for the specific language governing permissions and 14 | @rem limitations under the License. 15 | @rem 16 | 17 | @if "%DEBUG%" == "" @echo off 18 | @rem ########################################################################## 19 | @rem 20 | @rem Gradle startup script for Windows 21 | @rem 22 | @rem ########################################################################## 23 | 24 | @rem Set local scope for the variables with windows NT shell 25 | if "%OS%"=="Windows_NT" setlocal 26 | 27 | set DIRNAME=%~dp0 28 | if "%DIRNAME%" == "" set DIRNAME=. 29 | set APP_BASE_NAME=%~n0 30 | set APP_HOME=%DIRNAME% 31 | 32 | @rem Resolve any "." and ".." in APP_HOME to make it shorter. 33 | for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi 34 | 35 | @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 36 | set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" 37 | 38 | @rem Find java.exe 39 | if defined JAVA_HOME goto findJavaFromJavaHome 40 | 41 | set JAVA_EXE=java.exe 42 | %JAVA_EXE% -version >NUL 2>&1 43 | if "%ERRORLEVEL%" == "0" goto execute 44 | 45 | echo. 46 | echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 47 | echo. 48 | echo Please set the JAVA_HOME variable in your environment to match the 49 | echo location of your Java installation. 50 | 51 | goto fail 52 | 53 | :findJavaFromJavaHome 54 | set JAVA_HOME=%JAVA_HOME:"=% 55 | set JAVA_EXE=%JAVA_HOME%/bin/java.exe 56 | 57 | if exist "%JAVA_EXE%" goto execute 58 | 59 | echo. 60 | echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 61 | echo. 62 | echo Please set the JAVA_HOME variable in your environment to match the 63 | echo location of your Java installation. 64 | 65 | goto fail 66 | 67 | :execute 68 | @rem Setup the command line 69 | 70 | set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar 71 | 72 | 73 | @rem Execute Gradle 74 | "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* 75 | 76 | :end 77 | @rem End local scope for the variables with windows NT shell 78 | if "%ERRORLEVEL%"=="0" goto mainEnd 79 | 80 | :fail 81 | rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of 82 | rem the _cmd.exe /c_ return code! 83 | if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 84 | exit /b 1 85 | 86 | :mainEnd 87 | if "%OS%"=="Windows_NT" endlocal 88 | 89 | :omega 90 | -------------------------------------------------------------------------------- /settings.gradle.kts: -------------------------------------------------------------------------------- 1 | pluginManagement { 2 | repositories { 3 | gradlePluginPortal() 4 | google() 5 | mavenCentral() 6 | } 7 | } 8 | 9 | dependencyResolutionManagement { 10 | repositories { 11 | google() 12 | mavenCentral() 13 | maven(url = "https://jitpack.io" ) 14 | 15 | } 16 | } 17 | 18 | rootProject.name = "hlc" 19 | -------------------------------------------------------------------------------- /src/commonMain/kotlin/com/tap/hlc/HLCDecodeError.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | sealed interface HLCDecodeError { 4 | data class TimestampDecodeFailure(val encodedClock: String) : HLCDecodeError 5 | data class CounterDecodeFailure(val encodedClock: String) : HLCDecodeError 6 | data class NodeDecodeFailure(val encodedClock: String) : HLCDecodeError 7 | } 8 | -------------------------------------------------------------------------------- /src/commonMain/kotlin/com/tap/hlc/HLCError.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | sealed interface HLCError { 4 | data class DuplicateNodeError(val nodeID: NodeID) : HLCError 5 | data class ClockDriftError(val local: Timestamp, val now: Timestamp) : HLCError 6 | object CausalityOverflowError : HLCError 7 | } 8 | -------------------------------------------------------------------------------- /src/commonMain/kotlin/com/tap/hlc/HybridLogicalClock.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.github.michaelbull.result.Err 4 | import com.github.michaelbull.result.Ok 5 | import com.github.michaelbull.result.Result 6 | import com.github.michaelbull.result.flatMap 7 | import com.github.michaelbull.result.getOr 8 | import kotlinx.datetime.Clock 9 | import okio.FileSystem 10 | import okio.Path 11 | import kotlin.math.abs 12 | import kotlin.math.max 13 | import kotlin.math.pow 14 | 15 | /** 16 | * Implementation of a HLC [1][2] 17 | * 18 | * Largely a rip of Jared Forsyth's impl[3] with some kotlinisms 19 | * 20 | * [1]: https://cse.buffalo.edu/tech-reports/2014-04.pdf 21 | * [2]: https://muratbuffalo.blogspot.com/2014/07/hybrid-logical-clocks.html 22 | * [3]: https://jaredforsyth.com/posts/hybrid-logical-clocks/ 23 | */ 24 | data class HybridLogicalClock( 25 | val timestamp: Timestamp = Timestamp.now(Clock.System), 26 | val node: NodeID = NodeID.mint(), 27 | val counter: Int = 0, 28 | ) : Comparable { 29 | 30 | companion object { 31 | 32 | private const val CLOCK_FILE = "clock.hlc" 33 | 34 | /** 35 | * This should be called every time a new event is generated locally, the result becomes the events timestamp and the new local time 36 | */ 37 | fun localTick( 38 | local: HybridLogicalClock, 39 | wallClockTime: Timestamp = Timestamp.now(Clock.System), 40 | maxClockDrift: Int = 1000 * 60, 41 | ): Result { 42 | return if (wallClockTime.epochMillis > local.timestamp.epochMillis) { 43 | Ok(local.copy(timestamp = wallClockTime)) 44 | } else { 45 | Ok(local.copy(counter = local.counter + 1)).flatMap { clock -> 46 | validate(clock, wallClockTime, maxClockDrift) 47 | } 48 | } 49 | } 50 | 51 | /** 52 | * This should be called every time a new event is received from a remote node, the result becomes the new local time 53 | */ 54 | fun remoteTock( 55 | local: HybridLogicalClock, 56 | remote: HybridLogicalClock, 57 | wallClockTime: Timestamp = Timestamp.now(Clock.System), 58 | maxClockDrift: Int = 1000 * 60, 59 | ): Result { 60 | return when { 61 | local.node.identifier == remote.node.identifier -> { 62 | Err(HLCError.DuplicateNodeError(local.node)) 63 | } 64 | wallClockTime.epochMillis > local.timestamp.epochMillis && 65 | wallClockTime.epochMillis > remote.timestamp.epochMillis -> { 66 | Ok(local.copy(timestamp = wallClockTime, counter = 0)) 67 | } 68 | local.timestamp.epochMillis == remote.timestamp.epochMillis -> { 69 | Ok(local.copy(counter = max(local.counter, remote.counter) + 1)) 70 | } 71 | local.timestamp.epochMillis > remote.timestamp.epochMillis -> { 72 | Ok(local.copy(counter = local.counter + 1)) 73 | } 74 | else -> { 75 | Ok(local.copy(timestamp = remote.timestamp, counter = remote.counter + 1)) 76 | } 77 | }.flatMap { clock -> 78 | validate(clock, wallClockTime, maxClockDrift) 79 | } 80 | } 81 | 82 | private fun validate(clock: HybridLogicalClock, now: Timestamp, maxClockDrift: Int): Result { 83 | if (clock.counter > 36f.pow(5).toInt()) { 84 | return Err(HLCError.CausalityOverflowError) 85 | } 86 | 87 | if (abs(clock.timestamp.epochMillis - now.epochMillis) > maxClockDrift) { 88 | return Err(HLCError.ClockDriftError(clock.timestamp, now)) 89 | } 90 | 91 | return Ok(clock) 92 | } 93 | 94 | fun encodeToString(hlc: HybridLogicalClock): String { 95 | return with(hlc) { 96 | "${timestamp.epochMillis.toString().padStart(15, '0')}:${counter.toString(36).padStart(5, '0')}:${node.identifier}" 97 | } 98 | } 99 | 100 | fun decodeFromString(encoded: String): Result { 101 | val parts = encoded.split(":") 102 | 103 | if (parts.size < 3) return Err(HLCDecodeError.TimestampDecodeFailure(encoded)) 104 | 105 | val timestamp = parts.firstOrNull()?.let { 106 | Timestamp(it.toLong()) 107 | } ?: return Err(HLCDecodeError.TimestampDecodeFailure(encoded)) 108 | 109 | val counter = parts.getOrNull(1)?.toInt(36) ?: return Err(HLCDecodeError.CounterDecodeFailure(encoded)) 110 | val node = parts.getOrNull(2)?.let { NodeID(it) } ?: return Err(HLCDecodeError.NodeDecodeFailure(encoded)) 111 | 112 | return Ok(HybridLogicalClock(timestamp, node, counter)) 113 | } 114 | 115 | /** 116 | * Persists the clock to a disk file at the specified directory path 117 | * 118 | * This call is blocking 119 | * 120 | * Usage: 121 | * val directory = "/Users/alice".toPath() 122 | * HybridLogicalClock.store(hlc, path) 123 | */ 124 | fun store(hlc: HybridLogicalClock, directory: Path, fileSystem: FileSystem = FileSystem.SYSTEM, fileName: String = CLOCK_FILE) { 125 | fileSystem.createDirectories(directory) 126 | val filepath = directory / fileName 127 | fileSystem.write(filepath) { 128 | writeUtf8(hlc.toString()) 129 | } 130 | } 131 | 132 | /** 133 | * Attempts to load a clock from a disk file at the specified directory path 134 | * 135 | * This call is blocking and will return null if no file is found 136 | * 137 | * Usage: 138 | * val directory = "/Users/alice".toPath() 139 | * val nullableClock = HybridLogicalClock.load(path) 140 | */ 141 | fun load(directory: Path, fileSystem: FileSystem = FileSystem.SYSTEM, fileName: String = CLOCK_FILE): HybridLogicalClock? { 142 | val filepath = directory / fileName 143 | if (!fileSystem.exists(filepath)) { 144 | return null 145 | } 146 | val encoded = fileSystem.read(filepath) { readUtf8() } 147 | return decodeFromString(encoded).getOr(null) 148 | } 149 | } 150 | 151 | override fun compareTo(other: HybridLogicalClock): Int { 152 | return if (timestamp.epochMillis == other.timestamp.epochMillis) { 153 | if (counter == other.counter) { 154 | node.identifier.compareTo(other.node.identifier) 155 | } else { 156 | counter - other.counter 157 | } 158 | } else { 159 | timestamp.epochMillis.compareTo(other.timestamp.epochMillis) 160 | } 161 | } 162 | 163 | override fun toString(): String { 164 | return encodeToString(this) 165 | } 166 | } 167 | -------------------------------------------------------------------------------- /src/commonMain/kotlin/com/tap/hlc/NodeID.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.benasher44.uuid.Uuid 4 | import kotlin.jvm.JvmInline 5 | 6 | @JvmInline 7 | value class NodeID(val identifier: String) { 8 | companion object { 9 | fun mint(uuid: Uuid = Uuid.randomUUID()) = NodeID(uuid.toString().replace("-", "").takeLast(16)) 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /src/commonMain/kotlin/com/tap/hlc/Timestamp.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import kotlinx.datetime.Clock 4 | import kotlin.jvm.JvmInline 5 | 6 | @JvmInline 7 | value class Timestamp(val epochMillis: Long) { 8 | companion object { 9 | fun now(clock: Clock) = Timestamp(clock.now().toEpochMilliseconds()) 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/HLCComparisonTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import kotlinx.datetime.Clock 4 | import kotlin.test.Test 5 | import kotlin.test.assertNotEquals 6 | import kotlin.test.assertTrue 7 | 8 | class HLCComparisonTest { 9 | 10 | private val localNode = NodeID.mint() 11 | private val remoteNode = NodeID.mint() 12 | 13 | private val now: Timestamp = Timestamp.now(Clock.System) 14 | private val earlier: Timestamp = Timestamp(now.epochMillis - (1000 * 60 * 60)) 15 | 16 | @Test 17 | fun `test that when the local clocks timestamp is later comparing them returns a positive integer`() { 18 | val localClock = HybridLogicalClock(now, localNode) 19 | val remoteClock = HybridLogicalClock(earlier, remoteNode) 20 | 21 | assertTrue(localClock > remoteClock) 22 | } 23 | 24 | @Test 25 | fun `test that when the local clocks timestamp is earlier comparing them returns a negative integer`() { 26 | val localClock = HybridLogicalClock(earlier, localNode) 27 | val remoteClock = HybridLogicalClock(now, remoteNode) 28 | 29 | assertTrue(localClock < remoteClock) 30 | } 31 | 32 | @Test 33 | fun `test that when the local clocks timestamp is identical and counter is greater it returns a positive integer`() { 34 | val localClock = HybridLogicalClock(now, localNode, 1) 35 | val remoteClock = HybridLogicalClock(now, remoteNode) 36 | 37 | assertTrue(localClock > remoteClock) 38 | } 39 | 40 | @Test 41 | fun `test that when the local clocks timestamp is identical and counter is lesser it returns a negative integer`() { 42 | val localClock = HybridLogicalClock(now, localNode) 43 | val remoteClock = HybridLogicalClock(now, remoteNode, 1) 44 | 45 | assertTrue(localClock < remoteClock) 46 | } 47 | 48 | @Test 49 | fun `test that when the local clocks timestamp is identical and counter is equal it never returns 0`() { 50 | val localClock = HybridLogicalClock(now, localNode) 51 | val remoteClock = HybridLogicalClock(now, remoteNode) 52 | 53 | assertNotEquals(0, localClock.compareTo(remoteClock)) 54 | } 55 | } 56 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/HLCEncodingTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.benasher44.uuid.uuid4 4 | import com.github.michaelbull.result.fold 5 | import com.github.michaelbull.result.get 6 | import kotlin.test.Test 7 | import kotlin.test.assertEquals 8 | import kotlin.test.assertTrue 9 | 10 | class HLCEncodingTest { 11 | 12 | @Test 13 | fun `test that a hlc encodes to a string correctly`() { 14 | val epochMillis = 943920000000L 15 | val counter = 15 16 | val node = uuid4() 17 | 18 | val clock = HybridLogicalClock(Timestamp(epochMillis), NodeID.mint(node), counter) 19 | 20 | val expected = "${epochMillis.toString().padStart(15, '0')}:${counter.toString(36).padStart(5, '0')}:${node.toString().replace("-", "").takeLast(16)}" 21 | 22 | assertEquals(expected, HybridLogicalClock.encodeToString(clock)) 23 | } 24 | 25 | @Test 26 | fun `test that a hlc decodes from a string correctly`() { 27 | val epochMillis = 943920000000L 28 | val counter = 15 29 | val node = uuid4().toString().replace("-", "").takeLast(16) 30 | 31 | val encoded = "${epochMillis.toString().padStart(15, '0')}:${counter.toString(36).padStart(5, '0')}:$node" 32 | val decoded = HybridLogicalClock.decodeFromString(encoded) 33 | 34 | assertTrue(decoded.fold({ true }, { false })) 35 | assertEquals(decoded.get()?.timestamp?.epochMillis, epochMillis) 36 | assertEquals(decoded.get()?.counter, counter) 37 | assertEquals(decoded.get()?.node?.identifier, node) 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/HLCPersistTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.benasher44.uuid.uuid4 4 | import okio.Path.Companion.toPath 5 | import okio.fakefilesystem.FakeFileSystem 6 | import kotlin.test.Test 7 | import kotlin.test.assertEquals 8 | import kotlin.test.assertNotNull 9 | 10 | class HLCPersistTest { 11 | 12 | @Test 13 | fun `can store the hlc into a file at a given path`() { 14 | val fileSystem = FakeFileSystem() 15 | 16 | val epochMillis = 943920000000L 17 | val counter = 15 18 | val node = uuid4() 19 | 20 | val clock = HybridLogicalClock(Timestamp(epochMillis), NodeID.mint(node), counter) 21 | val path = "/Users/alice".toPath() 22 | val filename = "test.hlc" 23 | 24 | HybridLogicalClock.store(clock, path, fileSystem, filename) 25 | 26 | val expectedEncoded = "${epochMillis.toString().padStart(15, '0')}:${counter.toString(36).padStart(5, '0')}:${node.toString().replace("-", "").takeLast(16)}" 27 | val result = fileSystem.read(path / filename) { 28 | readUtf8() 29 | } 30 | 31 | assertEquals(expectedEncoded, result) 32 | fileSystem.checkNoOpenFiles() 33 | } 34 | 35 | @Test 36 | fun `can load a hlc from a given path`() { 37 | val fileSystem = FakeFileSystem() 38 | val path = "/Users/alice".toPath() 39 | fileSystem.createDirectories(path) 40 | val filename = "test.hlc" 41 | 42 | val epochMillis = 943920000000L 43 | val counter = 15 44 | val node = uuid4().toString().replace("-", "").takeLast(16) 45 | 46 | val encoded = "${epochMillis.toString().padStart(15, '0')}:${counter.toString(36).padStart(5, '0')}:$node" 47 | 48 | fileSystem.write(path / filename) { 49 | writeUtf8(encoded) 50 | } 51 | 52 | val result = HybridLogicalClock.load(path, fileSystem, filename) 53 | 54 | assertNotNull(result) 55 | assertEquals(result.timestamp.epochMillis, epochMillis) 56 | assertEquals(result.counter, counter) 57 | assertEquals(result.node.identifier, node) 58 | fileSystem.checkNoOpenFiles() 59 | } 60 | 61 | @Test 62 | fun `can store and load a hlc to and from a given path`() { 63 | val fileSystem = FakeFileSystem() 64 | val path = "/Users/alice".toPath() 65 | fileSystem.createDirectories(path) 66 | val filename = "test.hlc" 67 | 68 | val epochMillis = 943920000000L 69 | val counter = 15 70 | val node = uuid4() 71 | 72 | val clock = HybridLogicalClock(Timestamp(epochMillis), NodeID.mint(node), counter) 73 | HybridLogicalClock.store(clock, path, fileSystem, filename) 74 | val result = HybridLogicalClock.load(path, fileSystem, filename) 75 | 76 | assertNotNull(result) 77 | assertEquals(result.timestamp.epochMillis, epochMillis) 78 | assertEquals(result.counter, counter) 79 | assertEquals(result.node.identifier, NodeID.mint(node).identifier) 80 | fileSystem.checkNoOpenFiles() 81 | } 82 | } 83 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/HLCReceiveTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.github.michaelbull.result.fold 4 | import com.github.michaelbull.result.get 5 | import com.github.michaelbull.result.getError 6 | import kotlinx.datetime.Clock 7 | import kotlin.math.max 8 | import kotlin.math.pow 9 | import kotlin.test.Test 10 | import kotlin.test.assertEquals 11 | import kotlin.test.assertFalse 12 | import kotlin.test.assertTrue 13 | 14 | class HLCReceiveTest { 15 | 16 | private val localNode = NodeID.mint() 17 | private val remoteNode = NodeID.mint() 18 | 19 | private val now: Timestamp = Timestamp.now(Clock.System) 20 | 21 | @Test 22 | fun `test that when receiving a hlc with a duplicate node id an error is thrown`() { 23 | val localClock = HybridLogicalClock(now, localNode) 24 | val remoteClock = HybridLogicalClock(now, localNode) 25 | 26 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 27 | 28 | assertFalse(result.fold({ true }, { false })) 29 | assertEquals(HLCError.DuplicateNodeError(localNode), result.getError()) 30 | } 31 | 32 | @Test 33 | fun `test that when receiving a hlc with a timestamp that has drifted an error is thrown`() { 34 | // Only forward drift from remote matters 35 | val driftedTimestamp = Timestamp(now.epochMillis + (1000 * 60 * 60)) 36 | 37 | val localClock = HybridLogicalClock(now, localNode) 38 | val remoteClock = HybridLogicalClock(driftedTimestamp, remoteNode) 39 | 40 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 41 | 42 | assertFalse(result.fold({ true }, { false })) 43 | assertEquals(HLCError.ClockDriftError(remoteClock.timestamp, now), result.getError()) 44 | } 45 | 46 | @Test 47 | fun `test that when receiving a hlc with a counter that is too large an error is throw`() { 48 | val localClock = HybridLogicalClock(now, localNode) 49 | val remoteClock = HybridLogicalClock(now, remoteNode, (36f.pow(5) + 1).toInt()) 50 | 51 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 52 | 53 | assertFalse(result.fold({ true }, { false })) 54 | assertEquals(HLCError.CausalityOverflowError, result.getError()) 55 | } 56 | 57 | @Test 58 | fun `test when receiving a hlc for a future event inside acceptable drift the timestamp is updated`() { 59 | val futureTimestamp = Timestamp(now.epochMillis + (1000 * 59)) 60 | 61 | val localClock = HybridLogicalClock(now, localNode) 62 | val remoteClock = HybridLogicalClock(futureTimestamp, remoteNode) 63 | 64 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 65 | 66 | assertTrue(result.fold({ true }, { false })) 67 | assertEquals(localNode, result.get()?.node) 68 | assertEquals(futureTimestamp, result.get()?.timestamp) 69 | } 70 | 71 | @Test 72 | fun `test that when receiving a hlc for an event in the past the local nodes counter is incremented`() { 73 | val pastTimestamp = Timestamp(now.epochMillis - (1000 * 60 * 60)) 74 | 75 | val localClock = HybridLogicalClock(now, localNode, 1) 76 | val remoteClock = HybridLogicalClock(pastTimestamp, remoteNode, 3) 77 | 78 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 79 | 80 | assertTrue(result.fold({ true }, { false })) 81 | assertEquals(localNode, result.get()?.node) 82 | assertEquals(now, result.get()?.timestamp) 83 | assertEquals(localClock.counter + 1, result.get()?.counter) 84 | } 85 | 86 | @Test 87 | fun `test that when receiving a hlc for an event in the future the remote nodes counter is incremented`() { 88 | val futureTimestamp = Timestamp(now.epochMillis + (1000 * 59)) 89 | 90 | val localClock = HybridLogicalClock(now, localNode, 1) 91 | val remoteClock = HybridLogicalClock(futureTimestamp, remoteNode, 3) 92 | 93 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 94 | 95 | assertTrue(result.fold({ true }, { false })) 96 | assertEquals(localNode, result.get()?.node) 97 | assertEquals(futureTimestamp, result.get()?.timestamp) 98 | assertEquals(remoteClock.counter + 1, result.get()?.counter) 99 | } 100 | 101 | @Test 102 | fun `test that when receiving a hlc with a matching timestamp the largest counter is incremented`() { 103 | val localClock = HybridLogicalClock(now, localNode, 1) 104 | val remoteClock = HybridLogicalClock(now, remoteNode, 3) 105 | 106 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 107 | 108 | assertTrue(result.fold({ true }, { false })) 109 | assertEquals(localNode, result.get()?.node) 110 | assertEquals(now, result.get()?.timestamp) 111 | assertEquals(max(localClock.counter, remoteClock.counter) + 1, result.get()?.counter) 112 | } 113 | 114 | @Test 115 | fun `test that when both hlcs are behind the wall clock the counter is reset`() { 116 | val pastTimestamp = Timestamp(now.epochMillis - (1000 * 60 * 60)) 117 | 118 | val localClock = HybridLogicalClock(pastTimestamp, localNode, 1) 119 | val remoteClock = HybridLogicalClock(pastTimestamp, remoteNode, 3) 120 | 121 | val result = HybridLogicalClock.remoteTock(localClock, remoteClock, now) 122 | 123 | assertTrue(result.fold({ true }, { false })) 124 | assertEquals(localNode, result.get()?.node) 125 | assertEquals(now, result.get()?.timestamp) 126 | assertEquals(0, result.get()?.counter) 127 | } 128 | } 129 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/HLCSendTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import com.github.michaelbull.result.fold 4 | import com.github.michaelbull.result.get 5 | import kotlinx.datetime.Clock 6 | import kotlin.test.Test 7 | import kotlin.test.assertEquals 8 | import kotlin.test.assertTrue 9 | 10 | class HLCSendTest { 11 | 12 | private val localNode = NodeID.mint() 13 | 14 | private val now: Timestamp = Timestamp.now(Clock.System) 15 | 16 | @Test 17 | fun `test that when generating a local event the hlc advances to that of the current system clock`() { 18 | val currentCounter = 0 19 | val pastTimestamp = Timestamp(now.epochMillis - (1000 * 60)) 20 | val localClock = HybridLogicalClock(pastTimestamp, localNode, currentCounter) 21 | 22 | val result = HybridLogicalClock.localTick(localClock, now) 23 | 24 | assertTrue(result.fold({ true }, { false })) 25 | assertEquals(result.get()?.timestamp, now) 26 | assertEquals(result.get()?.counter, currentCounter) 27 | } 28 | 29 | @Test 30 | fun `test that when generating a local event the hlc advances the counter`() { 31 | val currentCounter = 0 32 | val futureTimestamp = Timestamp(now.epochMillis + (1000 * 60)) 33 | val localClock = HybridLogicalClock(futureTimestamp, localNode, currentCounter) 34 | 35 | val result = HybridLogicalClock.localTick(localClock, now) 36 | 37 | assertTrue(result.fold({ true }, { false })) 38 | assertEquals(result.get()?.timestamp, futureTimestamp) 39 | assertEquals(result.get()?.counter, currentCounter + 1) 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /src/commonTest/kotlin/com/tap/hlc/NodeIDTest.kt: -------------------------------------------------------------------------------- 1 | package com.tap.hlc 2 | 3 | import kotlin.test.Test 4 | import kotlin.test.assertEquals 5 | import kotlin.test.assertTrue 6 | 7 | class NodeIDTest { 8 | 9 | @Test 10 | fun `test minting a new node id is a success`() { 11 | val node = NodeID.mint() 12 | 13 | assertEquals(node.identifier.length, 16) 14 | assertTrue(node.identifier.contains("-").not()) 15 | } 16 | } 17 | --------------------------------------------------------------------------------