├── 101.txt ├── ALL.payload ├── CRaC_checkpoints.txt ├── Collections.txt ├── Google_guava.txt ├── GraalVM.txt ├── JSON.txt ├── JVM.txt ├── QA_and_testing.txt ├── RXJAVA.txt ├── TODO.txt ├── TornadoVM.txt ├── async_reactive.txt ├── concurrent_programming.txt ├── containerization.txt ├── debug_and_profiling.txt ├── functional_java.txt ├── gradle.txt ├── input_output.txt ├── jakarta_EE.txt ├── jee_war_apps.txt ├── jhipster.txt ├── maven.txt ├── monitorization.txt ├── monitorization_micrometer.txt ├── monitorization_opentelemetry.txt ├── persistence.payload ├── persistence_HikariCP.txt ├── persistence_JPA_Hibernate.txt ├── persistence_Jooq.txt ├── persistence_QueryDSL.txt ├── persistence_mybatis.txt ├── persistence_non_classified.txt ├── persistence_speedment.txt ├── persistence_stalactite.txt ├── security.txt ├── tomcat.txt ├── vertx.txt ├── whats_new.txt └── who_is_who.txt /ALL.payload: -------------------------------------------------------------------------------- 1 | 101.txt 2 | functional_java.txt 3 | Collections.txt 4 | Google_guava.txt 5 | input_output.txt 6 | JSON.txt 7 | QA_and_testing.txt 8 | security.txt 9 | persistence.payload 10 | RXJAVA.txt 11 | async_reactive.txt 12 | concurrent_programming.txt 13 | debug_and_profiling.txt 14 | jee_war_apps.txt 15 | maven.txt 16 | containerization.txt 17 | gradle.txt 18 | CRaC_checkpoints.txt 19 | monitorization.txt 20 | monitorization_opentelemetry.txt 21 | monitorization_micrometer.txt 22 | JVM.txt 23 | whats_new.txt 24 | TornadoVM.txt 25 | vertx.txt 26 | ../cryptography/java_cryptography.txt 27 | who_is_who.txt 28 | GraalVM.txt 29 | jhipster.txt 30 | TODO.txt 31 | eclipse_microprofile.txt 32 | -------------------------------------------------------------------------------- /CRaC_checkpoints.txt: -------------------------------------------------------------------------------- 1 | [[{scalability.JVM.CRAC,DevOps.CrAC]] 2 | # CrAC Project: Coordinate and Restore Checkpoints in running JVMs. 3 | 4 | * Summary from 5 | by Gerrit Grunwald, JUG Leader. 6 | * 7 | * 8 | 9 | * CrAC will not make apps run faster, but it avoid the JVM start-up and warm-up 10 | providing much better scalability when the process is restarted 11 | frequently. 12 | * It's probably a better alternative to GraalVM since we continue 13 | to have all the standard monitorization tools of an standard JVM 14 | environment. 15 | * primary aim: develop a new standard mechanism-agnostic API to 16 | notify Java programs about the checkpoint and restore events. 17 | 18 | 19 | * Currently (2024) only available on Linux. 20 | 21 | ## SpringBoot (3.2+) Automatic Checkpoint 22 | 23 | NOTE: A JVM with support for CRaC is needed (ex. Azul Zulu 21.0.1 + CRaC) 24 | 25 | ``` 26 | | $ sudo chown root:root $JAVA_HOME/lib/criu # fix permission to use Linux 27 | | $ sudo chmod u+s $JAVA_HOME/lib/criu # CRIU (Checkpoint/Restore in 28 | | # Userspace) 29 | | $ editor build.gradle.kts # or similar in maven 30 | | ... 31 | | dependencies { 32 | | + implementation 'org.crac:crac:1.4.0' // gradle, similar for Maven 33 | | └─────────┬─────────┘ 34 | | In linux it just invoke the native jdk.crac API of the underlying 35 | | JDK (supported by Linux CRIU), on MacOSX/Win it implements the API.º 36 | | ... } 37 | | 38 | | $ R_DIR_01="/tmp/jvmCheckpoint/app01" # folder for stored checkpoint for app01 39 | | $ mkdir -p ${R_DIR_01} 40 | | 41 | | $ ./gradlew assemble # or ./gradle clean build 42 | | 43 | | $ S_OPTS="-Xms512m ..." # Standard JVM startup options. 44 | | $ C_OPTS="" # <·· Checkpoint Options for JVM startup 45 | | # STEP 1): Create checkpoint and exists --------- 46 | | $ java \ 47 | | -Dspring.context.checkpoint=onRefresh \ 48 | | -XX:CRaCCheckpointTo=${R_DIR_01} \ 49 | | -jar .../myapp.jar 50 | | └──┬─────────────────────────────────┘ 51 | | Creates checkpoint right before the application is started 52 | | during the LifecycleProcessor.onRefresh phase. 53 | | """ all non-lazy initialized singletons have been 54 | | instantiated and InitializingBean#afterPropertiesSet 55 | | callbacks have been invoked; but the lifecycle has 56 | | not started, and the ContextRefreshedEvent has not 57 | | yet been published. """ 58 | | 59 | | # STEP 2): startup using the existing Checkpoint -- 60 | | $ java \ 61 | | -XX:CRaCRestoreFrom=${R_DIR_01} \ 62 | | -jar .../myapp.jar 63 | | 64 | ``` 65 | * startup time now is >10x faster with no code changes !!! [[{PM.low_code}]] 66 | * The previous checkpoint only contains the Spring framework code and not 67 | the application code. 68 | 69 | * WARN: See comments with problems in the original post when the image contains 70 | external resources (tcp/DDBB connections,...) 71 | * Q: "... Were you able to verify how was it performing with the DB connections 72 | using manual checkpoint..."
73 | A: Nope, but feel free to test that and share the results. 74 | * I try on a demo app using a simple controller calling a JPA object in database. 75 | ``` 76 | org.springframework.context.ApplicationContextException: Failed to take CRaC checkpoint 77 | ... 78 | 79 | Caused by: java.lang.Exception: This file descriptor was created by 80 | HikariPool-1 connection adder at epoch:1701351807255 here 81 | ... 82 | ``` 83 | * ... Same issue 84 | ``` 85 | ... jdk.internal.crac.impl.CheckpointOpenSocketException: tcp localAddr 86 | 172.18.0.6 localPort 46868 remoteAddr 172.18.0.3 remotePort 6379 87 | ``` 88 | 89 | ## Manual Checkpoint (after App warmup) 90 | 91 | * Checkpoint creation: 92 | ``` 93 | | # STEP 1: Start JVM with Checkpoint enabled 94 | | $ java \ 95 | | -XX:CRaCRestoreFrom=${R_DIR_01} 96 | | $ java -jar .../myapp.jar 1>app.log 2>&1 & 97 | | 98 | | ... wait now for warmup, maybe invoking some REST APIs ... 99 | | 100 | | # STEP 2: Checkpoint and stop the running app. 101 | | $ jcmd .../myapp.jar JDK.checkpoint 102 | | 103 | | # STEP 3: Restart with the new checkpoint after warmup. 104 | | $ java \ 105 | | -XX:CRaCRestoreFrom=${R_DIR_01} 106 | | $ java -jar .../myapp.jar 1>app.log 2>&1 & 107 | ``` 108 | 109 | * startup time now is >40x faster with no code changes. !!! 110 | The app is still running on a normal JVM (vs GraalVM). 111 | [[{PM.low_code}]] 112 | 113 | [[{DevOps.containarization.CRaC]] 114 | ## Running containerized Java Apps with CRaC 115 | 116 | * 117 | [[DevOps.containarization.CRaC}]] 118 | 119 | 120 | ## Se also 121 | 122 | * Checkpointing outside the JVM 123 | 124 | * Checkpoint inside JVM HOW-TO 125 | 126 | 127 | * Checkpointing with Linux CRIU: 128 | 129 | ``` 130 | | CONSOLE 1 | CONSOLE 2 131 | | $ setsid java -XX:-UsePerfData \ | $ sudo criu dump -t $pid \ ← stops and checkpoint 132 | | -XX:+UseSerialGC Scooby | --shell-job -o dump.log app 133 | | | 134 | | | $ sudo restore --shell-job \ ← Restore app 135 | | | -d -vvv -o restore.log 136 | ``` 137 | [[}]] 138 | 139 | [[scalability.JVM.CRAC}]] 140 | -------------------------------------------------------------------------------- /Collections.txt: -------------------------------------------------------------------------------- 1 | [[{data_structures.101,java_lang.101,scalability.101,doc_has.decission_tree,doc_has.diagram,qa.data]] 2 | # Collections 3 | 4 | ## Collections 101 5 | 6 | ### COLLECTION DECISSION TREE 7 | ``` 8 | | ┌──────────┐ 9 | | │ Allows │ 10 | | ┌─── YES ─────┤Duplicates├── NO ───────┐ 11 | | │ List to └──────────┘ Set to │ 12 | | │ be selected be selected │ 13 | | │ v 14 | | v ┌───────────┐☜ order established at 15 | | ┌─────────────────────┐ │ Maintains │ write time 16 | | │ Unknown number │ │ INSERTION │ 17 | | ┌─NO─┤of elements will be ├YES─┐ ┌───YES───┤ ORDER ? ├──NO──┐ order requested 18 | | │ │added and/or index │ │ │ └───────────┘ │ at read time 19 | | │ │based search will not│ │ v ↓ ☟ 20 | | │ │be frequent? │ │ LinkedHashSet ┌────────────┐ 21 | | │ └─────────────────────┘ │ │ Mantains │ 22 | | v v ┌─NO─┤ READ ORDER ├YES┐ 23 | | ArrayList LinkedList │ │(alpha,...)?│ │ 24 | | │ └────────────┘ │ 25 | | │ │ 26 | | v v 27 | HashSet TreeSet 28 | ``` 29 | 30 | ### Collection Comparative 31 | 32 | ``` 33 | | Standard non-concurrent SDK: 34 | | ┌────────────────────────────────────────────────────────────────────────────────────────── 35 | | │ IMPLEMENTATIONS 36 | | ├────────────────────────────────────────────────────────────────────────────────────────── 37 | | │ Hash Table │ Resizable Array │Balanced Tree │ Linked List │ HashTable+LinkedList 38 | | │ │ │ │ │ 39 | | ┌───────┼───────────────────┼───────────────────┼──────────────┼─────────────┼───────────────────── 40 | | │ │ HashSet │ │ TreeSet │ │ LinkedHashSet 41 | | │ │ │ │ │ │ 42 | | ├───────┼───────────────────┼───────────────────┼──────────────┼─────────────┼───────────────────── 43 | | │ │ │ ArrayList │ │ LinkedList │ 44 | | │ │ │ Vector │ │ LinkedList │ 45 | | ├───────┼───────────────────┼───────────────────┼──────────────┼─────────────┼───────────────────── 46 | | │ │ HashMap,Hashtable │ │ TreeMap │ │ LinkedHashMap 47 | | └───────┴───────────────────┴───────────────────┴──────────────┴─────────────┴───────────────────── 48 | | 49 | | WARN : There is a huge performance difference LinkedList and ArrayList. 50 | | - when there is a large number of add/remove operations LinkedList is much faster. 51 | | - When there is a lot of random access operations ArrayList is much faster. 52 | ``` 53 | 54 | ### INTERFACES 55 | * 56 | * 57 | * 58 | * 59 | * 60 | * 61 | * 62 | * 63 | * 64 | 65 | ### Cheat-sheet 66 | * REF: 67 | ``` 68 | | ┌─────────────────┬─────────────────────────────────────────────────────────────┐ 69 | | │Collection │ Thread-safe ┃ DATA ┃ OPERATIONS ALLOWED │ 70 | | │ │ alternative ┃ PROPER- ┃────────────────────┤ 71 | | │class │ ┃ TIES ┃Iterat. ┃ ┃Random │ 72 | | │ │ ┃ ┃ Order ┃ ┃ Access │ 73 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┬──┬──┃──┃──┬──┬──┤ 74 | | │HashMap │ ConcurrentHashMap ┃ │OK│ │ ┃ │ │ ┃OK┃OK│ │ │ 75 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 76 | | │SortedMap │ ? ┃ │OK│ │ ┃ │ │ ┃? ┃OK│ │ │ 77 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 78 | | │NavigableMap ¹ │ ? ┃ │OK│ │ ┃ │ │ ┃? ┃OK│ │ │ 79 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 80 | | │HashBiMap(Guava) │ Maps.syncrhonizedBiMap ┃ │OK│ │ ┃ │ │ ┃OK┃OK│OK│ │ 81 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 82 | | │ArrayListMultimap│ Maps.synchronizedMultiMap ┃ │OK│OK│ ┃ │ │ ┃OK┃OK│ │ │ 83 | | │(Guava) │ (new ArrayListMultimap()) ┃ │ │ │ ┃ │ │ ┃ ┃ │ │ │ 84 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 85 | | │LinkedHashMap │ Collections.syncrhonizedMap┃ │OK│ │ ┃OK│ │ ┃OK┃OK│ │ │ 86 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 87 | | │TreeMap │ ConcurrentSkipListMap ┃ │OK│ │ ┃ │OK│ ┃OK┃OK│ │ │ 88 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 89 | | │Int2IntMap (Fastutils) ┃ │OK│ │OK┃ │ │ ┃OK┃OK│ │OK│ 90 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 91 | | │ArrayList │ CopyOnWriteArrayList ┃OK│ │OK│ ┃OK│ │OK┃ ┃ │ │OK│ 92 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 93 | | │HashSet │ Collections.newSetFromMap ┃OK│ │ │ ┃ │ │ ┃OK┃ │OK│ │ 94 | | │ │ (new ConcurrentHashMap()) ┃ │ │ │ ┃ │ │ ┃ ┃ │ │ │ 95 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 96 | | │IntArrayList(FastUtils) ┃OK│ │OK│OK┃OK│ │OK┃ ┃ │ │OK│ 97 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 98 | | │PriorityQueue │ PriorityBlockingQueue ┃OK│ │OK│ ┃ │OK│ ┃ ┃ │ │ │ 99 | | ┼─────────────────┼────────────────────────────┃──┼──┼──┼──┃──┼──┼──┃──┃──┼──┼──┤ 100 | | │ArrayDeque │ ArrayBlockingQueue ┃OK│ │OK│ ┃OK│ │OK┃ ┃ │ │ │ 101 | | └─────────────────┴────────────────────────────┃──┴──┴──┴──┃──┴──┴──┃──┃──┴──┴──┘ 102 | | Individual elements ·····┘ │ │ │ │ │ │ ^ │ │ └ By Index 103 | | Key─value pairs ···········┘ │ │ │ │ │ │ │ └··· By Value 104 | | Duplicate element support ·····┘ │ │ │ │ │ └······ By Key 105 | | Primite support ··················┘ │ │ │ │ 106 | | FIFO ··┘ │ │ Fast 107 | | Sorted ·····┘ │'has(el)' 108 | | LIFO ········┘ 109 | | 110 | | Collection class │ Random access by idx/key │ Search/Contains │ Instert 111 | | ─────────────────┼───────────────────────────┼─────────────────┼─────────── 112 | | ArrayList │ O(1) │ O(n) │ O(n) 113 | | HashSet │ O(1) │ O(1) │ O(1) 114 | | HashMap │ O(1) │ O(1) │ O(1) 115 | | TreeMap │ O(log(n)) │ O(log(n)) │ O(log(n)) 116 | | 117 | |¹ NavigableMap: SortedMap with additional methods for finding entries 118 | | by their ordered position in the key set. 119 | | So potentially this can remove the need for iterating 120 | | in the first place - you might be able to find the 121 | | specific entry you are after using the higherEntry, 122 | | lowerEntry, ceilingEntry, or floorEntry methods. The 123 | | descendingMap method even provides an explicit method 124 | | of reversing the traversal order. 125 | ``` 126 | 127 | ## GRAPH STRUCTURES [[{data_structures.graphs]] 128 | * [REF](https://guava.dev/releases/20.0/api/docs/com/google/common/graph/package-summary.html) 129 | 130 | ``` 131 | |Interface Description 132 | |Graph An interface for graph-structured data, whose edges are 133 | | anonymous entities with no identity or information of their own. 134 | |MutableGraph A subinterface of Graph which adds mutation methods. 135 | |MutableNetwork A subinterface of Network which adds mutation methods. 136 | |MutableValueGraph A subinterface of ValueGraph which adds mutation methods. 137 | |Network An interface for graph-structured data, whose edges are unique objects. 138 | |ValueGraph An interface for graph- structured data, whose edges have associated 139 | | non-unique values. 140 | ``` 141 | [[data_structures.graphs}]] 142 | 143 | ## Lists [[{data_structures.lists,data_structures.101,qa]] 144 | 145 | ### INITIALIZING A LIST 146 | 147 | ``` 148 | | import java.util.ArrayList; 149 | | import java.util.Arrays; 150 | | import java.util.Collections; 151 | | import java.util.stream.Collectors; 152 | | import java.util.stream.Stream; 153 | | public class Main { 154 | | public static void main(String[] args) { 155 | | final List // <·· WARN: `final` keyword forbids re-asigning the list, 156 | | list01 = new ArrayList() , // but its content is still mutable. ¹ 157 | | ilist01 = Collections // <·· Inmutable version (thread safe) of list01 158 | | .unmodifiableList(list01), 159 | | list02 = new ArrayList(10), 160 | | ilist03 = Arrays.asList("s1", "s2"), // <·· READ-ONLY inmutable ArrayList. list03 will lack 161 | | set(), get(), contains() or "adding" methods 162 | | providing for a read-only view of the original String[] 163 | | list03 = new ArrayList( // <·· mutable version built by "copying" the inmutable 164 | | ilist03) , // ilist03 version. 165 | | 166 | | list04 = Stream.of("s1", "s2") 167 | | .collect( 168 | | Collectors.toCollection( 169 | | ArrayList::new) ); 170 | | list05 = new ArrayList() ; 171 | | // └────┴─────────────┬────┐ 172 | | list05.add("s1"); · · 173 | | Collections.addAll(list05, "s2","s3"); 174 | | } 175 | | } 176 | ``` 177 | 178 | ### Walk over Lists(and Sets) collections 179 | 180 | ``` 181 | | for ( int idx = 0; idx < collection .length ; idx++) {// <·· Alt 1: Java 1.0 for-walk over collection index. 182 | | type array_element = collection .get(idx); // WARN: Very slow for LinkedLists 183 | | } // (Faster for other List implementations) 184 | | // 185 | | for ( Iterator iterator = collection .iterator() ; // <·· Alt 2: for-walk over iterator. Preferred to Alt 1. 186 | | iterator.hasNext(); ) { // safer when removing/modifying the collection 187 | | type type = (type) iterator.next() ; // while iterating over it. 188 | | } // 189 | | // 190 | | for ( iterable_type iterable_element :collection ) { // <·· Alt 3: Preferred to Alt 2 option when NOT 191 | | ... // removing/modifying elements while in the "loop" 192 | | } // 193 | | 194 | | collection.stream() // <·· Alt 4: STREAMS (Java 8+): UNIX "pipe like" iteration. 195 | | .peek ( e -> { System.err.print("[" + e + "]"); } ) // Functional approach. Preferred for "complex" business 196 | | .forEach( e -> { System.out.print(e + ","); } ) // logic. 197 | |; 198 | ``` 199 | 200 | [[data_structures.lists}]] 201 | 202 | ## Maps ("Dictionaries") [[{data_structures.maps]] 203 | 204 | ### Initializing maps 205 | 206 | ``` 207 | | final Map myMap = // <·· Mutable map (WARN: even if 'final' used) 208 | | new HashMap(); 209 | | myMap.put("k1", "k2"); 210 | | ... 211 | | final Map myInmutableMap = // <·· Inmutable version of map 212 | | Collections.unmodifiableMap(temp); 213 | | 214 | | final Map inumtableMap01 = // <·· Java 9+ must have even number of elements 215 | | Map.of("k1", "v1", "k2", "v2"); 216 | | 217 | | final Map inmutableMap02 = // <·· Java 9+ (Safer syntax). Preferred 218 | | Map:ofEntries( 219 | | Map.entry("k1", "v1"), 220 | | Map.entry("k2", "v2"), 221 | | ... 222 | | ); 223 | | 224 | | final Map test = // <·· Guava ImmutableMap 225 | | ImmutableMap:of("k1","v1", ...); // works only with up to 5 key/value pairs 226 | | 227 | | final Map test = // <·· Guava ImmutableMap alternative 228 | | ImmutableMap.builder() 229 | | .put("k1", "v1").put("k2", "v2") 230 | | ... 231 | | .build(); 232 | ``` 233 | 234 | ### WALK-OVER/ITERATE A MAP 235 | 236 | ``` 237 | | Map map = ... 238 | | for ( Map.pair pair : 239 | | map .pairSet() ) { ← Alt 1: pairSet (Java 5+) 240 | | ... pair.getKey() ... pair.getValue() ... 241 | | } 242 | | 243 | | for ( Map.Entry pair : ← Alt 2: USING "for : entrySet" 244 | | myMap .entrySet() ) { 245 | | ... pair.getKey() ... pair.getValue() ... 246 | | } 247 | | 248 | | map.forEach((k, v) -> { ← forEach (java 8+) 249 | | ... pair.getKey() ... pair.getValue() ... 250 | | } 251 | | 252 | | map.entrySet().stream().forEach( (pair) -> { ← Stream API (1.8+) 253 | | ... pair.getKey() ... pair.getValue() ... "Functional" Unix Pipe like style 254 | | }); 255 | | 256 | | map.entrySet().stream(). ← Stream API parallel (1.8+) 257 | | .parallel().forEach( pair -> { "Functional" Unix Pipe like style 258 | | ... pair.getKey() ... pair.getValue() ... 259 | | } ) ); 260 | ``` 261 | 262 | ### UPDATING MAPS 263 | 264 | ``` 265 | | Map map01 = new HashMap<>(); 266 | | map01.put("key1", 1); // <·· Insert. 267 | | map01.put("key2", 2); 268 | | 269 | | if (!map01.containsKey("key3")) { // <·· Update checking for key existence (Before Java 8) 270 | | throw RuntimeException("..."); 271 | | } 272 | | map01.put("key3", 3); 273 | | 274 | | Integer oldValue = map01 // <·· Java 8+. Returns nulls if key1 didn't exists 275 | | .replace("key1", 10); 276 | | boolean isOK = map01.replace("key1", // <·· Java 8+. Safer/preferred variant. 277 | | 1 /*old value*/, // Update fails if old value is null or changed. 278 | | 10 /* new value */ ); // 279 | | 280 | | map01.getOrDefault("key4", 0)); // <·· Return default for non-existing key. Probably 281 | | // throwing is a safer approach in most scenarios. 282 | | 283 | | map01.putIfAbsent ("key4", 4)); // <·· Update only if key4 value is absent/null. 284 | | 285 | | const BiFunction 286 | | sumFun = (x1, x2) -> x1 + x2; 287 | | 288 | | map01.compute("key1", // <·· Use bifunction to update. NullPointerException 289 | | (key, currentValue) // if key doesn't exit. 290 | | -> sumFun.apply(currentValue, 2)); // (alternatively computeIfAbsent / computeIfPresent ) 291 | | // 292 | | map01.merge("key1", defaultValue, // <·· Update with BiFunction if key exists. 293 | | (key, currentValue) // or add defValue to key otherwise. 294 | | -> sumFun.apply(v, 2)); 295 | ``` 296 | [[data_structures.maps}]] 297 | 298 | ## java.util.Collections [[{data_structures.collections,]] 299 | 300 | * 301 | * Utility class with static methods that operate on or return collections 302 | ``` 303 | | Collections.EMPTY_SET ( == Collections.emptySet() ) See also Collections.singleton(T o) 304 | | Collections.EMPTY_LIST ( == Collections.emptyList() ) See also Collections.singletonList(T o) 305 | | Collections.EMPTY_MAP ( == Collections.emptyMap() ) See also Collections.singletonMap(K key, V value) 306 | | Collections.emptyEnumeration() 307 | | Collections.emptyIterator() 308 | | Collections.emptyListIterator() 309 | | 310 | | boolean Collections.addAll(Collection c, T... elements) Adds all elements to collection 'c' 311 | | Queue Collections.asLifoQueue(Deque deque) deque to Last-In/First-Out "LIFO" Queue view 312 | | int Collections.binarySearch(List list, T key) Searches key in list using binary search. 313 | | int Collections.binarySearch(List list, T key, Comparator c) Searches key in list using binary search + comparator. 314 | | 315 | | Collection Collections.checkedCollection(Collection c, Class type) Returns dynamic&typesafe view of input collection. 316 | | List Collections.checkedList(List list, Class type) 317 | | Map Collections.checkedMap 318 | | (Map m, Class keyType, Class valueType) 319 | | Set Collections.checkedSet(Set s, Class type) 320 | | SortedMap Collections.checkedSortedMap 321 | | (SortedMap m, Class keyType, Class valueType) 322 | | 323 | | SortedSet Collections.checkedSortedSet(SortedSet s, Class type) 324 | | void Collections.copy(List dest, List src) Copies src-list elements to dest-list 325 | | boolean Collections.disjoint(Collection c1, Collection c2) true if c1/c2 have no elements in common. 326 | | 327 | | Enumeration Collections.enumeration(Collection c) Returns an enumeration over the specified collection. 328 | | 329 | | void Collections.fill(List list, T obj) Replaces all elements in list with obj. 330 | | int Collections.frequency(Collection c, Object o) Returns number of el. in c equals to o. 331 | | int Collections.indexOfSubList(List list, List sublist) -1 if not found. 332 | | int Collections.lastIndexOfSubList(List list, List sublist) -1 if not found. 333 | | ArrayList Collections.list(Enumeration e) Enum to array list (in order as returned by input enum). 334 | | T Collections.max/min(Collection coll (, Comparator comp)) max/min element of collection, according to comparator 335 | | (Def to natural ordering) 336 | | List Collections.nCopies(int nCopies, T inputObject) 337 | | Set Collections.newSetFromMap(Map map) 338 | | boolean Collections.replaceAll(List list, T oldVal, T newVal) NOTE: Replaces in list 339 | | void Collections.reverse(List list) 340 | | Comparator Collections.reverseOrder() Return <> Comparator imposing 341 | | reverse natural order. 342 | | Comparator Collections.reverseOrder(Comparator cmp) Returns comparator imposing 343 | | reverse ordering of input comp. 344 | | void Collections.rotate(List list, int distance) Rotates elements in list. 345 | | void Collections.shuffle(List list (, Random rnd) ) Permutes elements using source of randomness 346 | | (or default source of randomness). 347 | | void Collections.sort(List list (, Comparator c)) Sorts in comparator (def. to natural) order. 348 | | void Collections.swap(List list, int pos1, int pos2) Swap elemnts in pos 1 and 2. 349 | | [[qa.thread-safe]] 350 | | Collection Collections.synchronizedCollection(Collection c) Returns thread-safe collection 351 | | List Collections.synchronizedList(List list) Returns thread-safe list 352 | | Map Collections.synchronizedMap(Map m) Returns thread-safe map 353 | | Set Collections.synchronizedSet(Set s) Returns thread-safe set 354 | | SortedMap Collections.synchronizedSortedMap(SortedMap m) Returns thread-safe sorted map 355 | | SortedSet Collections.synchronizedSortedSet(SortedSet s) Returns thread-safe sorted set 356 | | [[qa.thread-safe]] 357 | | [[qa.inmutability.data_structures]] 358 | | Collection Collections.unmodifiableCollection(Collection c) Returns inmutable view 359 | | List Collections.unmodifiableList(List list) Returns inmutable view 360 | | Map Collections.unmodifiableMap(Map m) Returns inmutable view 361 | | Set Collections.unmodifiableSet(Set s) Returns inmutable view 362 | | SortedMap Collections.unmodifiableSortedMap(SortedMap m) Returns inmutable view 363 | | SortedSet Collections.unmodifiableSortedSet(SortedSet s) Returns inmutable view 364 | | [[qa.inmutability.data_structures]] 365 | ``` 366 | [[data_structures.collections}]] 367 | 368 | [[{data_structures.collections.eclipse]] 369 | ## Eclipse Collections 370 | * 371 | * 372 | (Originated from Goldman Sachs [gs-collection](https://github.com/goldmansachs/gs-collections) 373 | * This library includes almost any collection needed: 374 | primitive type collections, multimaps, bidirectional maps, ... 375 | * Rich, functional and fluent API. 376 | * Preferred to standard Collections in JDK when memory efficiency is a "MUST". 377 | REF: 378 | [[{performance.memory}]] 379 | [[data_structures.collections.eclipse}]] 380 | 381 | [[{data_structures.iterator,data_structures.101,doc_has.comparative]] 382 | ## `<>`(1.0) vs `<>`(1.7) 383 | 384 | * both interfaces will give successive elements 385 | * Iterators allow the caller to remove elements from the underlying 386 | collection during the iteration with well-defined semantics. 387 | (additional remove method) 388 | * Iterators Method names have been improved. 389 | * Iterators are fail-fast: 390 | * If thread A changes the collection, while thread B is traversing 391 | it, the iterator implementation will try to throw a 392 | ConcurrentModificationException (best effort since it can not always 393 | be guaranteed) 394 | * The fail-fast behavior of iterators can be used only to 395 | detect bugs sin the best effort doesn't warrant its trigger. 396 | * newer 'concurrent' collections will never throw it. 397 | Reading thread B will traverse the collection "snapshot" at 398 | the start of read. 399 | 400 | * Iterator should be preferred over Enumeration taking the place of 401 | Enumeration in collections framework 402 | ``` 403 | | Enumeration │ Iterator 404 | | ────────────────┼──────────── 405 | | hasMoreElement()│ hasNext() 406 | | nextElement() │ next() 407 | | │ remove() ← optional: not implemented in many classes 408 | ``` 409 | [[data_structures.iterator}]] 410 | [[data_structures.101}]] 411 | 412 | ## SummaryStatistics(1.8+) [[{java_lang.functional,PM.TODO]] 413 | - Three of the new classes introduced in JDK 8 are 414 | DoubleSummaryStatistics, IntSummaryStatistics, 415 | andLongSummaryStatistics of the java.util package. These classes make 416 | quick and easy work of calculating total number of elements, minimum 417 | value of elements, maximum value of elements, average value of 418 | elements, and the sum of elements in a collection of doubles, 419 | integers, or longs. Each class's class-level Javadoc documentation 420 | begins with the same single sentence that succinctly articulates 421 | this, describing each as "A state object for collecting statistics 422 | such as count, min, max, sum, and average." 423 | [[}]] 424 | 425 | ## Collectors 1.8+ [[{java_lang.101,JAVA_LANG.FUNCTIONAL,PM.TODO}]] 426 | -------------------------------------------------------------------------------- /Google_guava.txt: -------------------------------------------------------------------------------- 1 | # Google Guava 2 | 3 | ## Collection Library Comparative 4 | * 5 | 6 | Which is best? Java Collectoon vs Eclipse Collections, Guava or Apache Collections? 7 | ... 8 | 9 | Guava, in particular, has been somewhat of a maintenance headache in the past and I 10 | personally avoid it now. Their attitude of breaking APIs is simply not something 11 | that's fun to deal with especially when you start getting larger projects. 12 | 13 | ## Guava ListenableFuture [[{architecture.async.ListenableFuture,doc_has.comparative,qa]] 14 | 15 | * 16 | * Concurrency is a hard problem, but it is significantly simplified by 17 | working with powerful and simple abstractions. To simplify matters, 18 | Guava extends the Future interface of the JDK with ListenableFuture. 19 | * """We strongly advise that you always use ListenableFuture instead 20 | of Future in all of your code, because: 21 | * Most Futures methods require it. 22 | * It's easier than changing to ListenableFuture later. 23 | * Providers of utility methods won't need to provide Future and ListenableFuture 24 | variants of their methods. 25 | 26 | ## Listenable vs CompletableFutures 27 | ``` 28 | 29 | ListenableFuture │ CompletableFuture 30 | ─────────────────────────────────────────────────────┼─────────────────────────────────────────────────── 31 | │ It is different from ListenableFuture in that it 32 | │ can be completed from any thread 33 | ─────────────────────────────────────────────────────┼─────────────────────────────────────────────────── 34 | ListenableFuture listenable = service.submit(...); │ CompletableFuture completableFuture = 35 | Futures.addCallback(listenable, │ new CompletableFuture(); 36 | new FutureCallback() { │ completableFuture.whenComplete(new BiConsumer() { 37 | @Override │ @Override 38 | public void onSuccess(Object o) { │ public void accept(Object o, Object o2) { 39 | //handle on success │ //handle complete 40 | } │ } 41 | │ }); // complete the task 42 | @Override │ completableFuture.complete(new Object()) 43 | public void onFailure(Throwable throwable) { │ 44 | //handle on failure │ When a thread calls complete on the task, 45 | } │ the value received from a call to get() is 46 | }) │ set with the parameter value if the task is 47 | │ not already completed. 48 | 49 | ..."CompletableFuture is dangerous because it exposes complete methods." 50 | ..."CompletableFuture would have been good if it extended Future 51 | and did not expore toCompletableFuture,... and they could have named 52 | it something meaningful like ChainableFuture " 53 | ``` 54 | [[architecture.async.ListenableFuture}]] 55 | 56 | -------------------------------------------------------------------------------- /GraalVM.txt: -------------------------------------------------------------------------------- 1 | # GraalVM Summary [[{jvm.graalvm,devops.k8s,scalability.jvm,PM.TODO]] 2 | 3 | 4 | 5 | 6 | - Graal: How to Use the New JVM JIT Compiler in Real Life 7 | 8 | 9 | - GraalVM Native Image 10 | 11 | "native-image" utility: 12 | - ahead-of-time compiler to a standalone executable . 13 | - JVM is replaced with necesary components (memory mngr, 14 | Thread scheduler) in "Substrate VM" runtime: 15 | Substrate VM runtime is actually the name for the runtime components 16 | (like the deoptimizer, garbage collector, thread scheduling etc.). 17 | - Result has faster startup time and lower runtime memory . 18 | - It statically analyses which classes and methods are reachable 19 | and used during application execution and passes all this 20 | reachable code as the input to the GraalVM compiler for 21 | ahead-of-time compilation into native-library. 22 | Ex Ussage: 23 | 24 | # tested with graalvm 19.3.1 25 | ./gradlew spotlessApply 26 | ./gradlew build 27 | ./gradlew shadowJar // ← create fat JARs, relocate packages for apps/libs 28 | cd "build/libs" || exit 29 | native-image \ 30 | -cp svm-1.0-SNAPSHOT-all.jar \ 31 | org.web3j.svm.MainKt \ 32 | --no-fallback \ 33 | --enable-https \ 34 | --enable-http 35 | [[}]] 36 | 37 | # GraalVM Dockerfile [[{devops.containarization,scalability.jvm.graalvm,architecture.serverless,PM.TODO]] 38 | 39 | 40 | ``` 41 | | FROM oracle/graalvm-ce:20.0.0-java11 as builder 42 | | WORKDIR /app 43 | | COPY . /app 44 | | 45 | | RUN gu install native-image 46 | | 47 | | # Build the app (via Maven, Gradle, etc) and create the native image 48 | | FROM scratch 49 | | COPY --from=builder /app/target/my-native-image /my-native-image 50 | | ENTRYPOINT ["/my-native-image"] 51 | ``` 52 | 53 | * to build a statically linked native image: 54 | ``` 55 | --static 56 | ``` 57 | 58 | * ...Luckily GraalVM has a way to also include the necessary system 59 | libraries in the static native image with musl libc: 60 | * In your Dockerfile download the musl bundle for GraalVM: 61 | ``` 62 | RUN curl --silent -L -o - \ 63 | https://github.com/gradinac/musl-bundle-example/releases/download/v1.0/musl.tar.gz && \ 64 | tar -xvzf musl.tar.gz 65 | ``` 66 | And then add a native-image parameter that points to the extracted location of the bundle, like: 67 | ``` 68 | -H:UseMuslC=bundle/ 69 | ``` 70 | Now your native image will include the standard library system calls that are needed! 71 | 72 | - If AOT thing fails, it will fallback to just running the app in the JVM. 73 | To avoid it running on the JVM: 74 | ``` 75 | --no-fallback 76 | ``` 77 | 78 | * FAIL-FAST: Don't Defer Problems to Runtime. Make sure native-image 79 | is NOT being run with any of these params: 80 | ``` 81 | --report-unsupported-elements-at-runtime 82 | --allow-incomplete-classpath 83 | -H:+ReportUnsupportedElementsAtRuntime 84 | ``` 85 | 86 | ## Reflection Woes 87 | * reflection happens at runtime, making it hard for an AOT complier. 88 | * you can tell GraalVM about what needs reflection access, 89 | but this can quickly get a bit out-of-hand, hard to derive and maintain. 90 | * Micronaut and Quarkus do a pretty good job generating the reflection 91 | configuration at compile time but you might need to augment the 92 | generated config. (tricky with shaded transitive dependencies). 93 | * To reliably generate a reflection config you need to exercise as many 94 | execution code paths as possible, ideally by running unit/integration tests. 95 | 96 | GraalVM has a way to keep track of reflection and output the configuration. 97 | Run the app on GraalVM and use a special Java agent that will be able to 98 | see the reflective calls. 99 | 100 | STEPS: 101 | 102 | 1. grab GraalVM Community Edition. 103 | 2. set JAVA_HOME and PATH. 104 | 3. From release assets grab the right native-image-installable-svm-BLAH.jar file 105 | and extract it in the root of your GraalVM JAVA_HOME directory. 106 | 4. run tests with parameter: 107 | ``` 108 | -agentlib:native-image-agent=config-output-dir=src/graal" 109 | ``` 110 | This will generate the reflection config (and possibly other configs for 111 | dynamic proxies, etc). 112 | 5. tell native-image about those configs, like: 113 | -H:ReflectionConfigurationFiles=src/graal/reflect-config.json 114 | 6. For Quarkus & Micronaut see their docs (Quarkus / Micronaut) for details on 115 | how to add your own reflection config files. 116 | [[}]] 117 | 118 | 119 | ## Compressed GraalVM Native Images 120 | 121 | * REF: 122 | "the best startup for Java apps comes in tiny packages" 123 | 124 | ### TLDR 125 | * Ultimate Packer for eXecutables (UPX for sort) allows to 126 | compress GraalVM 20.3.0 Native Images by a factor of x3/x4 with 127 | very low impacts on the startup time.
128 | * DRAGON Stack manager Java CLI tool, improves the 129 | developer experience because of the resulting perception: a lot 130 | of capabilities in less than 20 MB. 131 | 132 | * UPX is portable, free and open-source: 133 | * Decompression is simple and very fast. 134 | * Requires no memory for decompression. 135 | * Excellent compression ratios. 136 | * Several compression levels are available. 137 | 138 | * CI/CD Pipeline 139 | ``` 140 | src code -> |GraalVM AOT| -> exe -> |UPX| -> micro-exe 141 | · · 142 | └─┬─┘ 143 | $ upx -7 -k myapp 144 | ``` 145 | 146 | ## Quarkus (GraalVM) Framework [[{jvm.graalvm,devops.k8s,scalability.jvm,persistence.jpa,PM.TODO]] 147 | 148 | * 149 | * 150 | * 151 | * 152 | * .. 153 | 154 | 155 | [[}]] 156 | 157 | 158 | -------------------------------------------------------------------------------- /JSON.txt: -------------------------------------------------------------------------------- 1 | # JSON Processing 2 | 3 | 4 | * Summary 5 | ``` 6 | 20XX JAX-RS 1.1: Java API for RESTful Web Services 7 | Official in JEE 6. 8 | 9 | 2013 JAX-RS 2.0: 10 | 11 | 201? JSON-P(rocessing) : 12 | - API for parsing and generating JSON data. 13 | with imperative and streaming API. 14 | 15 | 16 | 2017 JSON-B(inding) : 17 | - API for bidirectional binding between Java objects and 18 | json string. (later spec sitting on top of JSON-P) 19 | Similar in concept to JAXB (Java Object to XML binding) 20 | - Unfortunately the standard falls sort and extensions to 21 | it are needed. For example Jackson allows for inmutable 22 | constructor with @JsonConstructor, but it is not part of 23 | the standard. 24 | ``` 25 | 26 | ## JSON-P(rocessing) /JSR-374 [[{persistence.JSON.json-p,security.TLS/X509,security.aaa,PM.TODO]] 27 | * Patterns of JSON Matching: 28 | * Streaming based, binding based, expression based. 29 | 30 | 31 | * REF:JSON processing public review: 32 | 33 | 34 | * JSR 374, API for JSON Processing (JSON-P) version 1.1. 35 | * Java 8 streams and lambdas alternative to Moshi and Jackson. 36 | * expected to be included in J2EE 8 . 37 | tun* compatible with JSON IETF standards. 38 | * It includes support for: 39 | * JSON Pointer 40 | * JSON Patch 41 | * JSON Merge Patch 42 | * Query and transformation operations 43 | * Designed to parse/generate/query standard JSON documents. 44 | [[persistence.JSON.json-p}]] 45 | 46 | ## JSON-B(binding) / JSR-367 [[{persistence.json,io,]] 47 | 48 | - B stands for Ojbect binding 49 | - Standard binding layer for converting 50 | Java objects to/from JSON messages, defining a default mapping algorithm 51 | for converting existing Java classes to JSON, while enabling developers 52 | to customize it through annotations. 53 | 54 | - Real World REST APIT Example: 55 | ``` 56 | package com.mycomp.project1; 57 | 58 | import java.io.BufferedReader; 59 | import java.io.DataOutputStream; 60 | import java.io.IOException; 61 | import java.io.InputStreamReader; 62 | import java.net.HttpURLConnection; 63 | import java.net.URL; 64 | import java.security.KeyManagementException; 65 | import java.security.NoSuchAlgorithmException; 66 | 67 | import javax.net.ssl.HttpsURLConnection; 68 | import javax.net.ssl.SSLContext; 69 | import javax.net.ssl.SSLSocketFactory; 70 | import javax.net.ssl.HostnameVerifier; 71 | import javax.net.ssl.SSLSession; 72 | import javax.net.ssl.TrustManager; 73 | import javax.net.ssl.X509TrustManager; [security.aaa] 74 | import java.security.cert.X509Certificate; 75 | 76 | import org.json.JSONObject; ←··············· https://github.com/stleary/JSON-java/ 77 | - reference implementation demonstrating: 78 | - how to parse JSON docs to Java objects 79 | - how to generate JSON documents from the Java objects. 80 | - Project goals include: 81 | - Adherence to the JSON spec. 82 | - No external dependencies 83 | - Fast execution and low memory footprint 84 | - It can also convert to/from: 85 | JSON, XML, HTTP headers, Cookies, Comma Delimited Text 86 | (org.json.CDT or CSV). 87 | import java.util.Date; 88 | import java.util.Scanner; 89 | 90 | public class TestAPI { 91 | static String userpass = "operator1:ecllqy"; 92 | private static SSLSocketFactory sslSocketFactory = null; 93 | 94 | private JSONObject sendPost(String url, String post_body, String token) throws Exception 95 | { 96 | URL obj = new URL(url); 97 | String basicAuth = "Basic " + 98 | javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes()); 99 | 100 | HttpsURLConnection con = (HttpsURLConnection) obj.openConnection(); 101 | 102 | setAcceptAllVerifier((HttpsURLConnection)con); // TODO: WARN Add certificate validation. 103 | 104 | con.setRequestMethod("POST"); //add request header 105 | con.setRequestProperty("Content-Type", "application/json"); 106 | con.setRequestProperty("Cache-Control", "no-cache"); 107 | if (token.isEmpty()) { con.setRequestProperty("Authorization", basicAuth); 108 | } else { con.setRequestProperty("Authorization", "Bearer "+token); 109 | } 110 | con.setDoOutput(true); 111 | DataOutputStream wr = new DataOutputStream(con.getOutputStream()); 112 | wr.writeBytes(post_body); 113 | wr.flush(); 114 | wr.close(); 115 | int responseCode = con.getResponseCode(); 116 | 117 | BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream())); 118 | StringBuffer response = new StringBuffer(); 119 | String inputLine; while ((inputLine = in.readLine()) != null) { response.append(inputLine); } 120 | in.close(); 121 | return new JSONObject(response.toString()); //String myJSONStr 122 | } 123 | 124 | /* */ 125 | /** 126 | * Overrides the SSL TrustManager and HostnameVerifier to allow 127 | * all certs and hostnames. 128 | * WARNING: This should only be used for testing, or in a "safe" (i.e. firewalled) 129 | * environment. 130 | * 131 | * @throws NoSuchAlgorithmException 132 | * @throws KeyManagementException 133 | */ 134 | protected static void setAcceptAllVerifier(HttpsURLConnection connection) throws NoSuchAlgorithmException, KeyManagementException { 135 | // Create the socket factory. 136 | // Reusing the same socket factory allows sockets to be 137 | // reused, supporting persistent connections. 138 | if( null == sslSocketFactory) { 139 | SSLContext sc = SSLContext.getInstance("SSL"); 140 | sc.init(null, ALL_TRUSTING_TRUST_MANAGER, new java.security.SecureRandom()); 141 | sslSocketFactory = sc.getSocketFactory(); 142 | } 143 | 144 | connection.setSSLSocketFactory(sslSocketFactory); 145 | 146 | // Since we may be using a cert with a different name, we need to ignore 147 | // the hostname as well. 148 | connection.setHostnameVerifier(ALL_TRUSTING_HOSTNAME_VERIFIER); 149 | } 150 | 151 | private static final TrustManager[] ALL_TRUSTING_TRUST_MANAGER = new TrustManager[] { 152 | new X509TrustManager() { 153 | public X509Certificate[] getAcceptedIssuers() { 154 | return null; 155 | } 156 | public void checkClientTrusted(X509Certificate[] certs, String authType) {} 157 | public void checkServerTrusted(X509Certificate[] certs, String authType) {} 158 | } 159 | }; 160 | 161 | private static final HostnameVerifier ALL_TRUSTING_HOSTNAME_VERIFIER = new HostnameVerifier() { 162 | public boolean verify(String hostname, SSLSession session) { 163 | return true; 164 | } 165 | }; 166 | } 167 | ``` 168 | 169 | ## JSON-B vs GSON vs Moshi vs ... [[{persistence.JSON,performance,io]] 170 | * https://www.reddit.com/r/javahelp/comments/bx6ded/jsonb_vs_jackson/ 171 | ... I've had issues with nested types (example: List) when 172 | decoding Json with most libraries (see usage examples [here](https://www.baeldung.com/java-json), 173 | since they request a class as a parameter, which would be a raw type (in 174 | this example, List.class).
175 | * Gson: deprecated as of 2024. 176 | Gson offers a helper class called TypeToken to circumvent this through 177 | the usage of type instead of class (ex: https://www.baeldung.com/gson-json-to-map). 178 | It's very complicated but at the same time very elegant.
179 | I don't know if other Json libraries offer something equivalent, I 180 | found Google's code to be concise and easy to understand so I didn't 181 | edit: I did a quick search about Json-B (never really read much about 182 | it before) and it doesn't seem to provide any way to handle the 183 | scenario described above. You would still need a library at least to 184 | obtain the type. 185 | [[persistence.JSON}]] 186 | 187 | ## Jackson Summary 188 | 189 | * json file to JsonNode: 190 | ``` 191 | import com.fasterxml.jackson.databind.JsonNode; 192 | import com.fasterxml.jackson.databind.ObjectMapper; 193 | ... 194 | ObjectMapper objectMapper = new ObjectMapper(); 195 | JsonNode jsonNode = objectMapper.readTree(new FileReader("computer.json")); 196 | System.out.println(jsonNode.get("hardware").get("cpu")); 197 | ``` 198 | * JsonNode to String: 199 | ``` 200 | String s = jsonNode.toString(); 201 | ``` 202 | * JsonNode to Class instance : 203 | ``` 204 | TODO: 205 | ``` 206 | 207 | ## GSON 208 | 209 | * JSON library By Google. 210 | * Gson alsoallows to map objects we don't have the source code for. 211 | * Simple API: 212 | ``` 213 | toJson() 214 | fromJson() 215 | ``` 216 | 217 | 218 | [[{io.json.jsoniter]] 219 | ## jsoniter: (Much) faster JSON iterator en/de-coder (Abandoned project) 220 | * WARNS: Looks to be abandoned. 221 | * 222 | * available in Java and Go. 223 | * Up to 3x times faster than jackson/gson/fastjson [[doc_has.comparative]] 224 | [[io.json.jsoniter}]] 225 | 226 | 227 | 228 | -------------------------------------------------------------------------------- /JVM.txt: -------------------------------------------------------------------------------- 1 | # JVM 2 | 3 | [[{JVM,java_lang.profiling,architecture.real-time]] 4 | ## JVM Safepoints 5 | * [[{PM.TODO}]] 6 | 7 | * Definition: Mutator threads: threads which manipulate the JVM heap. 8 | all Java Threads are mutators, Non*Java (native?) threads may also be 9 | regarded as mutators when they call into JVM APIs which interact with the heap. 10 | * Safepoint: 11 | * range of execution where the state of the executing thread 12 | is well described since thread is NOT interacting with the heap: 13 | * used to put mutator threads on hold while the JVM 'fixes stuff up' 14 | * particularly useful to let JVM examine|change the heap (GC, ...) 15 | (no objects still alive and referenced from the stack) 16 | * thread is at safepoints when: 17 | * thread de-scheduling events: thread blocked on lock/synch.lock, waiting on a monitor, parked, 18 | or blocked on blocking*IO. 19 | * thread is executing JNI code. 20 | * thread is NOT at safepoints when: 21 | * executing bytecode (maybe, but not for sure). 22 | * thread interrupted (by the OS) while not at a safepoint. 23 | * JVM cannot force any thread into a safepoint state but ... 24 | JVM can stop threads from leaving a safepoint state. 25 | * Q: How then bring ALL threads into a safepoint? 26 | A: Java threads poll a 'safepoint flag' (global or thread level) at 'reasonable' intervals 27 | and transition into a safepoint state (thread is blocked at a safepoint) when active. 28 | * Q: how to avoid waste time checking if C1/C2 (client/server) JIT compilers need to stop? 29 | how to keep safepoint polls to a minimum. 30 | A: considerations combined lead to the following locations for safepoint polls: 31 | ``` 32 | | - Between any 2 bytecodes while running in the interpreter (effectively) 33 | | - On 'non-counted' loop back edge in C1/C2 compiled code 34 | | - Method entry (Zing,...) or exit (OpenJDK,...) in C1/C2 compiled code. 35 | ``` 36 | * Ex: 37 | ``` 38 | | public class WhenWillItExit { 39 | | public static void main(String[] argc) 40 | | throws InterruptedException { 41 | | const UP = Integer.MAX_VALUE; 42 | | final Thread t = new Thread(() -> { 43 | | long l = 0; 44 | | for (int i = 0; i < UP ; i++) { ┐ Result: 45 | | for (int j = 0; j < UP ; j++) { │ 46 | | if ((j & 1) == 1) l++; ├ long-type loops: 'uncounted' code. safepoints injected at each loop. 47 | | } │ int-type loops: 'counted' code. No safepoints injected. 48 | | } ┘ - (Much) Better performance BUT ... 49 | | System.out.println("How Odd:" + l); other threads forced to suspend at their 50 | | }); next safepoint operation. 51 | | t.setDaemon(true); ┐ 52 | | t.start(); ├ Expected: exit in ~5 seconds. 53 | | Thread.sleep(5000); ┘ Result : no safepoints means threads, JMX connections, ... 54 | | } will have to expect to daemon thread to exits to 55 | | } be able to reach a global safepoint. Use -Xint to 56 | | disable C1/C2 compilation or replace int → long 57 | | in loop index to respect the 5 second behaviour. 58 | ``` 59 | (See original source for many interesing details on safepoint tunning) 60 | 61 | ### SUMMARY 62 | - Safepoint polls are dispersed at fairly arbitrary points and depending 63 | on execution mode, mostly at uncounted loop back edge or method return/entry. 64 | - Bringing the JVM to GLOBAL safepoint is high cost 65 | - ☞ For real-time applications, it's critical to know about safepoints 66 | avoiding 'counted' code. 67 | -XX:+PrintApplicationStoppedTime will log contained safepoint pauses. 68 | 69 | ## Problems with (most) Sampling Profilers 70 | ([REF]) 71 | * A large number of samples needed to get statistically significant results. 72 | * profiler should sample all points in a program run with equal probability 73 | * Generic profilers rely on the JVMTI spec: 74 | * JVMTI offers only safepoint sampling stack trace collection options : 75 | * only the safepoint polls in the running code are visible skipping optimized 76 | (counted-code) for-loops!!! 77 | * samples are biased towards the next available safepoint poll location 78 | * A sample profiler can blame a "cheap method" 9 levels down the stack when the 79 | real culprit is the topmost method loop. 80 | [[}]] 81 | 82 | ## Inside the JVM [[{jvm.101,PM.WiP]] 83 | * 84 | 85 | * JVM anatomy Park 86 | * 87 | * 88 | 89 | ``` 90 | | ┌────────────────────┐ 91 | | │ JVM StartUp thread │ 92 | | └─────────┬──────────┘ 93 | | ┌────────┼───────────┐ 94 | | v v v 95 | | GC Compiler JAVA 96 | | Threads Thread Threads 97 | | ┌┐┌┐┌┐┌┐ ┌┐ ┌┐┌┐┌┐┌┐┌┐┌┐┌┐... 98 | | ││││││││ ││ ││││││││││││││ 99 | | ││││││││ ││ ││││││││││││││ 100 | | ││││││││ ││ ││││││││││││││ 101 | | ││││││││ ││ ││││││││││││││ 102 | | ││││││││ ││ ││││││││││││││ 103 | | ········ ·· ·············· 104 | ``` 105 | * JIT compiler optimization levels: 106 | ``` 107 | | - cold 108 | | - warm 109 | | - hot 110 | | - very hot (with profiling) 111 | | - scorching. 112 | | ^^^^^^^^ 113 | | The hotter the optimization level, the better the 114 | | expected performance, but the higher the cost in terms of 115 | | CPU and memory. See also <#jvm_app_checkpoint> 116 | ``` 117 | [[}]] 118 | 119 | [[{jvm.101,scalability.jvm,monitoring.jvm]] 120 | ## JVM Memory 121 | * REF: 122 | * JAVA MEMORY MODEL: 123 | ``` 124 | | │ STACK ("SMALL") │ HEAP ("HUGE") 125 | | │ private to each Thread │ Shared by Threads 126 | | ─────────┼─────────────────────────────┼────────────────── 127 | | Contain │ - references to heap objects│ - objects 128 | | │ - value types │ - instance fields 129 | | │ - formal method params │ - static fields 130 | | │ - exception handler params │ - array elements 131 | | * 1: Ref(erence) types on the stack point to real object in HEAP memory. 132 | ``` 133 | 134 | * Reference Types regarding how the object on the heap is eligible for garbage collection 135 | 136 | ``` 137 | |┌─────────┬─────────────────────────────────────────────────────────────────────────── 138 | |│ STRONG │ - Most popular. 139 | |│ │ - The object on the heap it is not garbage collected 140 | |│ │ while there is a strong reference pointing to it, or if it is 141 | |│ │ strongly reachable through a chain of strong references. 142 | |├─────────┼─────────────────────────────────────────────────────────────────────────── 143 | |│ WEAK │ - most likely to not survive after the next garbage collection process. 144 | |│ │ - Is created like 145 | |│ │ WeakReference reference = 146 | |│ │ = new WeakReference<>(new StringBuilder()); 147 | |│ │ - Ex.use case: caching: 148 | |│ │ We let the GC remove the object pointed to by the weak reference, 149 | |│ │ after which a null will be returned 150 | |│ │ See JDK implementation at 151 | |│ │ 152 | |├─────────┼─────────────────────────────────────────────────────────────────────────── 153 | |│ SOFT │ - used for more memory-sensitive scenarios 154 | |│ │ - Will be garbage collected only when the application is running low on memory. 155 | |│ │ - Java guarantees that all soft referenced objects 156 | |│ │ are cleaned up before throwing OutOfMemoryError 157 | |│ │ - is created as follows: 158 | |│ │ SoftReference reference = new SoftReference<>(new StringBuilder()); 159 | |├─────────┼─────────────────────────────────────────────────────────────────────────── 160 | |│ PHANTOM │ - Used to schedule post-mortem cleanup actions, since we know for 161 | |│ │ sure that objects are no longer alive. 162 | |│ │ - Used only with a reference queue, since the .get() method of 163 | |│ │ such references will always return null. 164 | |│ │ - These types of references are considered preferable to finalizers 165 | |└─────────┴─────────────────────────────────────────────────────────────────────────── 166 | ``` 167 | [[}]] 168 | 169 | [[{jvm.memory,scalability.cache,scalability.jvm]] 170 | ## Mark&Sweep GC: 171 | * JVM analyzes the variables from the stack and "marks" all the objects that need to be kept alive. 172 | Then, all the unused objects are cleaned up. 173 | * The more garbage there is, and the fewer that objects are marked alive, the faster the process is. 174 | * To optimize even more heap memory actually consists of multiple parts (Java 8+): 175 | ``` 176 | | ──────────┬──────────────────────────────────────────────────── 177 | | HEAP │ 178 | | SPACES │ 179 | | ──────────┼──────────────────────────────────────────────────── 180 | | Eden │ * object are place here upon creation. 181 | | │ * "small" ··> gets full quite fast. 182 | | │ * GC runs on the Eden space and marks objects as alive 183 | | ──────────┼──────────────────────────────────────────────────── 184 | | S0 │ * Eden Objects surviving 1st GC are moved here 185 | | │ 186 | | ──────────┼──────────────────────────────────────────────────── 187 | | S1 │ * Eden Objects surviving 2nd GC are moved here 188 | | │ * S0 Objects surviving GC are moved here 189 | | ──────────┼──────────────────────────────────────────────────── 190 | | Old │ * Object survives for "N" rounds of GC (N depends on 191 | | │ implementation), most likely that it will survive 192 | | │ forever, and get moved here 193 | | │ * Bigger than Eden and S0,S1. GC doesn`t run so often 194 | | ──────────┼──────────────────────────────────────────────────── 195 | | Metaspace│ * metadata about loaded classes 196 | | │ (PermGen Before Java 8) 197 | | ──────────┼──────────────────────────────────────────────────── 198 | | String │ 199 | | pool │ 200 | | ──────────┴──────────────────────────────────────────────────── 201 | ``` 202 | [[}]] 203 | 204 | ## GC Types [[{jvm.memory]] 205 | * default GC type is based on the underlying hardware 206 | * programmer can choose which one should be used 207 | * ``` 208 | | ┌─────────────────────────────────────────────────────────────────────────────┐ 209 | | GC TYPE │ Description / Use-Cases │ 210 | | ┌──────────────┼─────────────────────────────────────────────────────────────────────────────┤ 211 | | │Serial GC │ · Single thread collector. │ 212 | | │ │ · Halt all app threads while executing │ 213 | | │ │ · Mostly applies to small apps with small data usage │ 214 | | │ │ · Can be enabled through : -XX:+UseSerialGC │ 215 | | ├──────────────┼─────────────────────────────────────────────────────────────────────────────┤ 216 | | │Parallel GC │ · Multiple threads used for GC │ 217 | | │ │ · Halt all app threads while executing │ 218 | | │ │ · Also known as throughput collector │ 219 | | │ │ · Can be enabled through : -XX:+UseParallelGC │ 220 | | ├──────────────┼─────────────────────────────────────────────────────────────────────────────┤ 221 | | │Mostly │ · works concurrent to the application, "mostly" not halting threads │ 222 | | │Concurrent GC │ · "mostly": There is a period of time for which the threads are paused. │ 223 | | │ │ Still, the pause is kept as short as possible to achieve the best GC per.│ 224 | | │ │ · 2 types of mostly concurrent GCs: │ 225 | | │ │ * Garbage First · high throughput with a reasonable application pause time│ 226 | | │ │ · Enabled with the option: -XX:+UseG1GC │ 227 | | │ │ Concurrent Mark Sweep: app pause is kept to minimum. Deprecated Java9+*│ 228 | | │ │ · Enabled with the option: -XX:+UseConcMarkSweepGC │ 229 | | └──────────────┴─────────────────────────────────────────────────────────────────────────────┘ 230 | ``` 231 | * [See also](https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/sizing.html) 232 | [[}]] 233 | 234 | 235 | ## sysctl/ulimit+JAVA [[{performance.jvm.linux]] 236 | 237 | Reference (non-mandatory) Linux OS setup for JVM server tasks extracted from: 238 | . 239 | 240 | * ``` 241 | | $ sysctl vm.max_map_count <·· Ensure it's greater than or equal to 524288 242 | | $ sysctl fs.file-max <·· Ensure it's greater than or equal to 131072 243 | | $ ulimit -n <·· Ensure at least 131072 file descriptors 244 | | $ ulimit -u <·· Ensure at least 8192 threads 245 | ``` 246 | * To Modify Kernel limits (permanently) add next lines to `/etc/sysctl.conf` 247 | (or `/etc/sysctl.d/99-sonarqube.conf`) 248 | ``` 249 | | + sysctl -w vm.max_map_count=524288 250 | | + sysctl -w fs.file-max=131072 251 | ``` 252 | * To modify temporarelly (changes lost at system restart) 253 | ``` 254 | | $ ulimit -n 131072 255 | | $ ulimit -u 8192 256 | ``` 257 | * To modify user limits permanently Add next lines to: 258 | * Alt 1: /etc/security/limits.conf (non SystemD) 259 | ``` 260 | | + sonarqube - nofile 131072 ← 261 | | + sonarqube - nproc 8192 262 | ``` 263 | * Alt 2: SystemD unit definition (SystemD) 264 | ``` 265 | | [Service] 266 | | ... 267 | |+ LimitNOFILE=131072 . 268 | |+ LimitNPROC=8192 269 | | ... 270 | ``` 271 | [[}]] 272 | 273 | ## JVM Implementations [[{jvm,PM.WiP]] 274 | 275 | 276 | In practical terms, there is only one set of source code for the JDK. 277 | 278 | * Anyone can take that source code,build and publish it. 279 | * The certification process ensures that the build is valid. 280 | * Certification run by the Java Community Process, which provides a 281 | Technology Compatibility Kit (TCK, sometimes referred to as the JCK). 282 | If build passes the TCK then it is described as "Java SE compatible". 283 | Note: Built can NOT be referred to as "Java SE" without paying a commercial 284 | license from Oracle. 285 | Ex: AdoptOpenJDK passing TCK are "Java SE compatible" (vs "Java SE"). 286 | * WARN : certification is currently on a trust-basis: results are 287 | not submitted to the JCP/Oracle for checking, neither can 288 | be made public. 289 | 290 | * Existing builds include: 291 | * Oracle Java 292 | * OpenJ9 (Eclipse "IBM") 293 | ``` 294 | · https://en.wikipedia.org/wiki/OpenJ9 295 | · Pre-built binaries available at AdoptOpenJDK 296 | · Compared to Oracle's HotSpot VM, i touts higher 297 | start-up performance and lower memory consumption 298 | at a similar overall throughput. 299 | · JIT with all optimization levels. 300 | ``` 301 | * OpenJDK 302 | * GraalVM 303 | * Bellsoft Liberica: 304 | - $free TCK verified OpenJDK distribution for x86, ARM32 and ARM64. 305 | * Azul Systems 306 | * Sap Machine 307 | JDK for Java 10 and later under the GPL+CE license. 308 | They also have a commercial closed-source JVM 309 | * Amazon Corretto: 310 | zero-cost build of OpenJDK with long-term support that passes the 311 | TCK. It is under the standard GPL+CE license of all OpenJDK builds. 312 | Amazon will be adding their own patches and running Corretto on AWS 313 | [[}]] 314 | 315 | ## Jabba JDK Version Management [[{]] 316 | 317 | * pain-free JDK installing on Linux x86/x86_64/ARMv7+, macOS, Windows x86_64. 318 | * Support for: 319 | ``` 320 | · Oracle JDK (latest-version only) 321 | · Oracle Server JRE (latest-version only), 322 | · Adopt OpenJDK (jabba >=0.8.0 is required) 323 | Hotspot 324 | Eclipse OpenJ9 325 | · Zulu OpenJDK (jabba >=0.3.0 is required) 326 | · IBM SDK, Java Technology Edition (jabba >=0.6.0 is required) 327 | · GraalVM CE 328 | · OpenJDK 329 | · OpenJDK Reference Implementation 330 | · OpenJDK with Shenandoah GC (jabba >=0.10.0 is required) 331 | · Liberica JDK 332 | · Amazon Corretto 333 | ``` 334 | 335 | ### INSTALL/UPGRADE: 336 | ``` 337 | $ $ curl -sL https://github.com/shyiko/jabba/raw/master/install.sh | bash 338 | $ $. ~/.jabba/jabba.sh 339 | 340 | Use $ ... | bash --skip-rc to avoid modifying common rc files. 341 | In that case add next lines to .bashrc / ... 342 | + export JABBA_VERSION=... 343 | + [ -s "$JABBA_HOME/jabba.sh" ] && source "$JABBA_HOME/jabba.sh" 344 | ``` 345 | 346 | * Daily use: 347 | ``` 348 | | $ jabba ls-remote ← list available JDK's 349 | | $ jabba ls-remote zulu@~1.8.60 ← Narrow results 350 | | $ jabba ls-remote --latest=minor\ ← semver allowed 351 | | "*@>=1.6.45 <1.9" 352 | | 353 | | $ jabba ls ← list all installed JDK's 354 | | $ jabba use adopt@1.8 355 | | $ jabba use zulu@~1.6.97 356 | | $ echo "1.8" > .jabbarc ← switch to JDK in .jabbarc 357 | | It must be a valid YAML file. 358 | | 'jdk: 1.8' or simply '1.8' are valid 359 | | $ jabba alias default 1.8 ← set def.java ver. on shell (since 0.2.0) 360 | | automatically used on new terminals 361 | | 362 | | $ jabba install 1.15.0 ← install Oracle JDK 363 | | $ jabba install sjre@1.8 ← install Oracle Server JRE 364 | | $ jabba install adopt@1.8-0 ← install Adopt OpenJDK (Hotspot) 365 | | $ jabba install adopt-openj9@1.9-0 ← install Adopt OpenJDK (Eclipse OpenJ9) 366 | | $ jabba install zulu@1.8 ← install Zulu OpenJDK 367 | | $ jabba install ibm@1.8 ← install IBM SDK, Java Technology Edition 368 | | $ jabba install graalvm@1.0-0 ← install GraalVM CE 369 | | $ jabba install openjdk@1.10-0 ← install OpenJDK 370 | | $ jabba install openjdk-shenandoah@1.10-0 ← install OpenJDK with Shenandoah GC 371 | | └──┬──┘ 372 | | everything is installed under ~/.jabba. Removing this directory clean install 373 | | $ jabba uninstall zulu@1.6.77 ← uninstall JDK 374 | | $ jabba link system@1.8.72 \ ← link system JDK 375 | | /usr/lib/jvm/jdk1.8.0_72.jdk 376 | ``` 377 | * To modify JDK system-wide: 378 | ``` 379 | | $ sudo update-alternatives \ 380 | | --install /usr/bin/java java \ 381 | | ${JAVA_HOME%*/}/bin/java 20000 382 | | $ sudo update-alternatives \ 383 | | --install /usr/bin/javac javac \ 384 | | ${JAVA_HOME%*/}/bin/javac 20000 385 | ``` 386 | * To swith among GLOBAL JDK system-wide: 387 | ``` 388 | | $ sudo update-alternatives --config java 389 | ``` 390 | [[}]] 391 | 392 | -------------------------------------------------------------------------------- /TornadoVM.txt: -------------------------------------------------------------------------------- 1 | # TornadoVM 2 | 3 | TornadoVM is an open-source software technology that automatically 4 | accelerates Java programs on multi-core CPUs, GPUs, and FPGAs. 5 | 6 | 7 | ## TornadoVM Plugin for IntelliJ [[{scalability.JVM.TornadoVM,qa.UX]] 8 | * 9 | * TornadoVMTornadoInsight is an open-source IntelliJ IDEA plugin for 10 | enhancing the developer experience when working with TornadoVM. 11 | * It was introduced by the TornadoVM team this past week. 12 | * Key features include: 13 | * an on-the-fly static checker that scans TornadoVM code in real-time 14 | and reports any Java features that are not supported by TornadoVM. 15 | * A dynamic testing framework to simplify the testing process for 16 | individual TornadoVM tasks. 17 | [[scalability.JVM.TornadoVM}]] 18 | 19 | ## Running on XPUs via Docker 20 | 21 | * 22 | 23 | -------------------------------------------------------------------------------- /async_reactive.txt: -------------------------------------------------------------------------------- 1 | # Async/Reactive Arch in Java Summary 2 | 3 | ``` 4 | Document status: Draft/WiP 5 | ``` 6 | 7 | * NOTE: Reactive (to events) programming is just an oppinionated (but efficient and bug "free") way 8 | to implement asyncronous architectures. 9 | 10 | [[{architecture.async.reactive.javalite,PM.low_code,messaging.jms,messaging.activemq,architecture.async,architecture.batch,architecture.distributed]] 11 | ## JavaLite lightweight async job processing 12 | 13 | * Use Cases: 14 | * website that needs to run batch process in background 15 | * We receive batch of inputs to be processed in a "best-effort" 16 | but replay as-soon-as-possible to client that we have the 17 | input batch ready for processing. 18 | * easy-to-use "Wrapper" on top o Apache Artemis (ActiveMQ "next Gen") 19 | the Async adds an abstraction layer based on a Command Pattern, 20 | which makes it trivial to add asynchronous processing. 21 | * Embedded broker instance with reasonable defaults 22 | * Ex: 23 | ``` 24 | Async async = new Async( ← CREATE one or more Queues 25 | "/opt/project1", ← place to store persistent messages 26 | false, 27 | new QueueConfig( 28 | "MESSAGES_QUEUE", ← queue 1 name 29 | new CommandListener(), 30 | 5) ← number of listeners(threads) to 31 | , 32 | new QueueConfig( 33 | "ERRORS_QUEUE", ← queue 2 name (No limit in the number o 34 | new ErrorListener(), queues) 35 | 5) 36 | ); 37 | async .start(); 38 | 39 | public class HelloCommand ← Create a command 40 | extends Command { 41 | private String message; 42 | 43 | public HelloCommand(String message) { 44 | this.message = message; 45 | } 46 | 47 | public HelloCommand() {} ← necessary (forbid finals) 48 | 49 | @Override 50 | public void execute() { 51 | System.out.println(message); 52 | } 53 | } 54 | 55 | for(int i = 0; i < 100; i++){ ← sending commands async 56 | async.send( 57 | "MESSAGES_QUEUE", 58 | new HelloCommand("Hello Number "+ i)); 59 | } 60 | Output will be similar to 61 | → Hello Number 0 62 | → Hello Number 1 63 | → ... 64 | 65 | List topCommands ← Peek (vs Consume) 3 "top" 66 | = async .getTopCommands( commands from "ERROR_QUEUE" 67 | 3, "ERROR_QUEUE"); 68 | ``` 69 | * Commands can be read and processed synchronously (one-by-one) 70 | from an individual queue one at the time without a listener.
71 | Ex: you do not want to process errors automatically . To do so: 72 | ``` 73 | ErrorCommand errorCommand = ← Consume message 74 | (ErrorCommand)receiveCommand("ERROR_QUEUE"); 75 | ... // Process manually 76 | ``` 77 | 78 | * Text vs Binary messages 79 | * To be compatible with JMS the communication protocol is limited to 80 | ``` 81 | - javax.jms.TextMessage ← Default mode. 82 | - javax.jms.BytesMessage ← async.setBinaryMode(true); 83 | ``` 84 | In both cases, the serialization of a command is first done to XML 85 | with the use of XStream. 86 | * If a given command has transient field that must NOT be serialized, 87 | use the field annotation @XStreamOmitField to ignore it. 88 | * WARN: Do not switch from mode to mode while having persistent 89 | messages stored in your queues. 90 | * Commands with DB access 91 | * If queue processing requires a database connetion, DBCommandListener 92 | can be used: 93 | ``` 94 | Async async = new Async(filePath, false, new 95 | QueueConfig("MESSAGES_QUEUE", new 96 | DBCommandListener( ← If JNDI connection is setup, the 97 | "java:comp/env/jdbc/conn01"), 5) listener will find and open it 98 | ); Use your tomcat/Jboss/... container 99 | ``` 100 | 101 | * (Artemis) Config API: For complex app configuration, the underlying Artemis API 102 | can be used: 103 | ``` 104 | org.apache.activemq.artemis.core.config.Condifuration 105 | artemisCOnfig = async.getConfig(); 106 | ``` 107 | * See also filequeue. It's faster but doesn't support Queue to DDBBs. 108 | [[architecture.async.reactive.javalite}]] 109 | 110 | [[{architecture.async.reactive.FileQueue,PM.low_code,messaging.jms,messaging.activemq,architecture.async,scalability.101]] 111 | ## FileQueue 112 | 113 | * KISS alternative using MVStore 114 | 115 | * All producers and consumers run within a JVM. 116 | * H2 MVStore DB used for storage. 117 | * Queue items are POJOs serialized into Json using jackson . 118 | * faster than JavaLite due to performance shortcut : 119 | * File Queue will transfer queued items directly to consumers 120 | without hitting the database provided there are consumers 121 | available, otherwise, message will be persisted 122 | * Doesn't support persistence to JNDI DDBB 123 | * fixed and exponential back-off retry is present. 124 | 125 | * PRE-SETUP : 126 | * maven/gradle package dependency: 127 | `com.stimulussoft:filequeue:1.1.4` 128 | 129 | * Usage: 130 | * Implement POJO extending FileQueueItem 131 | * Implement consume(FileQueueItem) on <> to process items 132 | * Instantiate a FileQueue object and call config() to configure 133 | * Call startQueue() to start the queue 134 | * Call stopQueue() to stop the queue processing 135 | * Call FileQueue.destroy() to shutdown all static threads (optional) 136 | 137 | * Queue Ussage example: 138 | ``` 139 | FileQueue queue = FileQueue.fileQueue(); 140 | FileQueue.Config config = FileQueue. 141 | config( 142 | queueName, 143 | queuePath, 144 | TestFileQueueItem.class, 145 | new TestConsumer() 146 | ) 147 | .maxQueueSize(MAXQUEUESIZE) // ← queueItem will block until an slot becomes 148 | available or ExceptionTimeout thrown 149 | .retryDelayAlgorithm( 150 | QueueProcessor.RetryDelayAlgorithm.EXPONENTIAL) 151 | .retryDelay(RETRYDELAY) 152 | .maxRetryDelay(MAXRETRYDELAY) 153 | .maxRetries(0); // ← Infinite retries 154 | .persistRetryDelay( // ← delay between DDBB scans. 155 | PERSISTENTRETRYDELAY); 156 | queue.startQueue(config); // ← Start queue 157 | for (int i = 0; i < ROUNDS; i++) 158 | queue.queueItem( // ← Submit items 159 | new TestFileQueueItem(i) 160 | ); 161 | queue.stopQueue(); // ← stopQueue 162 | ``` 163 | * Consumer example implementation: 164 | ``` 165 | static class TestConsumer implements Consumer { 166 | public TestConsumer() { } 167 | 168 | @Override 169 | public Result consume(FileQueueItem item) 170 | throws InterruptedException { 171 | try { 172 | TestFileQueueItem retryFileQueueItem = 173 | (TestFileQueueItem) item; 174 | if (retryFileQueueItem.getTryCount() == RETRIES ) 175 | return Result.SUCCESS; 176 | return Result.FAIL_REQUEUE; 177 | } catch (Exception e) { 178 | logger.error(e.getMessage(), e); 179 | return Result.FAIL_NOQUEUE; 180 | } 181 | } 182 | } 183 | ``` 184 | * FileQueueItem implementation: 185 | ``` 186 | import com.stimulussoft.filequeue.*; 187 | 188 | static class TestFileQueueItem extends FileQueueItem { 189 | Integer id; 190 | public TestFileQueueItem() { super(); }; 191 | private TestFileQueueItem(Integer id) { 192 | this.id = id; 193 | } 194 | @Override 195 | public String toString() { return String.valueOf(id); } 196 | public Integer getId() { return id; } 197 | public void setId(Integer id) { this.id = id; } 198 | } 199 | ``` 200 | #### File Caching: 201 | If there is the need to cache a file to disk or perform resource 202 | availability checks prior to items being placed on the queue, 203 | implement availableSlot() on the QueueCallback interface. This method 204 | is called as soon as a slot becomes available, just before the item 205 | is place on the queue. It may be used to cache a file to disk, or 206 | perform resource availability pre-checks (e.g. disk space check). 207 | 208 | [[architecture.async.reactive.FileQueue}]] 209 | 210 | ## java.util.concurrent.Flow(1.9+) [[{java_lang.functional,architecture.async.reactive.io,io.101,PM.TODO]] 211 | 212 | * JDK 1.9+ 213 | * Reactive Streams was adopted by the JDK in the form of the java.util.concurrent.Flow API. 214 | * It allows two different libraries that support asynchronous 215 | streaming to connect to each other, with well specified semantics 216 | about how each should behave, so that backpressure, completion, 217 | cancellation and error handling is predictably propagated between the 218 | two libraries. 219 | * There is a rich ecosystem of open source libraries that support 220 | Reactive Streams, and since its inclusion in JDK9, there are a few in 221 | development implementations that are targetting the JDK, including 222 | the incubating JDK9 HTTP Client, 223 | and the [Asynchronous Database Adapter (ADBA)](https://twitter.com/brunoborges/status/915302682939711488) 224 | * See also 225 | [[}]] 226 | -------------------------------------------------------------------------------- /containerization.txt: -------------------------------------------------------------------------------- 1 | [[{containerization,image.build.java]] 2 | # DevOps: Containerization 3 | 4 | ## Dockerfile "pipeline": mvn to Container [[{devops.containarization,devops.maven,devops.k8s,doc_has.diagram,PM.TODO]] 5 | 6 | * Example Dockerfile 7 | ``` 8 | | FROM maven:3.6-jdk-12-alpine as build <·· state 1) Use build image 9 | | WORKDIR /builder 10 | | ADD pom.xml /builder/pom.xml 11 | | ADD src /builder/src 12 | | 13 | | RUN mvn install -DskipTests=true 14 | | 15 | | 16 | | FROM gcr.io/distroless/java:11 <·· stage 2) copy build to distroless ¹ runtime image 17 | | alter openjdk:11-jre 18 | | ARG APP_NAME_ARG=myapp-1.0.jar 19 | | ENV APP_PROFILE docker 20 | | ENV APP_NAME $APP_NAME_ARG 21 | | WORKDIR /app 22 | | COPY --from=build /builder/target/$APP_NAME /app 23 | | COPY --from=build /builder/src/main/resources /app/src/main/resources 24 | | 25 | | EXPOSE 8080 26 | | 27 | | # CMD java -Dspring.profiles.active=$APP_PROFILE -jar $APP_NAME <·· Alt 1: java as entrypoint 28 | | CMD dockerize \ <·· Alt 2: dockerize as entrypoint 29 | | -template /config.tmpl:/app/config.xml \ to make app docker friendly 30 | | -stdout /app/app.log -stderr /app/error.log \ 31 | | java -Dspring.profiles.active=$APP_PROFIE -jar $APP_NAME 32 | | 33 | | 34 | | "Distroless" images contain only your application and its runtime dependencies. 35 | | They do not contain package managers, shells or any other programs you would 36 | | expect to find in a standard Linux distribution. 37 | ``` 38 | [[image.build.java}]] 39 | 40 | ## Jib: Image Builder 41 | * Docker+mvn/gradle integration: 42 | 43 | 44 | 45 | * Build Java Container without Docker/Dockerfile 46 | * Jib's build approach separates the Java application 47 | into multiple layers, so when there are any code changes, 48 | only those changes are rebuilt, rather than the entire application. 49 | * these layers are layered on top of a distroless base image. 50 | containing only the developer's application and its runtime deps. 51 | 52 | * Docker build flow 53 | ``` 54 | │JAR│ ← (build) ← │Project│ 55 | · │Container │ 56 | ├·····→ │Build Context│ →(build)→ │ Container Image │ →(push)→ │Image │ 57 | · │ (docker cache) │ │(registry)│ 58 | │Dockerfile│ 59 | 60 | Jib Build Flow: 61 | │Container │ 62 | │Project│ ───────────────(Jib)─────────────────────────────────────→│Image │ 63 | │(registry)│ 64 | ``` 65 | 66 | * Ex: Creating images from command line: 67 | Once jib is installed and added to PATH, to create a new image do something like: 68 | ``` 69 | $ /opt/jib/bin/jib 70 | --insecure \ ← allow conn. to HTTP (non TLS) dev.registries 71 | build \ ← build image 72 | --registry \ ← Push to registry 73 | gcr.io/distroless/java \ ← Base image (busybox, nginx, gcr.io/distroless/java,...) 74 | 192.168.1.3:5000/jib:latest \ ← Destination registry / image 75 | --entrypoint "java,-cp,/app/lib/*,\ 76 | com.google.cloud.tools.jib.cli.JibCli" \ 77 | build/install/jib/lib,/app/lib 78 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 79 | Other options include: 80 | (See jib build --help for more options) 81 | 82 | p=perms set file and directory permissions: 83 | actual use actual values in file-system 84 | fff:ddd octal file and directory 85 | (Default to 644 for files and 755 for dir.) 86 | ts=timestamp set last-modified timestamps: 87 | actual use actual values in file-system 88 | "seconds since Unix epoch" 89 | "date-time in ISO8601 format" 90 | (Default to 1970-01-01 00:00:01 UTC). 91 | -a, --arguments=arg container entrypoint's default arguments 92 | -c, --creation-time=time Set image creation time (default: 1970-01-01T00:00:00Z) 93 | -l, --label=key=val[,key=va l...] 94 | -p, --port=port[,port...] Expose port/type (ex: 25 or 25/tcp) 95 | -u, --user=user Set user for execution (uid or existing user id) 96 | -V, --volume=path1,path2... Configure specified paths as volumes 97 | ``` 98 | 99 | * Ex pom.xml to create tomcat container with war: 100 | REF: 101 | 102 | ``` 103 | $ mvn clean package jib:dockerBuild 104 | $ docker run --rm -p 8082:8080 \ 105 | registry.localhost/hello-world:latest 106 | 107 | 108 | 112 | 4.0.0 113 | 114 | org.example 115 | mvn-jib-example 116 | 1.0 117 | war 118 | 119 | 120 | UTF-8 121 | false 122 | 123 | 124 | 125 | 126 | javax.servlet 127 | javax.servlet-api 128 | 4.0.1 129 | provided 130 | 131 | 132 | 133 | 134 | servlet-hello-world 135 | 136 | 137 | org.apache.maven.plugins 138 | maven-compiler-plugin 139 | 3.8.1 140 | 141 | 1.8 142 | 1.8 143 | 144 | 145 | 146 | com.google.cloud.tools 147 | jib-maven-plugin 148 | 2.5.0 149 | 150 | true 151 | tomcat:9.0.36-jdk8-openjdk 152 | 153 | registry.localhost/hello-world 154 | ... ... 155 | 156 | latest 157 | 158 | 159 | 160 | /usr/local/tomcat/webapps/ROOT 161 | 162 | 163 | 164 | 165 | ./src/main/resources/extra-stuff 166 | /path/in/docker/image/extra-stuff 167 | 168 | 169 | /absolute/path/to/other/stuff 170 | /path/in/docker/image/other-stuff 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | ``` 180 | 181 | * alternatives to Distroless include Chainguard images and Chiselled Ubuntu.
182 | Extracted from 183 | 184 | > Chiselled Ubuntu containers solve a similar need for cut-down 185 | > container base images as Google's Distroless and Chainguard's images, 186 | > bringing the same benefits such as minimising dependency challenges, 187 | > reducing bloat and resource usage, speeding up startup, and enhancing 188 | > security through reducing the number of unneeded files in the image. 189 | > Chisel itself uses Slice Definition Files, which relate to the 190 | > upstream packages in the Ubuntu archives, defining subsets of those 191 | > package contents needed at runtime. This provides fine-grained 192 | > dependency management through a developer-friendly CLI, enabling more 193 | > efficient containerization with enhanced security by reducing the 194 | > container image attack surface and entirely eliminating some 195 | > potential attack vectors. 196 | 197 | See also: jKube notes 198 | [[devops.containarization}]] 199 | 200 | ## From 318.7 MB to 32 MB [[{devops.containarization.101,qa.101]] 201 | ``` 202 | ┌───────────────────────┬───────────┬────────────────────────────┬─────┐ 203 | │ │ Modules │ **jlink flags** │ MB │ 204 | │───────────────────────│───────────│────────────────────────────│─────│ 205 | │ JDK 12 │ whole SDK!│ (no flags) │318.7│ 206 | │───────────────────────│───────────│────────────────────────────│─────│ 207 | │openjdk-11-jre-slim 11 │(default) │ │217.0│ 208 | │(12-jre-slim pending) │ │ │ │ 209 | ├───────────────────────┼───────────┼────────────────────────────┼─────┴─────┐ 210 | │ JRE 12 │all │ -add-module │168.3│ 100%│ 211 | │ │(explicit) │ $(java --list-modules) │ │ │ 212 | ├───────────────────────┼───────────┼────────────────────────────┼─────┼─────┤ 213 | │ JRE 12 │ │ --no-header-files \ │143.0│ 85%│ 214 | │ │ │ --no-man-pages \ │ │ │ 215 | │ │ │ --strip-debug \ │ │ │ 216 | │ │ │ ───────────────────────────┼─────┼─────┤ 217 | │ │ │ --compress=1 \ │ 07.8│64.1%│ 218 | │ │ │ ───────────────────────────│─────┼─────┤ 219 | │ │ │ --compress=2 │ 83.7│49.7%│ 220 | ├───────────────────────┼───────────┼────────────────────────────┼─────┼─────┼─────┐ 221 | │ Custom JRE 12 │ base, │ --add-module \ │ 47.4│28.2%│ 100%│ 222 | │ │ logging │ $(jdeps │ │ │ │ 223 | │ │ │ --print-modules-deps │ │ │ │ 224 | │ │ │ func.jar) \ │ │ │ │ 225 | │ │ ├────────────────────────────┼─────┼─────┼─────┤ 226 | │ │ │ --no-header-files \ │ 41.6│24.7%│87.8%│ 227 | │ │ │ --no-man-pages \ │ │ │ │ 228 | │ │ │ --strip-debug \ │ │ │ │ 229 | │ │ ├────────────────────────────┼─────┼─────┼─────┤ 230 | │ │ │ --compress=2 │ 32.0│19.0%│67.5%│ 231 | └───────────────────────┴───────────┴────────────────────────────┴─────┴─────┴─────┘ 232 | [[devops.containarization.101}]] 233 | 234 | ``` 235 | 236 | [[{devops.containarization.101,PM.TODO]] 237 | ## JLink 238 | 239 | * 240 | * You can use the jlink tool to assemble and optimize a set of modules 241 | and their dependencies into a custom runtime image 242 | 243 | TODO: 244 | 245 | * 246 | [[}]] 247 | 248 | [[containerization}]] 249 | -------------------------------------------------------------------------------- /functional_java.txt: -------------------------------------------------------------------------------- 1 | # Functional Programming 2 | 3 | [[{java_lang.101,java_lang.functional.lambdas,architecture.async.reactive]] 4 | ## `TLDR` Examples: 5 | ``` 6 | | int counter = 1; 7 | | val stream01 = List.of(1,2,3,4,5,6,7,9,10).stream(); // <·· Init from List 8 | | val stream02 = IntStream.of(1,2,3,4,5,6,7,9,10); // <·· Init from int[] 9 | | val stream03 = Stream.generate(() -> counter++).limit(10); // <·· stream02 == stream03 10 | | val Stream stream04 = 11 | | Stream.iterate(10, n -> n).limit(10); // <·· stream generated "on demand" 12 | | 13 | | intStream1_10 14 | | .skip(2) // <·· discard first 2 elements 15 | | .limit(3) // <·· returns stream no longer than 3 elements. 16 | | // └──────┴─ skip + limit == "paginate". 17 | | .filter( i -> i % 2 == 0 ) // <·· filter out if result is not true 18 | | // └─────────────┴─················ Java 8+ lambda function ¹ 19 | | .distinct() // <·· Remove duplicated. (a "hash strategy can be used") 20 | | .map(i -> i*i) // <·· map to new elements. 21 | | ... 22 | | .findFirst() // <·· Alt 1) Complete stream to single element 23 | | .peek ( e -> System.out.println(e)) // <·· Debug streams without changes [[{troubleshooting.stream}]] 24 | | .orElse(arr.length); 25 | | 26 | | // "Complete" stream 27 | | .toArray(); // <·· Alt 2) Complete stream to array 28 | | .count(); // <·· Alt 3) Complete stream to element number 29 | | .collect(Collectors.groupingBy(func)); // <·· Alt 4) Complete stream to Map grouping by func 30 | | .collect(Collectors.toList()); // <·· Alt 5) Complete stream to List 31 | | .collect(Collectors.toCollection(TreeSet::new)) // 6) Complete stream to TreeSet 32 | | 33 | | Stream.concat( stream1, stream2 ).filter(...) // concat streams 34 | | 35 | | Stream stream = intStream1_10.boxed(); // <·· box primitive in Integer 36 | | needed to collect to list for example. 37 | | ¹ Lambda syntax ----------------------------- 38 | | (parameters) -> expression 39 | | (parameters) -> { statements; } 40 | | 41 | | Function f = (l) -> l.toString(); // <·· takes a Long , returns a String 42 | | Supplier s = Thread::currentThread; // <·· takes nothing, returns a Thread 43 | | Consumer c = System.out::println; // <·· takes a string as the parameter 44 | ``` 45 | 46 | 47 | [[{java_lang.functional.jOOλ,java_lang.functional.101,troubleshooting.stream,troubleshooting.checked_exceptions,PM.TODO]] 48 | ## jOOλ functional extensions to Java 8 49 | 50 | * [jOOλ](https://github.com/jOOQ/jOOL) 51 | 52 | - **Problem Context**: "Pains with checked exceptions and lambdas" [[java_lang.101]] 53 | - 54 | - 55 | - [[java_lang.101}]] 56 | - **Solution**: `org.jooq.lambda.Unchecked` 57 | 58 | ``` 59 | | STANDARD JAVA 8 60 | | Arrays 61 | | .stream(dir.listFiles()) 62 | | .forEach(file -> { 63 | | try { 64 | | System.out.println( 65 | | file.getCanonicalPath()); 66 | | } catch (IOException e) { 67 | | throw new RuntimeException(e); 68 | | } 69 | | }); 70 | ``` 71 | * using jooq labmda wrapper 72 | 73 | ``` 74 | | using jooq labmda wrapper: 75 | | ========================== 76 | | Arrays 77 | | .stream(dir.listFiles()) 78 | | .forEach( 79 | | Unchecked.consumer(file -> { 80 | | System.out.println( 81 | | file.getCanonicalPath()); 82 | | }) 83 | | ); 84 | ``` 85 | * Even simpler 86 | ``` 87 | | Arrays 88 | | .stream(dir.listFiles()) 89 | | .map(Unchecked.function( 90 | | File::getCanonicalPath)) 91 | | .forEach(System.out::println); 92 | ``` 93 | [[java_lang.functional.jOOλ}]] 94 | 95 | [[java_lang.functional.lambdas}]] 96 | 97 | ## Streams and Optional are Monads [[{java_lang.functional.101]] 98 | 99 | * two of the most commonly known Java 8 features are monad 100 | implementations, namely Stream and Optional. 101 | * Monad is a concept: we can view it as a wrapper which puts our 102 | value in some context and allows us to perform operations on the 103 | value. In this context, the output of an operation at any step is the 104 | input to the operation at the next step. 105 | [[java_lang.functional.101}]] 106 | 107 | ## Unix4j [[{java_lang.101,PM.low_code.unix4j,java_lang.functional]] 108 | * [Link@Github](https://github.com/tools4j/unix4j) 109 | * High level functional programming "the Unix way". 110 | 111 | > ...Working in the finance industry in Melbourne, the authors of 112 | > Unix4j spent a lot of time writing complex text processing 113 | > applications in bash. Frustrated with the inherent limitations of the 114 | > bash language; lack of language support, IDEs, test frameworks etc, 115 | > the authors decided to try and bring the convenience of some of the 116 | > Unix commands into the Java language. 117 | > You may notice that the implemented commands are more bent towards 118 | > text processing rather than OS and file manipulation. This is 119 | > intended as we see text processing to be the real benefit of Unix4j. 120 | > Not to say that this will always be the case. 121 | 122 | * Example Ussage: 123 | ``` 124 | | Unix4j.cat("test.txt").grep("Tuesday").sed("s/kilogram/kg/g").sort(); 125 | ``` 126 | [[PM.low_code.unix4j}]] 127 | 128 | # Functional Programming TODO 129 | 130 | ## Java Flow 131 | * 132 | [[{architecture.async.reactive.101,java_lang.flow_api,java_lang.101,PM.TODO]] 133 | * 134 | Trisha Gee shows via live coding how we can use the new Flow API to 135 | utilize Reactive Programming, how the improvements to the Streams API 136 | make it easier to control real-time streaming data and how the 137 | Collections convenience methods simplify code. She talks about other 138 | Java 9 features, including some of the additions to interfaces and 139 | changes to deprecation. 140 | [[}]] 141 | 142 | 143 | 144 | -------------------------------------------------------------------------------- /jakarta_EE.txt: -------------------------------------------------------------------------------- 1 | # What's New: 2 | 3 | ### Jakarta EE 11: 4 | 5 | * 6 | * On the road to MicroProfile 7.0, the second release candidate of 7 | MicroProfile Telemetry 2.0 provides notable changes such as: expose 8 | the OpenTelemetry APIs for an improved user experience; and an update 9 | to the MicroProfile Metrics TCK to ensure that metrics are present. 10 | Further details on this release may be found in the release notes. 11 | -------------------------------------------------------------------------------- /jee_war_apps.txt: -------------------------------------------------------------------------------- 1 | 2 | # Jakarta EE Core Profile 11 3 | 4 | 5 | * 6 | 7 | * targeted at developers of with focus on: 8 | * a minimal set of APIs for microservices. 9 | * enabling implementations that make use of ahead of time compilation build 10 | to minimize startup time and memory footprint. 11 | * Based on JDK 17 API 12 | 13 | * Required Components: 14 | * Jakarta Annotations 3.0* 15 | * Jakarta Contexts and Dependency Injection (CDI) 4.1 (CDI Lite section)* 16 | * Jakarta Dependency Injection 2.0 17 | * Jakarta Interceptors 2.2* 18 | * Jakarta JSON Processing 2.1 + JSON Binding 3.0 19 | * Jakarta RESTful Web Services 4.0* 20 | 21 | 22 | * Optional Components 23 | * Jakarta Expression Language support is not required in Jakarta 24 | Contexts and Dependency Injection Lite implementations. 25 | * The Java SE section of the CDI 4.1 specification is not required 26 | for Core Profile. 27 | 28 | # JEE WARs 29 | 30 | 31 | ## `WEB-INF/web.xml` [[{]] 32 | 33 | ``` 34 | $ mvn archetype:generate \ 35 | -DgroupId=com.mycomp.myprj \ 36 | -DartifactId=mylib -Dversion=1-SNAPSHOT 37 | output will be similar to: 38 | mylib/pom.xml 39 | mylib/src/main/resources/ 40 | mylib/src/main/resources/static <·· Spring and others will "pick up" all the content and put 41 | it in the base path of the web application. 42 | mylib/src/main/java/ 43 | mylib/src/webapp/WEB-INF/ <·· Anything inside WEB-INF is not visible to external apps 44 | mylib/src/webapp/WEB-INF/web.xml ¹ 45 | mylib/src/webapp/WEB-INF/index.jsp <·· Not even index.jsp 46 | 47 | 48 | ¹ mylib/src/webapp/WEB-INF/web.xml <·· Anything inside WEB-INF is not visible to external apps 49 | WEB-INF/web.xml 50 | 51 | 52 | MyService01 <·· name for servlet(or JSP)/compiled class 53 | 54 | com.myCompany.myDepartment.myProject.MyService01 55 | 56 | <·· config param for this servlet 57 | parametername (vs global contex-param for all servlets) 58 | parametervalue No other servlet can access it. 59 | 60 | 1 <·· Init on start up. It also indicates in 61 | which order if N servlets exists. 62 | Optional but good practice. 63 | Lower numbers are loaded first. 64 | Otherwise it is loaded on demand (on first 65 | "arriving" request). on-demand loading 66 | is more risky since we will not see configuration 67 | bugs on startup but just "later on" in a 68 | random future when the first request comes in. 69 | 70 | ... <·· TODO: servlet security roles. 71 | 72 | 73 | 74 | myservlet2 75 | /v1/service1 <·· URL path param. WildCards also allowed. 76 | Ex: *.doc 77 | 78 | <·· Global config param for all servlets 79 | debug In code: 80 | true String debug=request.getSession() 81 | .getServletContext() 82 | .getInitParameter("debug"); 83 | 84 | jdbc_db1 85 | jdbc://... 86 | 87 | ... <·· TODO: Filters and fiter configuration 88 | Authentication, 89 | Data compression 90 | Logging and Auditing 91 | Image conversion 92 | Encryption 93 | Tokenizing 94 | Filters triggering resource access events 95 | .... 96 | 97 | 98 | Servlet 1<···> ServletConfig object. 99 | --------------------- 100 | created by the web container for each servlet from web.xml 101 | + String getInitParameter(String param_name) // returns param_value 102 | + Enumeration getInitParameterNames() // returns (all) parameter names list 103 | + String getServletName(). 104 | + ServletContext getServletContext() <··· Common config to all Servlets 105 | └─────┬──────┘ 106 | * The ServletContext defines a set of methods used by a servlet to communicate 107 | with its servlet container, such as to get the MIME type of a file, to write 108 | into a log file, to dispatch requests. 109 | Usually, there is one context is per Web application inside each JVM. 110 | WARN: The context can NOT be used as a location to share global information. 111 | * There is also a method to return the name and version of 112 | the servlet container on which the servlet is running. We can get the 113 | context path of the web application. 114 | * We can get a SessionCookieConfig object form a servlet context through 115 | which various properties of the session tracking cookies created. 116 | * There is a possibility to set the session tracking modes(COOKIE, SSL, URL) 117 | that are to become effective for this ServletContext. 118 | You can set several types of them at once. 119 | * With a ServletContext object, we can write a specified explanatory message 120 | with or without stack trace for a given Throwable exception to a servlet 121 | log file, usually an event log. 122 | * There is a possibility to manage a servlet filter with the use of 123 | ServletContext object. A filter object performs filtering tasks on the 124 | request to a resource, on the response from a resource. It can also filter 125 | both. A resource can be a servlet or static content. 126 | 127 | 128 | class ... extends HttpServlet { 129 | 130 | public void doGet(...) throws ... { 131 | ServletConfig configuration = getServletConfig(); // <·· config of just this servlet 132 | ServletContext context = configuration.getServletContext(); // <·· Global config. 133 | Enumeration paramNames = context.getInitParameterNames(); 134 | while (paramNames.hasMoreElements()) { 135 | context_param = paramNames.nextElement(); 136 | } 137 | ... 138 | context.setAttribute("key1", "value1"); <·· Add attribute 139 | context.removeAttribute("key1"); <·· Remove attribute 140 | } 141 | 142 | public void init(ServletConfig servletConfig) throws ServletException{ 143 | this.testParam = servletConfig.getInitParameter("test"); 144 | } 145 | } 146 | ``` 147 | [[}]] 148 | 149 | 150 | -------------------------------------------------------------------------------- /jhipster.txt: -------------------------------------------------------------------------------- 1 | # JHipster [[{PM.low_code.jhipster,spring.101,architecture.frontend]] 2 | * HomePage: 3 | * Development platform to generate, develop and deploy 4 | Spring Boot + Angular / React / Vue Web applications and 5 | Spring microservices. 6 | * Created by Julien Dubois, currently (2021-06) Java Developer Advocacy 7 | manager at Microsoft 8 | 9 | ## JHipster install: 10 | 1. Install a LTS node version (through `nvm`) 11 | 2. Install latest JHipster. 12 | 13 | ## JHiipster project bootstrap: 14 | 15 | 1. Choose the desired Java version. Probably latest LTS JDK supported by 16 | the project and library transitive dependencies.
17 | This means, unfortunately, JDK 11 for many "old" libraries using XML. 18 | Otherwise opt for JDK 21 (as of 2024-03), since it's just better. 19 | 2. Choose a desired Spring version compatible with your JDK: 20 | [REF](https://github.com/spring-projects/spring-framework/wiki/Spring-Framework-Versions#jdk-version-range) 21 | ``` 22 | JDK Version Range 23 | - Spring Framework 6.2.x: JDK 17-25 (expected) 24 | - Spring Framework 6.1.x: JDK 17-23 25 | - Spring Framework 6.0.x: JDK 17-21 26 | - Spring Framework 5.3.x: JDK 8-21 (as of 5.3.26) 27 | ``` 28 | 3. Choose newest Spring-Boot version compatible with your Java version.
29 | ``` 30 | \ Spring \ Spring Boot 31 | JDK \ Version \ Version 32 | \----------------------------------\--------------- 33 | | 6.2.x 6.1.0 6.0.x 5.3.26+ | 3.0.X+ 2.7.X 34 | 25 | OK ERR ERR ERR | ?? ERR 35 | 23 | OK OK ERR ERR | ?? ERR 36 | 21 | OK OK OK OK | OK ERR 37 | 17 | OK OK OK OK | OK ERR 38 | 11 | ERR ERR ERR OK | ERR OK 39 | 8 | ERR ERR ERR OK | ERR OK 40 | 41 | C&P from 42 | https://github.com/spring-projects/spring-framework/wiki/Spring-Framework-Versions 43 | We fully test and support Spring on Long-Term Support (LTS) releases of the JDK: 44 | currently JDK 8, JDK 11, JDK 17, and JDK 21. Additionally, there is support for 45 | intermediate releases such as JDK 18/19/20 on a best-effort basis, ... 46 | We recommend JDK 17 and 21 for production use with Spring Framework 6.x as well as 5.3.x. 47 | -------------------------------------------------------- 48 | https://spring.io/projects/spring-boot#support 49 | Branch |Init. Relea.| End Suppor| End Comm. 50 | -------+------------+-----------+----------- 51 | 3.2.x | 2023-11-23 |2024-11-23 | 2026-02-23 52 | 3.1.x | 2023-05-18 |2024-05-18 | 2025-08-18 53 | 3.0.x | 2022-11-24 |2023-11-24 | 2025-02-24 54 | 2.7.x | 2022-05-19 |2023-11-24 | 2025-08-24 55 | ``` 56 | 3. Choose the jhipster version supporting the JDK version:
57 | TODO:(0) Map JDK to supported spring versions. 58 | ``` 59 | | Spring-boot | Min.Spring | JDK 60 | JHipster | version | Supported | Supported 61 | 8.0.x | 3.1.5 | ??? | ¿17,21? 62 | 7.9.0 | 2.7.2 | | ¿8,11? 63 | 7.8.0 | 2.6.6 | | ¿8,11? 64 | 65 | ``` 66 | 67 | [[PM.low_code.jhipster}]] 68 | 69 | -------------------------------------------------------------------------------- /monitorization_micrometer.txt: -------------------------------------------------------------------------------- 1 | [[{monitoring.micrometer,architecture.distributed,monitoring.distributed,PM.low_code,PM.Draft,PM.TODO]] 2 | # Micrometer Metrics 3 | 4 | * WARN: OpenTelemetry looks to be preferred, platform agnostic way to monitor apps as of 2022:
5 | Historically, OpenTelemetry was focused on tracing (see zipkin and Jaeger) more than metrics, 6 | but now metrics are "prominent".
7 | Current projects using Micrometry can integrate with OpenTelemetry using 8 | the Micrometer OpenTelemetryMeterRegistry.
9 | Spring stills recomends Opentracing as preferred framework. 10 | 11 | * simple facade to instrumentation clients for JVM-based apps without vendor lock-in. 12 | (Think SLF4J, but for application metrics! supporting AppOptics, 13 | Atlas, Datadog, Dynatrace, Elastic, Ganglia, Graphite, 14 | Influx,Instana, JMX (hierarchical mapping), KairosDB, New Relic, 15 | Prometheus, SignalFx, Stackdriver, StatsD, Wavefront,) 16 | * Recorded metrics are intended to be used to 17 | observe/alert/react to current/recent operational state. 18 | 19 | * out-of-the-box instrumentation provided by Micrometer 20 | * JVM Metrics on classloaders, memory, garbage collection, 21 | threads, etc. 22 | * Spring Boot 2.0.0.M5+: Micrometer used as instrumentation library powering 23 | the delivery of application metrics from Spring. 24 | 25 | 2 simple steps setup: [low_code] 26 | * Declare maven dependency. 27 | * Add config. to application.yml 28 | 29 | * (legacy support) 30 | drop-down support for Spring Boot 1.5.x. 31 | * Cache instrumentation for most popular caching frameworks. 32 | * OkHttpClient Instrumentation 33 | 34 | * Guides: [TODO] 35 | * 36 | * 37 | * 38 | [[}]] 39 | 40 | [[monitoring}]] 41 | 42 | 43 | -------------------------------------------------------------------------------- /monitorization_opentelemetry.txt: -------------------------------------------------------------------------------- 1 | [[{monitoring.opentelemetry]] 2 | # Open Telemetry 3 | 4 | * WARN: Open Telemetry replaces the now outdated OpenTracing. 5 | 6 | * 838 supported frameworks as of 2024. 7 | 8 | 9 | [[{PM.low_code]] 10 | ## Instrument Java Apps in 5 minutes 11 | 12 | * REF: , 13 | 14 | 15 | * Add traces, metrics, and logs to the console with zero code. 16 | 17 | 1. Presetup: 18 | 1. JDK 17+ for Spring Boot 3; Java 8+ for Spring Boot 2 19 | ``` 20 | $ cat build.gradle.kts 21 | | plugins { 22 | | id("java") 23 | | id("org.springframework.boot") version "3.0.6" 24 | | id("io.spring.dependency-management") version "1.1.0" 25 | | } 26 | | 27 | | sourceSets { 28 | | main { 29 | | java.setSrcDirs(setOf(".")) 30 | | } 31 | | } 32 | | 33 | | repositories { 34 | | mavenCentral() 35 | | } 36 | | 37 | | dependencies { 38 | | implementation("org.springframework.boot:spring-boot-starter-web") 39 | | implementation('io.opentelemetry.instrumentation:opentelemetry-instrumentation-annotations:2.6.0') // <·· 40 | | } 41 | ``` 42 | 2. Download the opentelemetry-javaagent.jar agent to automatically instrument 43 | the application at launch time from 44 | 45 | 46 | 2. Create and launch an HTTP Server: 47 | ``` 48 | | $ cat DiceApplication.java 49 | | package otel; 50 | | 51 | | import org.springframework.boot.Banner; 52 | | import org.springframework.boot.SpringApplication; 53 | | import org.springframework.boot.autoconfigure.SpringBootApplication; 54 | | 55 | | @SpringBootApplication 56 | | public class DiceApplication { 57 | | public static void main(String[] args) { 58 | | SpringApplication app = new SpringApplication(DiceApplication.class); 59 | | app.setBannerMode(Banner.Mode.OFF); 60 | | app.run(args); 61 | | } 62 | | } 63 | ``` 64 | 3. Create a controller 65 | ``` 66 | | $ cat RollController.java 67 | | package otel; 68 | | 69 | | import java.util.Optional; 70 | | import java.util.concurrent.ThreadLocalRandom; 71 | | import org.slf4j.Logger; 72 | | import org.slf4j.LoggerFactory; 73 | | import org.springframework.web.bind.annotation.GetMapping; 74 | | import org.springframework.web.bind.annotation.RequestParam; 75 | | import org.springframework.web.bind.annotation.RestController; 76 | | import io.opentelemetry.instrumentation.annotations.WithSpan; // <·· 77 | | import io.opentelemetry.instrumentation.annotations.SpanAttribute; // <·· 78 | | 79 | | @RestController 80 | | public class RollController { 81 | | private static final Logger logger = 82 | | LoggerFactory.getLogger(RollController.class); 83 | | 84 | | @GetMapping("/rolldice") 85 | | public String index( 86 | | @RequestParam("player") Optional player) { 87 | | int result = this.getRandomNumber(1, 6); 88 | | if (player.isPresent()) { 89 | | logger.info("{} ... {}", player.get(), result); 90 | | } else { 91 | | logger.info("...: {}", result); 92 | | } 93 | | return Integer.toString(result); 94 | | } 95 | | 96 | | @WithSpan // <·· Instrument method duration and 97 | | public int getRandomNumber( // reports thrown exceptions !!! 98 | | // for span.name ==. 99 | | // (default name) 100 | | @SpanAttribute("getRandom_min") int min, // <·· Instrument!! 101 | | @SpanAttribute("getRandom_max") int max // <·· Instrument!! 102 | | ) 103 | | { 104 | | return ThreadLocalRandom.current().nextInt(min, max + 1); 105 | | } 106 | | } 107 | | ¹: If return type is one of the following (listed below), [[{architecture.async}]] 108 | | the span will not be ended until the future completes. 109 | | 110 | | · java.util.concurrent.CompletableFuture 111 | | · java.util.concurrent.CompletionStage 112 | | · com.google.common.util.concurrent.ListenableFuture 113 | | · org.reactivestreams.Publisher 114 | | · reactor.core.publisher.Mono 115 | | · reactor.core.publisher.Flux 116 | | · io.reactivex.Completable 117 | | · io.reactivex.Maybe 118 | | · io.reactivex.Single 119 | | · io.reactivex.Observable 120 | | · io.reactivex.Flowable 121 | | · io.reactivex.parallel.ParallelFlowable 122 | ``` 123 | 4. Assemble (Build) the app: 124 | ``` 125 | $ gradle assemble # <·· create app in ./build/libs/java-simple.jar 126 | ``` 127 | 7. Stop and restart the app like: 128 | ``` 129 | | $ export JAVA_TOOL_OPTIONS="-javaagent:PATH/TO/opentelemetry-javaagent.jar" 130 | | $ export OTEL_TRACES_EXPORTER=logging 131 | | $ export OTEL_METRICS_EXPORTER=logging 132 | | $ export OTEL_LOGS_EXPORTER=logging 133 | | $ export OTEL_METRIC_EXPORT_INTERVAL=15000 # "small value" just for testing 134 | | $ java -jar ./build/libs/java-simple.jar 1>app.log 2&1 & 135 | | (output from the otel.javaagent will be automatically displayed to app.log) 136 | ``` 137 | 8. Test setup by checking opentelemetry traces are in place. 138 | ``` 139 | $ curl localhost:8080/rolldice 140 | $ cat app.log 141 | .... 142 | [otel.javaagent 2023-04-24 17:33:54:567 +0200] [http-nio-8080-exec-1] INFO 143 | io.opentelemetry.exporter.logging.LoggingSpanExporter - 'RollController.index' : 144 | 70c2f04ec863a956e9af975ba0d983ee 7fd145f5cda13625 INTERNAL [tracer: 145 | io.opentelemetry.spring-webmvc-6.0:1.25.0-alpha] AttributesMap{data= 146 | {thread.id=39, thread.name=http-nio-8080-exec-1}, capacity=128, 147 | totalAddedValues=2} 148 | [otel.javaagent 2023-04-24 17:33:54:568 +0200] [http-nio-8080-exec-1] INFO 149 | io.opentelemetry.exporter.logging.LoggingSpanExporter - 'GET /rolldice' : 150 | 70c2f04ec863a956e9af975ba0d983ee 647ad186ad53eccf SERVER [tracer: 151 | io.opentelemetry.tomcat-10.0:1.25.0-alpha] AttributesMap{ <··· 152 | data={ 153 | user_agent.original=curl/7.87.0, 154 | thread.name=http-nio-8080-exec-1, 155 | thread.id=39, 156 | net.host.name=localhost, 157 | net.transport=ip_tcp, 158 | net.sock.peer.port=53422, 159 | net.sock.host.addr=127.0.0.1, 160 | net.protocol.name=http, 161 | net.protocol.version=1.1, 162 | net.host.port=8080, 163 | net.sock.peer.addr=127.0.0.1, 164 | http.target=/rolldice, 165 | http.route=/rolldice, 166 | http.status_code=200, 167 | http.scheme=http, 168 | http.response_content_length=1, 169 | http.method=GET 170 | }, 171 | capacity=128, 172 | totalAddedValues=17} 173 | 174 | ... at stop time we will see something similar to ... 175 | [otel.javaagent 2023-04-24 17:34:25:347 +0200] [PeriodicMetricReader-1] INFO 176 | io.opentelemetry.exporter.logging.LoggingMetricExporter - Received a collection 177 | of 19 metrics for export. 178 | [otel.javaagent 2023-04-24 17:34:25:347 +0200] [PeriodicMetricReader-1] INFO 179 | io.opentelemetry.exporter.logging.LoggingMetricExporter - metric: 180 | ImmutableMetricData{ 181 | resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.19.0, 182 | attributes={ 183 | host.arch="aarch64", 184 | host.name="myserver01", 185 | os.description="Mac OS X 13.3.1", 186 | os.... 187 | process.pid=12345, 188 | process.... 189 | instrumentationScopeInfo=InstrumentationScopeInfo{ 190 | name=io.opentelemetry.runtime-metrics, 191 | version=1.25.0, 192 | schemaUrl=null, attributes={} 193 | } 194 | name=process.runtime.jvm.buffer.limit, 195 | data=ImmutableSumData{ points=[ ... ] } 196 | ... 197 | ``` 198 | [[PM.low_code}]] 199 | 200 | [[monitoring.opentelemetry}]] 201 | -------------------------------------------------------------------------------- /persistence.payload: -------------------------------------------------------------------------------- 1 | persistence_HikariCP.txt 2 | persistence_mybatis.txt 3 | persistence_stalactite.txt 4 | persistence_JPA_Hibernate.txt 5 | persistence_Jooq.txt 6 | persistence_QueryDSL.txt 7 | persistence_speedment.txt 8 | persistence_non_classified.txt 9 | 10 | -------------------------------------------------------------------------------- /persistence_HikariCP.txt: -------------------------------------------------------------------------------- 1 | [[{]] 2 | # HikariCP Connection Pool 3 | 4 | * JDBC "zero-overhead" production ready JDBC connection pool 5 | 6 | * 7 | * 8 | 9 | * very light ~ 130KB. 10 | * one order of magniture faster than c3p0 (abandoned) dbcp2 tomcat vibur in 11 | connections/sec. C&P from 12 | """... In order to make HikariCP as fast as it is, we went down to 13 | bytecode-level engineering, and beyond. We pulled out every trick we 14 | know to help the JIT help you. We studied the bytecode output of the 15 | compiler, and even the assembly output of the JIT to limit key 16 | routines to less than the JIT inline-threshold. We flattened 17 | inheritance hierarchies, shadowed member variables, eliminated casts.""" 18 | [[}]] 19 | -------------------------------------------------------------------------------- /persistence_JPA_Hibernate.txt: -------------------------------------------------------------------------------- 1 | [[{persistence.sql.jpa,PM.TODO]] 2 | # JPA/Hibernate Persistence Summary 3 | 4 | [[{PM.risk]] 5 | ## Must we discard JPA? 6 | 7 | cite by Thorben Janssen: 8 | 9 | ... Domain Driven Design (DDD) adoption has massively grown. 10 | ... One of the reasons is that the concept of "bounded-contexts" fits 11 | incredibly well with microservices-design .... You will soon recognize 12 | some of the DDD's design principles ovelaping with JPA: 13 | 1. Both use entities, identifiers, and value objects. 14 | 1. Do they really mean the same in both contexts? 15 | 1. What about DDD's concept of an aggregate? (1+ entities): 16 | This might sound like you shouldn't use JPA if you want to follow DDD 17 | design principles. But that's not the case. They actually fit very 18 | well together if you follow a few simple rules when implementing your 19 | JPA entity mappings. 20 | [[PM.risk}]] 21 | 22 | * Hibernate Gotchas: 23 | - . 24 | - . 25 | hibernate, joins, and max results: a match made in hell. 26 | * Common Hibernate Exceptions Every Developer Must Know: 27 | - 28 | 29 | ## JPA tutorial: 30 | 31 | * Migrating from JPA 2.x to 3.0: 32 | 33 | * 34 | 35 | [[{]] 36 | ## Quarkus + Panache 37 | * 38 | 39 | (Similar to Spring Data JPA), Panache for Quarkus handles most of 40 | the repetitive boilerplate code for you. Hibernate makes complex 41 | mappings possible, but it does NOT make simple and common mappings 42 | trivial. Hibernate ORM with Panache for Quarkus focuses on making 43 | your entities trivial and fun to write in Quarkus. 44 | Example: 45 | ``` 46 | | @Entity 47 | | public class Person extends PanacheEntity { 48 | | public String name; 49 | | public LocalDate birth; 50 | | public Status status; 51 | | 52 | | public static Person findByName(String name) { 53 | | return find("name", name).firstResult(); 54 | | } 55 | | public static List findAlive () { 56 | | return list("status", Status.Alive); 57 | | } 58 | | public static void deleteStefs () { 59 | | delete("name", "Stef"); 60 | | } 61 | | } 62 | ``` 63 | [[}]] 64 | 65 | 66 | 67 | [[persistence.sql.jpa}]] 68 | -------------------------------------------------------------------------------- /persistence_Jooq.txt: -------------------------------------------------------------------------------- 1 | [[{persistence.sql,vertx.*,architecture.async.reactive,PM.low_code,qa.data,PM.TODO]] 2 | # Jooq Persistence: SQL made simple 3 | * 4 | * WARN: Free for OOSS databases, requires licence for Oracle and maybe others. 5 | * Q ¿Deprecated in favor of QueryDSL?, maybe not: 6 | from Thorben e-mail: 7 | " ... I shared the announcement video for Lukas Eder's Expert Session a few days ago. Did you see it? 8 | Starting at minute 1:30, Lukas shares a few interesting jOOQ features, including the new multiset support. 9 | **It will be interesting to see if other frameworks adopt that feature. It seems to be an efficient way 10 | to fetch nested data structures in 1 query.**"
11 | VIDEO Link: 12 | * Related: No More MultipleBagFetchException thanks to Multiset Nested Collections SQL and jOOQ. 13 | 14 | 15 | ### Vertx+Jooq 16 | 17 | jOOQ-CodeGenerator to create vertxified DAOs and POJOs. 18 | Now with JDBC, async and reactive support! 19 | 20 | 21 | [[}]] 22 | -------------------------------------------------------------------------------- /persistence_QueryDSL.txt: -------------------------------------------------------------------------------- 1 | [[{persistence.sql.querydsl,PM.TODO]] 2 | ## QueryDSL Persistence: Compile safe SQL 3 | * 4 | 5 | * Compile safe SQL like domain specific language compatible with JPA. 6 | 7 | [[persistence.sql.querydsl}]] 8 | 9 | 10 | -------------------------------------------------------------------------------- /persistence_mybatis.txt: -------------------------------------------------------------------------------- 1 | [[{persistence.sql.mybatis]] 2 | # MyBatis Persistence 3 | Summary from 4 | * """ SQL is everywhere in data processing, ... Humble detected an increasing discomfort 5 | with “classic” ORMs such as Hibernate. These tools have solved two problems:
6 | * The repetitiveness of CRUD
7 | * Caching and thus speeding up disk access
8 | """ 9 | * Intelligent caching can be quite hard, even with Hibernate, ... 10 | * JPA’s caching mechanism is only possible because the ORM strategy has 11 | been deeply integrated with JPQL. Short of native querying, it is hard 12 | to bypass the essential mechanisms used for caching. 13 | * Memory is getting cheaper ... What if data is no longer accessed from disk 14 | (milliseconds), but from memory (nanoseconds) instead? 15 | * Will we still need complicated and complex second level caches, when the 16 | database already “cached” all relevant data in memory? 17 | * Not only are databases capable of keeping live online transactional 18 | data in memory all the time, they have already been doing some 19 | sophisticated caching for quite a while now. Just consider Oracle’s 20 | cursor cache, query result cache, and scalar subquery cache. 21 | * ...an increasing use of more SQL-centric alternatives such as 22 | MyBatis or jOOQ (both being “post-JPA” frameworks) indicates a 23 | certain need to get closer to SQL again. 24 | * ANSI-SQL standard is continually evolving ... things which are 25 | very hard to get through the JCP and into next generations of JPA. 26 | ....In all my conference talks about jOOQ, I have been asking the 27 | audience about their happiness with CriteriaQuery. The answer was 28 | always unanimously hostile. 29 | 30 | ### MyBatis "vs" Jooq: 31 | * 32 | * jOOQ is an internal domain-specific language modelling SQL through a Java fluent API. 33 | * MyBatis is an SQL templating and mapping engine where dynamic SQL 34 | can be created through an XML-DSL. 35 | * ... MyBatis’ current success is mostly based on it having provided 36 | a viable alternative to JPA in a time when JPA was still a 37 | controversial standard, and when JPA had to prove that it is better 38 | than JDO, which solves very similar problems. 39 | [[persistence.sql.mybatis}]] 40 | -------------------------------------------------------------------------------- /persistence_non_classified.txt: -------------------------------------------------------------------------------- 1 | # SQL persistence (Non Classified) 2 | 3 | ## Flyway: SQL schema versioning [[{persistence.sql,qa.data,PM.low_code,spring.*,persistence.jpa,PM.TODO]] 4 | * tool providing version control for database (SQL) schemas 5 | and automated schema (tables, columns, sequences), data, views, procedures 6 | and packages evolution. 7 | * Single source of truth for DDBB versioning. 8 | * highly reliable 9 | * Supports for many different SQL databases, including [cloud] 10 | cloud ones (Amazon RDS, Azure Database, Google Cloud SQL). 11 | 12 | ### Flyway HowTO: 13 | * PRESETUP: The target database to manage and user with update privileges be created first, 14 | "outside" of flyway. 15 | 16 | * Database changee are called Migrations . They can be: 17 | · Versioned migrations: identified by a version number, applied in order exactly once. 18 | An optional revert migration version can be provided to roolback 19 | changes in case of error. 20 | · Repeatable migrations: 21 | 22 | * Flyway used an internal flyway_schema_history ddbb to keep track of migrations applied . 23 | 24 | * SQL and JAVA migrations in src/main/resources/db/migration/ are automatically [spring] 25 | applied in Spring Boot when 'org.flywaydb:flyway-core' compile dependency is 26 | detected. 27 | ``` 28 | | src/main/resources/db/migration/ 29 | | └ V1__Initial_schema.sql ← Flyway expected file name 30 | | "Verssion"__"Prefix".sql 31 | | CREATE TABLE table01 ( 32 | | id BIGSERIAL PRIMARY KEY NOT NULL, 33 | | column1 BIGINT NOT NULL, 34 | | column2 FLOAT8 NOT NULL, 35 | | column3 INTEGER NOT NULL, 36 | | column4 VARCHAR(255) UNIQUE NOT NULL, 37 | | ); 38 | 39 | | └ V2__Add_column5.sql 40 | | ALTER TABLE table01 41 | | ADD COLUMN column5 VARCHAR(255); 42 | ``` 43 | * Alternatives to Flyway include LiquidBase, Obevo, ... [TODO] 44 | 45 | ## Obevo: DDBB change manager 46 | * 47 | - Obevo: ddbb deployment tool handling enterprise scale schemas and complexity. 48 | - By Goldman Sachs. 49 | - Obevo is a database deployment tool that helps teams manage database 50 | changes in their Software Development Life Cycle (SDLC) process. In 51 | addition to handling production deployments, Obevo aids the 52 | development phase by defining a clean structure to maintain DB object 53 | code, and helps the testing phase with features such as in-memory 54 | database conversion. Notably, Obevo was designed for systems of 55 | enterprise scale and complexity and can manage hundreds of DB objects 56 | in a schema, while still handling new schemas in a simple manner. 57 | “We feel our ability to onboard a large and long-lived system to a 58 | clean SDLC process is a key differentiator in the open source 59 | space,” said Shant, a vice president in the Technology Division. 60 | “By publishing this to the open source community, we hope to aid 61 | others in their own DB deployment estates while growing a strong 62 | community around the tool.” 63 | 64 | 65 | > """ Deploying tables for a new application? 66 | > Or looking to improve the DB Deployment of a years-old system with 67 | > hundreds (or thousands) of tables, views, stored procedures, and 68 | > other objects?
69 | > 70 | > Obevo has your use case covered.
71 | > 72 | > Supported platforms: DB2, H2, HSQLDB, Microsoft SQL Server, MongoDB, 73 | > Oracle, PostgreSQL, Redshift (from Amazon), Sybase ASE, Sybase IQ 74 | > """ 75 | 76 | * As a reference Spring Boot includes auto-configuration support for both Flyway and Liquibase. 77 | [[}]] 78 | 79 | ## Other ORM/DSL SQL frameworks 80 | 81 | ### Excluded ORM Framework Comparative [[{]] 82 | 83 | * 84 | * .... The following frameworks have been excluded from the test due to lack 85 | of support of one or more of the required features or because of 86 | other problems. That isn't to say anything negative about 87 | them—they're certainly promising—but they didn't suit my needs 88 | for one or more reasons. 89 | * Exposed is the ORM for Kotlin implemented by JetBrains. I've tried 90 | to use it in Jav,a but it was really hard, almost impossible. For 91 | Scala developers, this may be a good option. 92 | * Reladomo is an enterprise ORM, but it does not support UDTs or other PostgreSQL advanced features. 93 | * Speedment does not support transactions in its current stable version. 94 | [[}]] 95 | 96 | ### frameworks no longer developed or maintained: [[{]] 97 | * ActiveJPA seems like it never reached stable release, with the last commits from 2014. 98 | * Apache Torque had its last commit in 2013. 99 | * Carbonado is an inactive project. 100 | * Fjorm's last commit is two years old. 101 | * Hydrate's last version is from 2006. 102 | * IBM PureQuery's last version is from 2009. 103 | * JDO Instruments doesn't even have a working homepage. 104 | * ORMLite's last version is from 2013. 105 | * QuickDB ORM is available only in Google Archive, with its last commit being from 2010. 106 | * Speedo's last commit is from 2005. 107 | * TJDO Project's last commit is from 2008. 108 | [[}]] 109 | 110 | ### Reladomo ORM (by Goldman Sachs) [[{persistence.sql.ORM,PM.low_code,PM.TODO]] 111 | * 112 | Enterprise grade ORM framework featuring: 113 | - Strongly typed compile-time checked query language. 114 | - Bi-temporal chaining. 115 | - Transparent multi-schema support. 116 | - Full support for unit-testable code. 117 | [[}]] 118 | 119 | 120 | # JAVA Persistence "Others" 121 | 122 | [[{scalability.persistence.compression,persistence.fs,PM.TODO]] 123 | ## Snappy Fast de/compressor 124 | * 125 | Java port of the snappy 126 | [[scalability.persistence.compression}]] 127 | 128 | ## JCache [[{scalability.cache,PM.TODO]] 129 | - Map-Like API optimized for caching. 130 | - 1.0 drawbacks: 131 | - No async operations. 132 | - Implemented by Hazelcast and others 133 | [[}]] 134 | 135 | [[{architecture.async.reactive,persistence.sql,PM.TODO]] 136 | 137 | ## R2DBC API 138 | 139 | 140 | * Experimental Reactive Relational Database Connectivity Driver, Announced at SpringOne. 141 | 142 | * Project initiated by Pivotal to provide an async/reactive alternative to JDBC. 143 | 144 | * WARN : Since Spring Data R2dbc evolved very quickly, thre are plenty 145 | of breaking changes introduced since Spring 5.2 and Spring Data R2dbc 1.2. 146 | Breaking changes (Compared to Spring Data R2dbc 1.1): 147 | * Spring Data R2dbc 1.1 DatabaseClient was split into two parts. 148 | a simple new DatabaseClient is part of Spring framework, as an 149 | alternative of Jdbc. 150 | * Another part of the old DatabaseClient is reorganized into a new 151 | class R2dbcEntityTemplate which acts as the role of JdbcTemplate. 152 | Ex: 153 | ``` 154 | connectionFactory.create() 155 | .flatMapMany ( conn -> 156 | conn.createStatement ( "SELECT value FROM test" ) 157 | .execute() 158 | .flatMap (result -> 159 | result.map(( row, metadata -→ row.get("value")))) 160 | ``` 161 | 162 | ``` 163 | spring: 164 | r2dbc: ← ... (src/main/resources/)application.yml example: 165 | username: XXX 166 | password: XXX 167 | url: r2dbc:postgresql://...:5432/ddbb_app01 168 | pool: 169 | max-create-connection-time: 5s 170 | initial-size: 5 ←················ probably much lower numbers than 171 | max-size: 10 those used for JDBC. 172 | ``` 173 | * 174 | [[}]] 175 | [[architecture.async.reactive}]] 176 | 177 | [[{scalability.distributed,scalability.cache,persistence.jpa,PM.low_code,PM.TODO]] 178 | ## SwarmCache 179 | 180 | * SwarmCache: simple but effective distributed cache using IP 181 | multicast to efficiently communicate with any number of hosts on a 182 | LAN. 183 | * Specifically designed for use by clustered, database-driven web 184 | applications typically having many more read operations than 185 | write operations, which allows SwarmCache to deliver the greatest 186 | performance gains. 187 | * Internally it uses JavaGroups manage the membership and communications of 188 | its distributed cache. 189 | * Wrappers have been written that allow SwarmCache to be used with the 190 | Hibernate and JPOX persistence engines. 191 | [[}]] 192 | 193 | ## jFleet [[{scalability.persistence,persistence.jpa,persistence.sql,PM.TODO]] 194 | 195 | * 196 | * JFleet is a Java library which persist in database large collections 197 | of Java POJOs as fast as possible, using the best available technique 198 | in each database provider, achieving it with alternate persistence 199 | methods from each JDBC driver implementation. 200 | * Its goal is to store a large amount of information in a single table 201 | using available batch persistence techniques. 202 | * despite being able to use JPA annotations to map Java objects to 203 | tables and columns, JFleet is not an ORM. 204 | [[}]] 205 | 206 | 207 | -------------------------------------------------------------------------------- /persistence_speedment.txt: -------------------------------------------------------------------------------- 1 | [[{persistence.sql.speedment,PM.low_code,qa.data,java_lang.functional]] 2 | ## Speedment Persistence: SQL as Streams 3 | 4 | - Stream ORM toolkit and runtime. 5 | - The toolkit analyzes the metadata of an existing SQL database and 6 | automatically creates a Java representation of the data model. 7 | - The powerful ORM enables you to create scalable and efficient Java 8 | applications using standard Java streams with no need to type SQL or 9 | use any new API. 10 | ``` 11 | SQL JAVA 8 Stream Equivalent 12 | -------------------------------------------------------------------- 13 | FROM stream() 14 | -------------------------------------------------------------------- 15 | COUNT count() 16 | -------------------------------------------------------------------- 17 | LIMIT limit() 18 | -------------------------------------------------------------------- 19 | DISTINCT distinct() 20 | -------------------------------------------------------------------- 21 | SELECT map() 22 | -------------------------------------------------------------------- 23 | WHERE filter() (before collecting) 24 | -------------------------------------------------------------------- 25 | HAVING filter() (after collecting) 26 | -------------------------------------------------------------------- 27 | JOIN flatMap() 28 | -------------------------------------------------------------------- 29 | UNION concat(s0, s1).distinct() 30 | -------------------------------------------------------------------- 31 | ORDER BY sorted() 32 | -------------------------------------------------------------------- 33 | OFFSET skip() 34 | -------------------------------------------------------------------- 35 | GROUP BY collect(groupingBy()) 36 | -------------------------------------------------------------------- 37 | SEARCH FILMS WITH LENGTH > 120 MINUTES: 38 | SELECT ← final 39 | `film_id`,`title`,`description`, OptionallongFilm = 40 | `release_year`, `language_id`, films.stream() 41 | `original_language_id`, .filter( 42 | `rental_duration`,`rental_rate`, Film.LENGTH.greaterThan(120) 43 | `length`,`replacement_cost`, ) 44 | `rating`,`special_features`, .findAny(); 45 | `last_update` 46 | FROM Searches optimized in background! 47 | `sakila`.`film 48 | WHERE 49 | (`length` > 120) 50 | ``` 51 | [[persistence.sql.speedment}]] 52 | -------------------------------------------------------------------------------- /security.txt: -------------------------------------------------------------------------------- 1 | [[{security.101,PM.low_code,spring,security.jvm]] 2 | # Security 101 3 | 4 | * REF: 5 | ## Make code immutable 6 | * Tag variables as 'final' by default. 7 | (a final variable is not variable but constant) 8 | * Try to initialize all classes in constructors. and centralize 9 | all input checks in the constructor. Raise a runtime exception 10 | if the constructor is not happy with input data. 11 | (vs using getters and setters). 12 | Doing so warrants that a class is properly instantiated and 13 | safe upon constructor exit. 14 | This also warrants fail-fast applications. If something is 15 | undefined at startup time constructors will not be able to 16 | initialize and the application will fail to start: This is 17 | life-saving for DevOps and normal deployments. 18 | 19 | ## Parameterize SQL 20 | 21 | ``` 22 | | query = "SELECT ... WHERE lastname = " + parameter ; // ← Incorrect: 23 | | Statement stm01 = con.createStatement(); 24 | | 25 | | query = "SELECT ... WHERE lastname = ?"; // ← Correct 26 | | PreparedStatement stm01 = con.prepareStatement(query); 27 | | statementB .setString(1, parameter); 28 | | 29 | | ... stm01.executeQuery(query); 30 | ``` 31 | 32 | ## Use OpenID Connect with 2FA 33 | 34 | * OAuth 2.0: Identity delegation. N applications delegate login 35 | authentication to a third service that emmits signed tokens 36 | with claims about authorization for a given authenticated 37 | user. 38 | * OpenID: OAuth 2.0 extension providing user information 39 | claims to the signed OAuth 2.0 token. 40 | It adds an ID token in addition to an access token plus 41 | a /userinfo endpoint to retrieve additional information 42 | plus endpoint discovery and dynamic client registration. 43 | * Example low-code OpenID in Spring: 44 | STEP 1: Addd next dependencies: 45 | ``` 46 | | org.springframework.boot:spring-boot-starter-oauth2-client 47 | | org.springframework.boot:spring-boot-starter-security 48 | ``` 49 | STEP 2: Spring Configuration 50 | ``` 51 | | spring: 52 | | ... 53 | | security: 54 | | oauth2: 55 | | client: 56 | | registration: 57 | | github: 58 | | client─id: ... 59 | | client─secret: .... 60 | | okta: 61 | | client─id: ... 62 | | client─secret: ... 63 | | client─name: ... 64 | | google: 65 | | ... 66 | | provider: 67 | | okta: 68 | | issuer─uri: https://.....okta.com/oauth2/default 69 | ``` 70 | 71 | ## Scan dependencies for known vulnerabilities 72 | 73 | * Ej.: Use a service like !!! 74 | 75 | ## Handle secrets with care 76 | 77 | * Use secrets on-demand and remove from memory as soon as possible. 78 | Ideally manage secrets through HSM. Secrets (signature keys, password, 79 | ...) must never leave the HSM. 80 | 81 | ## Sanitize all input 82 | 83 | * Example 1, sanitizing HTML
84 | Add dependency: `org.owasp.encoder:encoder:vX.Y.Z` 85 | ``` 86 | | final String 87 | | untrustedInput = "", 88 | | trustedInput = StEncode.forHtml(untrustedInput)); 89 | ``` 90 | 91 | ## disable XXE (eXternal Entity) in XML parsers 92 | 93 | ``` 94 | | 95 | | ] > <·· attack vector 97 | | 98 | | &xxe; 99 | | ... 100 | | 101 | ``` 102 | * e.g. In xerces 1/2 disable external entities and doctypes like: 103 | ``` 104 | | ... 105 | | factory = SAXParserFactory.newInstance(); 106 | | saxParser = factory.newSAXParser(); 107 | | factory.setFeature( 108 | | "https://xml.org/sax/features/external-general-entities", false ); 109 | | saxParser.getXMLReader().setFeature( 110 | | "https://xml.org/sax/features/external-general-entities", false ); 111 | | factory.setFeature( 112 | | "https://apache.org/xml/features/disallow-doctype-decl" , true ); 113 | | ... 114 | ``` 115 | 116 | ## Avoid Java serialization (as much as possible) 117 | 118 | * If really needed check some professional serialization tool (vs 119 | JDK 1.0+ serialization), for example: eclipse-serializer 120 | 121 | * Java serialization is also called "the gift that keeps on giving". 122 | Oracle is planning to eventually remove Java serialization as part 123 | of Project Amber. 124 | 125 | * If you really need to implement serializable on your domain entities, 126 | implement its own readObject(). 127 | ``` 128 | | private final void 129 | | readObject(ObjectInputStream in) throws java.io.IOException { 130 | | // check 1 131 | | // check 2 132 | | ... 133 | | throw new java.io.IOException("Deserialized not allowed"); 134 | | } 135 | ``` 136 | * If you need to Deserialize an inputstream yourself, you should use an 137 | ObjectsInputStream with restrictions.
138 | e.j Apache Commons IO ValidatingObjectInputStream, that checks whether 139 | the object being deserialized is allowed or not. 140 | ``` 141 | | final FileInputStream fileInput = new FileInputStream(fileName); 142 | | ValidatingObjectInputStream in = new ValidatingObjectInputStream(fileInput); 143 | | in.accept(Foo.class); 144 | | Foo foo_ = (Foo) in.readObject(); 145 | ``` 146 | * Object deserialization can also apply to JSON, XML, ... 147 | [REF@snyk.io](https://snyk.io/blog/jackson-deserialization-vulnerability/) 148 | 149 | ## Use strong encryption and hashing algorithms.
[[{security.cryptography}]] 150 | TIP: Prefer Google Tink (vs Low Level Java crypto libraries) 151 | 152 | ## Enable the Java Security Manager [[{security.jvm]] 153 | * By default, JVM imposes no restrictions to running apps. 154 | (file system, network, ..) 155 | * Ex. By default the Attach API is active allowing to easely 156 | change bytecode of running apps. (from inside the machine). 157 | * Activate it like: 158 | ``` 159 | | $ java ... -Djava.security.manager <·· Use default policy 160 | | 161 | | $ java ... -Djava.security.manager \ <·· Use custom policy 162 | | -Djava.security.policy==.../custom.policy <·· == : replace default 163 | | = : expand default 164 | ``` 165 | * More info at [oracle.com](https://docs.oracle.com/en/java/javase/11/security/permissions-jdk1.html) 166 | ``` 167 | | java.awt.AWTPermission java.sql.SQLPermission 168 | | java.io.FilePermission java.util.logging.LoggingPermission 169 | | java.io.SerializablePermission java.util.PropertyPermission 170 | | java.lang.RuntimePermission javax.management.MBeanPermission 171 | | java.lang.management.ManagementPermission javax.management.MBeanServerPermission 172 | | java.lang.reflect.ReflectPermission javax.management.MBeanTrustPermission 173 | | java.net.NetPermission javax.management.remote.SubjectDelegationPermission 174 | | java.net.URLPermission javax.net.ssl.SSLPermission 175 | | java.net.SocketPermission javax.security.auth.AuthPermission 176 | | java.nio.file.LinkPermission javax.security.auth.PrivateCredentialPermission 177 | | java.security.AllPermission javax.security.auth.kerberos.DelegationPermission 178 | | java.security.SecurityPermission javax.security.auth.kerberos.ServicePermission 179 | | java.security.UnresolvedPermission javax.sound.sampled.AudioPermission [[}]] 180 | ``` 181 | [[security.101}]] 182 | 183 | ## SPARTA (anti-malware) [[{security.antimalware,PM.TODO]] 184 | * 185 | * Research project funded by the DARPA Automated Program Analysis for Cybersecurity (APAC) program. 186 | * Aimed at preventing malware from appearing in an app store. 187 | * provides an information-flow type-checker customized to Android 188 | but can also be applied to other domains. 189 | * The paper "Collaborative verification of information flow for a 190 | high-assurance app store" appeared in CCS 2014. 191 | * verification approach is type-checking: 192 | 1. the developer states a security property, 193 | 2. annotates the source code with type qualifiers that express that security property. 194 | 3. Runs a pluggable type-checker to verify that the type qualifiers are right 195 | (and thus that the program satisfies the security property). 196 | * It was developed with two different types of users in mind. 197 | 1. Application vendors/authors of an app that submit the app to an app store for a security review. 198 | 2. App analysts, or verification engineers, who work on behalf of the app store to verify that 199 | apps meant specific security properties before they are accepted. 200 | [[security.antimalware}]] 201 | 202 | ## OSV.io [[{scalability.unikernel,security.OSV,devops,PM.backlog]] 203 | - 204 | versatile modular unikernel designed to run unmodified Linux 205 | applications securely on micro-VMs in the cloud. Built from the ground up for 206 | effortless deployment and management of micro-services and serverless apps, 207 | with superior performance. (Includes CRaSH shell) 208 | [[}]] 209 | 210 | [[{security.secret_mng.conclave,PM.TODO]] 211 | ## Conclave SDK 1.3 ("Intel SGX Trusted Computing") as Open Source 212 | 213 | * 214 | * 215 | 216 | * Milestone in confidential computing ... workloads in the cloud 217 | with the KDS ... under Apache 2.0 218 | 219 | * Conclave is R3’s privacy preserving platform enabling you to write 220 | secure confidential applications with ease ... based on Intel® SGX 221 | enclaves ... using Java, Kotlin, and JavaScript hiding the 222 | low-level complexities of SGX 223 | * It also powers Conclave Cloud privacy-preserving SaaS platform. 224 | * Confidential computing protects data at rest, in transit and 225 | **during processing**. 226 | * SGX hardware provides a secure region of the CPU .. 227 | physically isolated .. cannot be tampered with, not even 228 | by the system or cloud administrator. 229 | [[security.secret_mng.conclave}]] 230 | -------------------------------------------------------------------------------- /tomcat.txt: -------------------------------------------------------------------------------- 1 | # Tomcat 2 | 3 | ## External Links: 4 | * 5 | * 6 | * 7 | * 8 | * 9 | * 10 | 11 | ## Servlets 4.0 Spec: [[standards.jee,io.http.servlets]] 12 | 13 | "... In particular, you will need to understand the web application 14 | directory structure and deployment file (Chapter 10), methods of 15 | mapping request URIs to servlets (Chapter 12), container managed 16 | security (Chapter 13), and the syntax of the web.xml 17 | Web Application Deployment Descriptor (Chapter 14). " 18 | 19 | ## Tomcat Logical Componentes [[{standards.jee,io.http.servlets]] 20 | embedded servers usually comprise two logical components: 21 | 22 | ``` 23 | | ┌───────────────────────────────────┬─────────────────────────────────┐ 24 | | │ LOGICAL COMPONENT │ TOMCAT equivalent │ 25 | | ├───────────────────────────────────┼─────────────────────────────────┤ 26 | | │ a web server component │ Coyote │ 27 | | │ listening for HTTP requests and │ │ 28 | | │ returning HTTP responses │ │ 29 | | ├───────────────────────────────────┼─────────────────────────────────┤ 30 | | │ an execution context to make │ Catalina, based on Servlet API │ 31 | | │ a Java web application interact │ (usually called the Servlet │ 32 | | │ with the web server. │ Container) │ 33 | | └───────────────────────────────────┴─────────────────────────────────┘ 34 | ``` 35 | [[}]] 36 | 37 | 38 | 39 | ## General Schema [[{]] 40 | 41 | ``` 42 | ┌> 1 │INSTALLATION│ 43 | · ${CATALINA_HOME}: root path, static jar sources, binary files, ... 44 | · 45 | └> N │TOMCAT RUNTIME│ 1 <···················> N │Context │ 1 <·> 1 path URI 46 | ${CATALINA_BASE} (web app) └┬┘ 47 | root path of runtime └──┬──┘ sys.admins will 48 | └────────┬─────────┘ ┌───┘ create the map 49 | ┌────────────┘ (ServletContext root) 50 | Runtime filesystem Layout ├─ ?.html, ?.jsp, ?.js, ... 51 | ├─bin/ ├─ ?.html, ?.jsp, ?.js, ... 52 | │ - setenv.sh │ Visible to external clients 53 | │ - start/stop scripts │ 54 | │ - tomcat─juli.jar <·· replace ├─ /WEB─INF/web.xml 55 | │ to use custom logging (discouraged) │ WebApp Deployment descriptor 56 | │ │ "everything about your app 57 | │ ... │ that the server needs to know" 58 | ├─conf/ │ (except the context path) 59 | │ - Config files and related DTDs. │ Chapter 13@Servlet API Spec ver 2.3 60 | │ Read at startup. Changes require │ **WARN: tags are sensible to order** 61 | │ server restart │ ·>·> 62 | │ server.xml <·· main conf.file *1 ├─ /WEB-INF/lib/ Java class files 63 | │ web.xml ├─ /WEB-INF/classes/ with restricted access 64 | │ │ 65 | ├─logs/ └─ /META─INF/context.xml 66 | ├─lib/ Tomcat non-standard Context Descriptor for 67 | │ - extra resources to be added to Classpa. custom config (data sources, session manager,..) 68 | │ Recomended for apps that depends on element is considered child 69 | │ external libraries of the corresponding to the Host 70 | │ By default Servlet 4.0 and JSP 2.3 APIs to which the web application is being 71 | │ implementations are included deployed. 72 | │ └────────┬──────────┘ 73 | └─webapps - automatically loaded webapps can also be placed in: 74 | ├─ webapp01.war <·· "compressed war" ${CATALINA_BASE}/conf/[enginename] 75 | ├─ webapp02/... <·· "exploded" deployment /[hostname]/[webappname].xml 76 | ├─ ... 77 | └────┬─────┘ 78 | · 79 | Depployment can be: 80 | - Static : webapp setup before Tomcat start 81 | - Dynamic: - auto-deployment (Host autoDeploy = "true") triggers at: 82 | - "drop" the exploded or war file into webapps/ 83 | - "touch" /WEB-INF/web.xml (or any other resource) 84 | - Re-deployment of dependent webapps 85 | - Tomcat Manager (GUI or REST API) 86 | - Ant, Maven, custom scripts 87 | - Tomcat Client Deployer (TCD): 88 | cli tool that also compiles/validates/packages the app-,... 89 | 90 | 91 | *1 92 | ``` 93 | [[}]] 94 | 95 | # Web App Layout [[{]] 96 | 97 | * REF https://jakarta.ee/specifications/servlet/6.0/jakarta-servlet-spec-6.0#web-applications 98 | 99 | ``` 100 | webapp/ ("context") 101 | ├ index.html 102 | ├ report01.jsp 103 | ├ images/logo.png 104 | ├ ./META─INF/ <·· Added during war packaging 105 | ├ ./META─INF/MANIFEST.MF <·· Declare external dependencies that need 106 | │ to be present in container (jdbc libraries ...) 107 | └ WEB─INF/web.xml deployment descriptor. 108 | │ configuration and deployment information: 109 | │ - ServletContext Init Parameters 110 | │ - Session Configuration 111 | │ - Servlet/JSP Definitions & Mappings 112 | │ - MIME Type Mappings 113 | │ - Welcome File list 114 | │ 115 | │ index.html 116 | │ default.jsp 117 | │ 118 | │ - Error Pages 119 | │ - Security 120 | ├ classes/ <·· servlets + utility classes. 121 | ├ lib/*.jar <·· ./classes takes preference over *jar's 122 | ... 123 | 124 | 125 | ``` 126 | 127 | ## Web App bootstrap: 128 | 129 | before the web application begins processing client requests 130 | the following steps must be performed: 131 | 1. Instantiate an instance of each event listener identified by a 132 | `` element in the deployment descriptor. 133 | 2. For instantiated listener instances that implement 134 | ServletContextListener, call the contextInitialized() method. 135 | 3. Instantiate an instance of each filter identified by a `` 136 | element in the deployment descriptor and call each filter 137 | instance’s init() method. 138 | 4. Instantiate an instance of each servlet identified by a `` 139 | element that includes a `` element in the order [[{qa.fail_fast]] 140 | defined by the load-on-startup element values, and call each servlet 141 | instance’s init() method. [[}]] 142 | 143 | 144 | ## Listeners 145 | * Servlet context listeners are used to manage resources or state 146 | held at a JVM level for the application. (`jakarta.servlet.`) 147 | * ServletContextListener : init OK, ready to shut-down 148 | * ServletContextAttributeListener: servlet's attributes modified. 149 | 150 | * HTTP session listeners are used to manage state or resources 151 | associated with a series of requests made into a web application from 152 | the same client or user. (`jakarta.servlet.http.`) 153 | * HttpSessionListener HttpSession has been created/invalidated/timed out. 154 | * HttpSessionAttributeListener HttpSession's attributes modified 155 | * HttpSessionIdListener HttpSession'id changed 156 | * HttpSessionActivationListener HttpSession has been activated/passivated. 157 | * HttpSessionBindingListener Object has been bound to or unbound from HttpSession 158 | * 159 | 160 | * Servlet request listeners are used to manage state across the 161 | lifecycle of servlet requests. (`jakarta.servlet.`) 162 | * ServletRequestListener Servlet-request has started being processed 163 | * ServletRequestAttributeListener Servlet-request's attributes modified 164 | * AsyncListener timeout, connection termination/completion 165 | * Async listeners are used to manage async events such as time outs 166 | and completion of async processing. 167 | 168 | [[}]] 169 | 170 | ## TODO 171 | * Tomcat Manager 172 | 173 | 174 | deployed by default on context path /manager, 175 | - deploy/undeploy apps without tomcat restarting 176 | * Tomcat Specifications 177 | * Example war for testing 178 | * Advanced IO on Tomcat 179 | * APache Portable Runtime 180 | * Deployer Howto 181 | * Realm howto: 182 | * security manager howto: 183 | * JNDI resouces , 184 | 185 | * class loader howto: 186 | * jasper-howto.html 187 | * SSL how-to 188 | * CGI how-to 189 | * Proxy Howto 190 | * Mbeans descriptors howto 191 | * Default servlet 192 | * Cluster-HowTo 193 | * Balancer HowTo 194 | * Connectors 195 | * Monitoring 196 | * Logging 197 | * Virtual Hosting 198 | * Extras 199 | * Maven jars: 200 | * Security howto 201 | * JDBC Pool 202 | * Web Sockets 203 | * Rewrite 204 | * Config 205 | 206 | ## PSI-probe: Advanced manager&monitor for Tomcat 207 | 208 | 209 | ## Apache Tomcat Migration Tool for jakarta EE 210 | 211 | * 212 | 213 | * cli tool is to take a web application written for Java EE 8 running on 214 | Tomcat 9 and convert it automatically to run on Tomcat 10 implementing 215 | Jakarta EE 9. 216 | -------------------------------------------------------------------------------- /vertx.txt: -------------------------------------------------------------------------------- 1 | ## Apropos: 2 | Content is versioned in git. commits, issues and pull-requests welcome! 3 | 4 | 5 | [[{vertx]] 6 | ## VertX: Who-is-Who 7 | (Forcibly incomplete but still quite pertinent list of core developers and companies) 8 | * Tim Fox : Initiated VertX in 2012 9 | * Julien Viet : Project lead (as of 2020), RedHat, Marseille
10 | ,
11 | He is also core developer of Crash 12 | * Julien Ponge:
13 | Author of VertX in Action 14 | * Many others : 15 | 16 | [[{vertx.101,architecture.async.reactive,doc_has.diagram]] 17 | 18 | # Vert.X Summary 19 | REF: 20 | - Vert.X guide for java devs 21 | - VertX maven starter 22 | - Examples for amqp-bridge, 23 | grpc, core, docker, gradle/maven , ignite, jca, 24 | jdbc. kafka, kotlin, mail, metrics, mqtt, openshift3, 25 | reactjs-server-side-rendering, redis, resteasy, rx, 26 | service-discovery, service-proxy, shell, spring, 27 | sync, unit, web/web-client ... 28 | - Web server examples:
29 | 30 | angular, auth, authjdbc, authorisation, blockinghandler, chat, 31 | cookie, cors, custom_authorisation, form, helloworld, http2, , jwt, 32 | mongo, react, realtime, rest, sessions, staticsite, 33 | templating, upload, vertxbus. 34 | - Web/ JDBC server examples:
35 | 36 | 37 | ## Verticle 38 | * REF: 39 | ``` 40 | | Verticle = reusable unit of deployment 41 | | ┌───────────┴────────┘ 42 | | - Can be passed some configuration like 43 | | credentials, network address,... 44 | | - can be deployed several times. 45 | | - A verticle can deploy other verticles. 46 | | 47 | | |verticle| 1 <··> 1|event─loop|1 <········> 1|Thread| 48 | | · · · · · · 49 | | └────┬───┘ └────┬─────┘ └──┬───┘ 50 | | ^ "input" event like Must not handle I/O thread-blocking 51 | | │ network buffers, or CPU intensive operations 52 | | timing events, 'executeBlocking' can be used 53 | | │ verticles messages, ... to offload the blocking I/O operations 54 | | <> Base Class 56 | | ════════════════ 57 | |- .start() <·· life-cycle sync/async method to be overrided 58 | |- .stop () <·· life-cycle sync/async method to be overrided 59 | |- .vertx <·· - Points to the Vert.x environment where the verticle is deployed 60 | | · - provides methods to create HTTP/TCP/UDP/... servers/clients. Ex: 61 | | · · io.vertx.core.http.HttpServer server = thisO .vertx .createHttpServer(); 62 | | · - provides access to the event bus. 63 | | · Ex: 64 | | · SENDING VERTICLE │ RECEIVING VERTICLE 65 | | · ┌──────────────────────────────────────────┼──────────────────────────────────────── 66 | | · │ ... │ ... 67 | | · │ vertx .eventBus() │ public void onMessage( 68 | | · │ .request(wikiDbQueue, │ Message message) 69 | | · │ jsonObject, options , │ { 70 | | · │ └────┬───┘ └──┬──┘ │ String action = message. 71 | | · │ · headers │ headers().get("action"); 72 | | · │ · + payload codecs │ 73 | | · │ · + tiemouts │ switch (action) { 74 | | · │ · └──┬──┘ │ case "action1": 75 | | · │// Ussually jsonObject contains the data │ ... 76 | | · │// and an "action" header the action to │ message.reply( 77 | | · │// be executed by the receiving verticle │ new JsonObject() 78 | | · │ reply -> { │ .put("key1", value1)); 79 | | · │ if (reply.succeeded()) { │ break; 80 | | · │ ... │ case ...: 81 | | · │ } else { │ ... 82 | | · │ ... │ default: 83 | | · │ } │ message.fail( 84 | | · │ }); │ ErrorCodes.BAD_ACTION.ordinal(), 85 | | · │ │ "Bad action: " + action); 86 | | · │ │ } 87 | | · │ │ } 88 | | 89 | |- .config() <·· accessors to some deployment configuration to allow passing external configuration 90 | | Ex: 91 | | │ public static final String CONFIG_WIKIDB_QUEUE = "wikidb.queue"; 92 | | │ ... 93 | | │ wikiDbQueue = config() .getString(CONFIG_WIKIDB_QUEUE, "wikidb.queue"); 94 | | │ ^^^^^^ ^^^^^^^^^^^^^^ 95 | | or Integers, booleans Default param 96 | | complex JSON data, ... if first is null 97 | ``` 98 | 99 | ``` 100 | |import io.vertx.core.AbstractVerticle; 101 | |public class MainVerticle extends AbstractVerticle { 102 | | 103 | | @Override 104 | | public void start(Future startFuture) { 105 | | ^^^^^^^^^^^^^^^^^^^^^^^^ 106 | | No params for the sync version 107 | | startFuture.complete(); 108 | | } 109 | |} 110 | ``` 111 | 112 | ## VertX Event Bus [[{vertx.event_bus]] 113 | * main tool for communication between verticles using messages and one of: 114 | * point-to-point messaging 115 | * request-response messaging 116 | * publish / subscribe for broadcasting messages 117 | ``` 118 | | |verticle 01| |verticle 02| 119 | | (HTTP server) event─bus (DDBB client) 120 | | │ ║ ║ │ 121 | | ├─── user 1234 ? ─────────>║ ║ │ 122 | | │ ║ ║─────── user 1234 ? ───>│ 123 | | │ ║ ║ ├── ....─> 124 | | │ ║ ║ │<─ .... 125 | | │ ║ ║<────── user 1234 ? ────┤ 126 | | │<── user 1234 ──────────║ ║ │ 127 | | ^^^^^^^^^ ║ ║ 128 | | Message are free-form 129 | | strings. (JSON recomended ^ 130 | | for multi─language support │ 131 | | │ 132 | | 133 | | - It can be accessed through (simple) 134 | | TCP protocol for 3rd party apps 135 | | or exposed over general-purpose 136 | | messaging bridges (AMQP, Stomp,...) 137 | | - Support cluster support sending 138 | | messages to verticles deployed&running 139 | | in different application nodes 140 | | - a SockJS bridge allows web applications 141 | | to seamlessly communicate over the event bus 142 | | from JavaScript running in the browser by 143 | | receiving and publishing messages just like 144 | | any verticle would do. 145 | ``` 146 | [[vertx.event_bus}]] 147 | 148 | [[{vertx.101]] 149 | ## VertX Event-Loop to threading configuration 150 | 151 | * By default Vert.x attaches `CPU-core-thread 1<-->2 event loops` 152 | * VertX Threading Strategies: 153 | * Incoming network data -> accepting thread"N": 154 | * accepting thread "N" -> event-loop thread: +event with data 155 | 156 | * When a verticle opens a network server and is deployed more than once, 157 | then the events are being distributed to the verticle instances in a 158 | round-robin fashion which is very useful for maximizing CPU usage with 159 | lots of concurrent networked requests. 160 | [[}]] 161 | 162 | ## HTTPS AAA 163 | * 164 | 165 | [[}]] 166 | 167 | # Vert.x Testing [[{vertx.testing,qa.testing]] 168 | * 169 | * 170 | 171 | * Ex.1: 172 | ``` 173 | | @RunWith(VertxUnitRunner.class) // <·· annotation to JUnit tests to allow vertx-unit features 174 | | public class SampleHttpServerTest { 175 | | 176 | | private Vertx vertx ; 177 | | 178 | | @Before public void prepare() { vertx = Vertx.vertx(); } 179 | | 180 | | @After public void finish(TestContext context ) { 181 | | vertx.close( context .asyncAssertSuccess()); 182 | | } 183 | | 184 | | @Test 185 | | public void start_http_server 186 | | (TestContext context ) { 187 | | ^^^^^^^ 188 | | // provided by the runner 189 | | // provides access to basic assertions, 190 | | // a context to store data, 191 | | // and several async-oriented helpers 192 | | 193 | | Async async = context .async(); 194 | | 195 | | vertx 196 | | .createHttpServer().requestHandler( 197 | | req -> req.response().putHeader("Content-Type", "text/plain").end("Ok") 198 | | ) 199 | | .listen( 200 | | 8080, 201 | | context .asyncAssertSuccess( 202 | | server -> { 203 | | WebClient webClient = WebClient.create(vertx); 204 | | webClient.get(8080, "localhost", "/").send(ar -> { 205 | | if (ar.succeeded()) { 206 | | HttpResponse response = ar.result(); 207 | | context .assertTrue(response.headers().contains("Content-Type")); 208 | | context .assertEquals("text/plain", response.getHeader("Content-Type")); 209 | | context .assertEquals("Ok", response.body().toString()); 210 | | async .complete(); 211 | | } else { 212 | | async .resolve(Promise.failedPromise(ar.cause())); 213 | | } 214 | | }); 215 | | })); 216 | | } 217 | | } 218 | ``` 219 | 220 | * Ex.2: check/test that a timer task has been called once, 221 | and that a periodic task has been called 3 times.
222 | 223 | 224 | ``` 225 | | RunWith(VertxUnitRunner.class) 226 | | public class WikiDatabaseVerticleTest { 227 | | private Vertx vertx; 228 | | @Before public void prepare(TestContext context) { vertx = ... } 229 | | @After public void finish(TestContext context) { vertx.close(...); } 230 | | 231 | | @Test /*( timeout=5000 )*/ 232 | | public void async_behavior(TestContext context) { // 1 233 | | Vertx vertx = Vertx.vertx(); 234 | | Async a1 = context.async(); 235 | | Async a2 = context.async(3); // ← works as a countdown that 236 | | completes successfully after 3 calls. 237 | | vertx.setTimer(100, n -> a1.complete()); 238 | | vertx.setPeriodic(100, n -> a2.countDown()); 239 | | } 240 | | 241 | | @Test 242 | | public void crud_operations(TestContext context) { 243 | | Async async = context.async(); 244 | | 245 | | service.createPage(..., ..., 246 | | context.asyncAssertSuccess(v1 -> { 247 | | service.fetchPage(..., 248 | | context.asyncAssertSuccess(json1 -> { 249 | | context.assertTrue(json1.getBoolean("found")); 250 | | context.assert...; 251 | | ... 252 | | async.complete(); // 1 253 | | })); 254 | | })); 255 | | async.awaitSuccess( 5000 ); 256 | | } 257 | | } 258 | ``` 259 | [[vertx.testing}]] 260 | 261 | ## Reusable Verticles [[{vertx.101]] 262 | 263 | * 264 | * resulting verticles will not have direct references to each other 265 | as they will only agree on destination names in the event bus as well 266 | as message formats. 267 | * messages sent on the event bus will be encoded in JSON. 268 | 269 | * Ex: 270 | () 271 | ``` 272 | | HttpServerVerticle.java 273 | | WikiDatabaseVerticle.java 274 | | MainVerticle.java <······ its sole purpose is to bootstrap the app and deploy other verticles. 275 | ``` 276 | 277 | 278 | ``` 279 | | package io.vertx.guides.wiki; 280 | | 281 | | public class MainVerticle extends AbstractVerticle { 282 | | @Override 283 | | public void start(Promise promise) { 284 | | Promise promise01 = Promise.promise(); 285 | | vertx.deployVerticle(new WikiDatabaseVerticle(), promise01); 286 | | promise01.future().compose(id -> { 287 | | Promise promise02 = Promise.promise(); 288 | | vertx.deployVerticle( 289 | | "io.vertx.guides.wiki.HttpServerVerticle", // <4> 290 | | new DeploymentOptions().setInstances(2), // <5> 291 | | promise02); 292 | | return promise02.future(); 293 | | }).setHandler(ar -> { 294 | | if (ar.succeeded()) { 295 | | promise.complete(); 296 | | } else { 297 | | promise.fail(ar.cause()); 298 | | } 299 | | }); 300 | | } 301 | | } 302 | ``` 303 | [[}]] 304 | 305 | ## Creating an HttpServer [[{vertx.101]] 306 | * REF: 307 | 308 | ``` 309 | | import io.vertx.ext.web.handler.BodyHandler; 310 | | public class MainVerticle extends io.vertx.core.AbstractVerticle { 311 | | ... 312 | | private io.vertx.core.http.HttpServer server = 313 | | vertx.createHttpServer(); // vertx defined in Parent Class 314 | | 315 | | @Override 316 | | public void start(Future startFuture) { 317 | | Json.mapper.registerModule(new JavaTimeModule()); 318 | | FileSystem vertxFileSystem = vertx.fileSystem(); 319 | | vertxFileSystem.readFile("swagger.json", readFile -> { 320 | | if (readFile.succeeded()) { 321 | | // Swagger swagger = new SwaggerParser() 322 | | // .parse(readFile.result() 323 | | // .toString(Charset.forName("utf-8"))); 324 | | // SwaggerManager.getInstance().setSwagger(swagger); 325 | | Router router = Router.router(vertx); 326 | | router.get (ROUTE_ENTITY01+"/:id") .handler(this:: GetEntity01Handler); 327 | | router.post (ROUTE_ENTITY01 ).handler(BodyHandler.create()).handler(this::PostEntity01Handler); 328 | | // └───────┬───────────┘ └───────────┬───────────┘ 329 | | // decode POST req.body (forms,...) to function signature: 330 | | // Vert.x buffer objects void functionName( RoutingContext context ) 331 | | router.delete(ROUTE_ENTITY01+"/:id") .handler(this::DeleEntity01Handler); 332 | | log.info("Starting Server... Listening on "+RC.host+":"+RC.port); 333 | | vertx.createHttpServer(_getHTTPServerOptions()). 334 | | requestHandler(router::accept).listen( 335 | | 8080, // <·· Different deployments can share the port. Vertx will round-robin 336 | | /* AsyncResult */ar -> { 337 | | if (ar.succeeded()) { 338 | | LOGGER.info("HTTP server running on port 8080"); 339 | | startFuture.complete(); 340 | | } else { 341 | | LOGGER.error("Could not start a HTTP server", ar.cause()); 342 | | startFuture.fail(ar.cause()); 343 | | } 344 | | }); 345 | | } else { 346 | | startFuture.fail(readFile.cause()); 347 | | } 348 | | }); 349 | | } 350 | | } 351 | ``` 352 | [[}]] 353 | 354 | 355 | [[{vertx.101,devops]] 356 | ## Maven Bootstrap 357 | * 358 | * 359 | 360 | * Ex: minimally viable wiki written with Vert.x 361 | * Features: 362 | * server-side rendering 363 | * data persistence through a JDBC connection and async ddbb access 364 | * Dependencies: 365 | * Vert.x web: "elegant" APIs to deal with routing, request payloads, etc. 366 | * Vert.x JDBC client: asynchronous API over JDBC. 367 | * other libreries for HTML/md rendering 368 | 369 | ``` 370 | | ┌ · configured pom.xml: 371 | | $ URL="https://github.com/vert-x3" │ - Maven Shade Plugin configured to create a single 372 | | $ URL="${URL}/vertx─maven─starter" <─┤ "fat" Jar archive with all required dependencies 373 | | $ git clone ${URL} project01 │ - Exec Maven Plugin to provide the exec:java goal 374 | | $ cd project01 │ that in turns starts the application through the 375 | | │ Vert.x io.vertx.core.Launcher class. 376 | | │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 377 | | │ (equivalent to running using the vertx cli tool) 378 | | │ 379 | | │ · sample verticle 380 | | │ 381 | | │ · unit tests 382 | | │ 383 | | └ · redeploy.sh: auto compile+redeploy on code changes. 384 | | (adjust $VERTICLE in script to match main verticle) 385 | | $ mvn package exec:java ← check that maven install is correct 386 | | $ mvn clean package 387 | | $ java -jar target/project01-SNAPSHOT-fat.jar 388 | ``` 389 | 390 | * Tip: The SQL database modules supported by the Vert.x project do not currently 391 | offer anything beyond passing SQL queries (e.g., an object-relational mapper) 392 | as they focus on providing asynchronous access to databases. 393 | However, nothing forbids using more advanced modules from the community, 394 | and we especially recommend checking out projects like this jOOq generator 395 | for VertX 396 | [[}]] 397 | 398 | 399 | # Vert.X: Non-Classified / TODO [[{PM.TODO]] 400 | ## RxJava Integration: [[{vertx.101,architecture.async.reactive.rxjava,devops.k8s,architecture.frontend]] 401 | * 402 | Refactoring some code to use reactive programming with the 403 | popular RxJava library and its Vert.x integration. 404 | [[async.reactive.rxjava}] 405 | 406 | ## Vertx+k8s 407 | * 408 | 409 | ## SockJS: 410 | * 411 | * Event-bus bridge allowing web apps to seamlessly communicate over 412 | * event bus from JavaScript running in the browser by receiving and publishing 413 | * messsages just like any verticle would do 414 | ### Angular Client: 415 | 416 | 417 | ## Cookbook recipe: 418 | ``` 419 | Vertx vertx = Vertx.vertx(); 420 | HttpServer server = vertx.createHttpServer(); 421 | server.requestHandler(req -> { 422 | req.response().end("Hello world"); 423 | }); 424 | server.listen(8080); 425 | ``` 426 | 427 | ## Others: 428 | 429 | > There are many features and modules that we haven’t covered in this guide, such as:
430 | > - Clustering using Hazelcast, Infinispan, Apache Ignite or Zookeeper,
431 | > - Exposing&consuming over HTTP/2, possibly (but not necessarily) with gRPC
432 | > - Using NoSQL databases such as MongoDB or Redis.
433 | > - Sending emails over SMTP.
434 | > - Messaging with AMQP, Stomp, Kafka, MQTT or RabbitMQ.
435 | > - Using OAuth2 authentication from custom and popular providers.
436 | > - Vert.x sync for writing blocking-style code that is later
437 | > turned into fibers non-blocking code at runtime. 438 | > - Publishing and discovering micro-services from registries, for
439 | > instance when deploying on cloud environments like OpenShift, 440 | > - Exposing metrics and health checks.
441 | 442 | 443 | ## What's new Vert.X 4.0: (2020-12-09): 444 | * 445 | 446 | ## Vert.X Migration Guide (2020-12-09): 447 | * 448 | 449 | ## Vert.x 3.9 Fluent API Query [[{vertx,PM.TODO]] 450 | * 451 | * Red Hat build of Eclipse Vert.x 3.9 brings Fluent API Query 452 | [[}]] 453 | 454 | [[{standards.openapi,architecture.api_management]] 455 | [[vertx,PM.low_code,qa.documentation,PM.backlog]] 456 | ## OpenAPI: contract-driven Dev 457 | 458 | - "Contract Driven Development" (or API Design First approach) is a methodology 459 | that uses declarative API Contracts to enable developers to efficiently design, 460 | communicate, and evolve their HTTP APIs, while automating API implementation 461 | phases where possible. 462 | [[standards.openapi}]] 463 | 464 | 465 | ## Vert.X Mutiny APIs [[{vertx.*,PM.TODO.NOW]] 466 | * Reactive isn't Complicated with Vert.x and the new Mutiny APIs 467 | 468 | * VertX Mutiny Zero by Julien Ponge 469 | 470 | latest developments around Mutiny, Mutiny Zero and switching from legacy Reactive Streams to JDK Flow. 471 | 472 | [[}]] 473 | 474 | 475 | [[PM.TODO}]] 476 | 477 | ## Quarkus and VertX 478 | * 479 | 480 | ## Reactive Spring with Vert.x 481 | * 482 | * Reactive Spring Boot programming with Vert.x 483 | The latest bundle of Red Hat supported Spring Boot starters was recently 484 | released. In addition to supporting the popular Red Hat products for our Spring 485 | Boot customers, the Red Hat Spring Boot team was also busy creating new ones. 486 | The most recent technical preview added is a group of Eclipse Vert.x Spring 487 | Boot starters, which provide a Spring-native vocabulary for the popular JVM 488 | reactive toolkit. 489 | [[PM.TODO}]] 490 | 491 | 492 | [[{]] 493 | ## VertX Mutiny Zero 494 | 495 | * by Julien Ponge 496 | * 497 | 498 | * latest developments around Mutiny, Mutiny Zero and switching from 499 | legacy Reactive Streams to JDK Flow. 500 | 501 | * 502 | [[}]] 503 | 504 | [[vertx}]] 505 | -------------------------------------------------------------------------------- /whats_new.txt: -------------------------------------------------------------------------------- 1 | [[{PM.WHATS_NEW,doc_has.ext_resource,architecture.async.reactive,troubleshooting.debugging]] 2 | # What's new 3 | 4 | [[{cloud.101,PM.TODO]] 5 | ## JVM Journey to Cloud-native (by BellSoft) 6 | ``` 7 | · JDK 16: Elastic metaspace, Alpine Linux port 8 | · JDK 15: Reimplement Datagram Socker API, Hidden Classes. 9 | · JDK 14: JFR Event Streaming. 10 | · JDK 13: Uncommit unused memory for ZGC. 11 | · JDK 12: Return unused memory, Shenandoah GC. 12 | · JDK 11: ZGC. 13 | · JDK 10: Docker awareness. 14 | · JDK 09: Compact Strings, HTTP/2 client. 15 | ``` 16 | [[cloud.101}]] 17 | 18 | ## RELATED 19 | * [[{troubleshooting,PM.low_code,qa}]] 20 | OpenRewrite enables large-scale distributed source code refactoring 21 | for framework migrations, vulnerability patches, and API migrations. 22 | While the original focus was on the Java language, OpenRewrite is 23 | continuously expanding languages and frameworks. 24 | * 25 | service designed to simplify the process of upgrading existing Java 26 | application code through generative artificial intelligence. The new 27 | feature aims to minimize legacy code and automate common language 28 | upgrade tasks required to move off older language versions. 29 | * Currently, Code Transformation supports the upgrade to Java 17 of 30 | Java 8 and 11 applications built with Maven 31 | """Amazon Q Code Transformation uses OpenRewrite to accelerate Java 32 | upgrades for customers (...) Our internal experiments with 33 | OpenRewrite have been extremely positive, but there are situations 34 | where additional iterative debugging is needed to get to a fully 35 | building application. The iterative AI-powered debugging has me 36 | excited about Amazon Q Code Transformation."""
37 | """ An internal Amazon team of five people successfully upgraded 38 | one thousand production applications from Java 8 to 17 in 2 days. It 39 | took, on average, 10 minutes to upgrade applications, and the longest 40 | one took less than an hour. 41 | """ 42 | * it requires the CodeWhisperer Professional Tier (USD 43 | 19/user/month) in the AWS Toolkit for IntelliJ IDEA or the AWS 44 | Toolkit for Visual Studio Code. 45 | 46 | ### v.22 47 | * JEP 461, Stream Gatherers (Preview) enhances Stream API with custom intermediate operations. 48 | ``` 49 | | ┌─······· input elements 50 | | · ┌─···· optionally mutating intermediate state 51 | | · · ┌·· transformed type output 52 | | v v v 53 | | Gatherer Interface 54 | | 55 | | This operation is defined by four key functions: 56 | | 1st. Optional initializer, providing a private state for processing stream elements. 57 | | 2nd. integrator, responsible for integrating new elements from the input stream. 58 | | It may emit elements into the output stream or terminate processing early. 59 | | 3rd. Optional combiner, which facilitates parallel evaluation. 60 | | 4th. Optional finisher, invoked when there are no more input elements, potentially 61 | | emitting additional output elements. 62 | | Depending on the use of a combiner function, the entire 63 | | operation can be executed either sequentially or in parallel. 64 | ``` 65 | 66 | ### v.21 2023-09 67 | 68 | * JEP 453 (Preview): Structured Concurrency, simplifying concurrent 69 | programming by introducing an API for structured concurrency. 70 | Structured concurrency treats groups of related tasks running in 71 | different threads as a single unit of work, thereby streamlining 72 | error handling and cancellation, improving reliability, and enhancing 73 | observability. This is a preview API. 74 | 75 | 76 | * JEP 430: String Templates (Preview).
77 | ``` 78 | | val name = "Joan"; 79 | | String info = STR."My name is \{name}"; // <·· STR: Template processor. 80 | | > My name is Joan" 81 | | 82 | | int x = 10, y = 20; 83 | | String s = STR."\{x} + \{y} = \{x + y}"; // <·· Embedded expression is OK! 84 | | > "10 + 20 = 30" 85 | | 86 | | val filePath = "somePath.txt"; 87 | | val fileExists = true; 88 | | String msg = STR."The file \{filePath} \{fileExists} ? "does" : "does not"} exist"; 89 | | ^ ^ ^ ^ 90 | | No need to scape '"' 91 | ``` 92 | * JEP 431: Sequenced Collections 93 | * JEP 439: Generational ZGC 94 | * JEP 440: Record Patterns 95 | * JEP 441: Pattern Matching for switch 96 | * JEP 442: Foreign Function & Memory API (Third Preview) 97 | * JEP 443: Unnamed Patterns and Variables (Preview) 98 | * JEP 444: **Production ready Virtual Threads**. Now we can reserve a full 99 | thread for a connected client, even if we have 100.000 simultaneous 100 | users with long connections, since Threads are virtual (vs OS backed). 101 | No need anymore for complex asynchronous processing and pooling in 102 | servlets and related code. 103 | * JEP 445: Unnamed Classes and Instance Main Methods (Preview) 104 | * JEP 446: Scoped Values (Preview) 105 | * JEP 448: Vector API (Sixth Incubator) 106 | * JEP 449: Deprecate the Windows 32-bit x86 Port for Removal 107 | * JEP 451: Prepare to Disallow the Dynamic Loading of Agents 108 | * JEP 452: Key Encapsulation Mechanism API 109 | 110 | * JEP 453: Structured Concurrency (Preview) 111 | 112 | 113 | 114 | ### v.17 2021-09 115 | * 1st long-term support (LTS) release after JDK 11 (2018). 116 | * Algebraic Data Types (Records and Sealed Types -JEP 409-) is now complete. 117 | This is a major step forward in terms of data modeling and enhancing 118 | the OO capabilities of the Java language. 119 | WARN: Pattern Matching, complementing and built upon algebraic data 120 | types is not available until JDK 21. 121 | 122 | * 409: Sealed Classes 123 | ``` 124 | | package com.example.geometry; 125 | | 126 | | public abstract sealed class Shape ← Syntax 1: Permitted classes in 127 | | permits com.example.polar.Circle, different file 128 | | com.example.quad.Rectangle, 129 | | com.example.quad.simple.Square { ... } 130 | | 131 | | abstract sealed class Root { ... ← Syntax 2: Permitted classed inside 132 | | final class A extends Root { ... } parent class. 133 | | final class B extends Root { ... } 134 | | final class C extends Root { ... } 135 | | } 136 | ``` 137 | * 412: Foreign Function& Memory API (Incubator): 138 | API to efficiently interoperate with native code through foreign 139 | functions while safely accessing foreign memory (not managed by 140 | the JVM). Future replacement for the brittleness and danger of JNI. 141 | Easy of use and able to work with different kinds of foreign memory 142 | (native memory, persistent memory, managed heap memory,..). 143 | * 306: Restore Always-Strict Floating-Point Semantics for 144 | numerically-sensitive libraries, including `java.lang.Math` and 145 | `java.lang.StrictMath`. 146 | * 356: Enhanced Pseudo-Random Number Generators [[{security.cryptography,PM.TODO}]] 147 | * 403: **Strongly Encapsulate JDK Internals**. it will no longer be 148 | possible to bypass strong encapsulation via --illegal-access flag. 149 | * 406: Pattern Matching for switch (Preview) 150 | * 407: Remove RMI Activation 151 | * 410: Remove the Experimental AOT and JIT Compiler 152 | * 411: Deprecate the Security Manager for Removal 153 | * 414: Vector API (Second Incubator) 154 | * 415: Context-Specific Deserialization Filters 155 | * 382: New macOS Rendering Pipeline 156 | * 391: macOS/AArch64 Port 157 | * 398: Deprecate the Applet API for Removal 158 | 159 | 160 | ### v.15 2020-09-15 161 | 162 | 163 | * **RECORDS!!!**: non-verbose inmutable classes. [[{java_lang.records}]] 164 | * production ready ZGC low-latency garbage collector, expected to be 165 | quite impactful for a multitude of workloads. 166 | * text blocks (JEP 378): strings spanning several lines ("templates", ...) 167 | * JEP 360: Sealed Classes (Preview): Avoid extending class designed to NOT 168 | be extended by third parties. 169 | * JEP 383: Foreign (C/Assembler) Memory Access API (Preview) (Project Panama). 170 | 171 | 172 | ### v.14 2020-03-?? 173 | * [REF@infoq.com](https://www.infoq.com/news/2019/12/java14-feature-freeze/) 174 | * More container awareness... 175 | * NUMA container support added to hotspot (JDK-8198715) 176 | * Add Container MBean to JMX (JDK-8199944) 177 | * Record-types (Preview) 178 | * Shenandoah GC: 179 | [REF1@redhat.com](https://developers.redhat.com/blog/2020/03/04/shenandoah-gc-in-jdk-14-part-1-self-fixing-barriers/) 180 | """ ... self-fixing barriers aims to reduce local latencies 181 | spent in barrier mid-and slow paths. The second will cover 182 | concurrent root processing and concurrent class unloading. """ 183 | [REFw@redhat.com](https://developers.redhat.com/blog/2020/03/09/shenandoah-gc-in-jdk-14-part-2-concurrent-roots-and-class-unloading/) 184 | """... concurrent roots processing and concurrent class unloading, 185 | aim to reduce GC pause time by moving GC work from the pause 186 | to a concurrent phase.""" 187 | 188 | 189 | ### v 12,13 190 | * REFs: 191 | * 192 | * 193 | * 194 | * 195 | * add container support to jhsdb command (JDK-8205992) 196 | * Impove systemd slice memory limit suppot (JDK-8217338) 197 | * More container awareness: [[{devops.containarization]]
198 | * Flight Recorder impovements for containers (JDK-8203359) 199 | * Improve container support when Join Contollers option is used (JDK-8217766) 200 | * JFR jdk.CPUInformation event epots incorrect info. when running in 201 | Docker container (JDK-8219999) [[}]] 202 | 203 | 204 | ### v.11(LTS) 2018/09 205 | * REFs: 206 | * 207 | * 208 | * More container awareness... [[{devops.containarization]] 209 | * Remove -XX:+UnlockExperimentalVMOptions, -XX:+UseGroupMemoryLimitFoHeap (JDK-8194086) 210 | * jcmd -l & jps: do not list JVMs in Docker containers (JDK-8193710) 211 | * Container Metrics (-XshowSettings:system) (JDK-8204107) 212 | * Update CPU count algorithm when both cpu shares and quotas are used (JDK-8197867) 213 | --XX:+PrefectContainerQuotaForCPUCount [[}]] 214 | 215 | * Flight Recorder, ZGC, CDS, ... opensourced : Oracle JDK "==" OpenJDK 216 | #### New major features: 217 | * Autocompilation(JEP 330) 218 | ``` 219 | $ java someFile.java 220 | ``` 221 | * New string methods. 222 | * Local-Variable Syntax for Lambda Parameters (JEP 323) 223 | ``` 224 | (var s1, var s2) -> s1 + s2 225 | ``` 226 | (While it's possible to just skip the type in the lambda 227 | it becomes a need when for annotations like @Nullable ) 228 | * Nested Based Access Control: fix some issues when using (**discouraged**) reflection. 229 | * Dynamic Class-File Constants(JEP 309):
230 | `class-file` format now extends support a new constant pool form: 231 | CONSTANT_Dynamic , reduce the cost and disruption of developing 232 | new forms of materializable class-file constraints. 233 | 234 | * Epsilon: A No-Op Garbage Collector(JEP 318): [[{performance,architecture.batch]]
235 | (Experimental) 236 | Unlike the JVM GC which is responsible for allocating memory 237 | and releasing it, Epsilon only allocates memory. Useful for: 238 | * Extremely short lived jobs 239 | * Performance testing 240 | * Memory pressure testing 241 | * VM interface testing 242 | * Last-drop latency improvements 243 | * Last-drop throughput improvements [[}]] 244 | 245 | * Remove JavaEE and CORBA Modules(JEP 320) 246 | 247 | * New HTTP/1.1, HTTP/2 and WebSockets Client (JEP 321) [[{io.http]] 248 | designed to improve overall performance. [[}]] 249 | * TLS 1.3 [[{SECURITY.CRYPTOGRAPHY]] 250 | * ChaCha20,Poly1305 Crypto (JEP 329) implemented in 251 | the SunJCE provider. 252 | [[}]] 253 | 254 | * Convenient Reading/Writing Strings to/from Files [[{PM.TODO}]] 255 | 256 | * ZGC: (JEP 333) Experimental (Production in 15+) 257 | - Scalable Low-Latency Garbage Collector. 258 | - low latency GC. 259 | - sub-10ms pause times, less than 15% perf.penalty. 260 | 261 | 262 | ### v.10 (2018/03) 263 | 264 | #### More container awareness... [[{devops.containarization]] 265 | * Improve heap memory allocations (JDK-8196595): 266 | ``` 267 | | --XX:InitialRAMPercentage, --X:MaxRAMPercentage and -XX:MinRAMPercentage 268 | | --XX:InitialRAMFraction , --X:MaxRAMFraction and -XX:MinRAMFraction deprecated 269 | ``` 270 | * Total number of CPUs available to the Java Process calculated from 271 | ``` 272 | | --cpus, --cpu-shares, --cpu-quota (JDK-8146115) 273 | ``` 274 | (Use --XX:-UseContainerSupport to return to the old behaviour) 275 | * Attach in linux became elative to `/proc/pid/root` and namespace aware (jcmd, jstack,...) 276 | * See [REF@aboullaite.me](https://aboullaite.me/docker-java-10/) 277 | JVMs before 10 had been implemented before cgroups, hence not 278 | optimized for executing inside a container. 279 | [[}]] 280 | 281 | * Application Data-Class Sharing extends [[{devops,performance]] 282 | Class-Data Sharing ("CDS") allowing application classes 283 | to be placed in the shared archive in order to improve 284 | startup and footprint. [[}]] 285 | * Parallel Full GC for G1: improves G1 worst-case latencies [[{JVM.GC}]] 286 | * Consolidate JDK Forest into a Single Repository !!!! 287 | * Local-Variable Type Inference: var a = ....; 288 | * javah superseded by functionality in javac. 289 | * Thread-Local Handshakes allows to execute a callback on [[{JVM,performance]] 290 | threads without performing a global VM safepoint. 291 | Makes it both possible and cheap to stop individual 292 | threads and not just all threads or none. [[}]] 293 | * default set of Root Certificates CA in the JDK. 294 | * Heap Allocation on Alternative Memory Devices 295 | enables the HotSpot VM to allocate the Java object 296 | heap on an alternative memory device, such as an 297 | NV-DIMM, specified by the user. 298 | 299 | 300 | ### v.9(2017/09) 301 | * Containerization work [[{devops.containarization]] 302 | ``` 303 | | └ --XX:ParallerGCThreads and --XX:CICompilerCount are set based on 304 | | Containers CPU limits (can be overriden) 305 | | - Calculated from --cpuset-cpus 306 | | 307 | | └ Memory Configuration for containers 308 | | -XX:+UnlockExperimentalVMOptions 309 | | -XX:+UseGroupMemoryLimitFoHeap 310 | | - set -XX:MaxRAMFraction to 2 (default is 4) [[}]] 311 | ``` 312 | * jshell 313 | * jlink -release 314 | * Multi-release JARs 315 | 316 | #### **Java Platform Module System ("Jigsaw")** [[{]] 317 | * JPMS divides the JDK into a set of modules for combining at run, compile, or build time. 318 | * enabling understanding of dependencies across modules. 319 | * allows developers to more easily assemble and maintain sophisticated applications. 320 | * allows to scale down to smaller devices. 321 | * improves security and performance. 322 | * aspects include: 323 | * application packaging 324 | * JDK modularization 325 | * reorganizing source code into modules. 326 | * Build system is enhanced to compile modules 327 | and enforce module boundaries at build time. 328 | (Java 9 allows illegal reflective access to help migration) 329 | [[}]] 330 | 331 | 332 | 333 | * ahead-of-time (AoT) compilation (experimental) [[{performance]] 334 | improve startup time, with limited impact on peak performance.[[}]] 335 | * **REPL (read-eval-print loop) jShell:**
336 | jShell: interactively evaluates statements "a la script" with 337 | tab completion, automatic terminal semicolons and API for IDE integration. 338 | 339 | [[{java_lang.functional]] 340 | * Reactive Streams
341 | (https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.1/README.md#specification) 342 | Small spec, also adopted in Java 9, that defines the interaction 343 | between asynchronous components with back pressure. For example a 344 | data repository — acting as Publisher, can produce data that an 345 | HTTP server — acting as Subscriber, can then write to the 346 | response. The main purpose of Reactive Streams is to allow the 347 | subscriber to control how fast or how slow the publisher will produce 348 | data. 349 | * Streams API Enhancements:
350 | * Java 8 Stream API allows processing data declaratively 351 | while leveraging multicore architectures. 352 | * Java 9 adds methods to conditionally take and drop items 353 | from Stream, iterate over Stream elements, and create a 354 | stream from a nullable value while **expanding the set of 355 | Java SE APIs that can serve as Streams sources**. 356 | [[java_lang.functional}]] 357 | 358 | * Code cache can be divided into segments to improve performance 359 | and allow extensions such as fine-grained locking resulting in improved sweep 360 | times. 361 | 362 | * (Datagram Transport Layer Security) DTLS security API, [[{security]] 363 | preventing eavesdropping, tampering, and message forgery 364 | in client/server communications. [[}]] 365 | 366 | #### Java 9 deprecates and removes .... 367 | * Applet API and appletviewer (alternative: Java Web Start) 368 | * Concurrent Mark Sweep (CMS) GC. 369 | * JVM TI (Tool Interface) hprof (Heap Profiling) agent, superseded in the JVM. 370 | * jhat tool, obsoleted by superior heap visualizers and analyzers. 371 | 372 | 373 | ### Java 8 374 | 375 | * First version to support Containers. **WARN**: Do not use any version below that. 376 | 377 | * [[{PM.TODO]] 378 | 379 | 380 | [[}]] 381 | 382 | * TLS enhancements backported to 1.8 (HTTP2) 383 | 384 | 385 | [[PM.WHATS_NEW}]] 386 | -------------------------------------------------------------------------------- /who_is_who.txt: -------------------------------------------------------------------------------- 1 | # Who-is-Who [[{]] 2 | (Forcibly incomplete but still quite pertinent list of core developers and companies) 3 | * James Arthur Gosling: Founder and lead-designer of the Java Programming Language 4 | 5 | 6 | * Joshua J. Bloch: 7 | 8 | * Author of the book "Effective Java" (a must read) 9 | and co-author of two other books: 10 | * Java Puzzlers (2005) 11 | * Java Concurrency in Practice (2006) 12 | * Led the design and implementation of numerous 13 | Java platform features, including the 14 | Java Collections Framework, the java.math package, 15 | and the assert mechanism. 16 | 17 | * Julien Viet: 18 | 19 | Core developer of VertX, crash (http://www.crashub.org/), 20 | and many other interesting Java software. 21 | 22 | 23 | * Ben Evans: 24 | * jClarity Co-founder. 25 | * Java Champion, author, speaker, consultant 26 | * voting member on Java’s governing 27 | * Author of 5 books 28 | * “The Well-Grounded Java Developer”, 29 | * new editions of “Java in a Nutshell”, 30 | * “Java: The Legend” and “Optimizing Java”. 31 | * Track lead for Java / JVM at InfoQ. 32 | 33 | IMPLEMENTING A SIMPLE JVM IN JAVA AND RUST 34 | ... I will explain how we might start to implement a JVM from scratch.. then 35 | we will show how the Rust programming language provides a good alternative 36 | implementation language for our simple JVM. We will showcase some basic Rust 37 | language features and show how they can be used to provide a version of our 38 | JVM that is much cleaner and easier to understand, even if you've never 39 | seen Rust code before!""" 40 | 41 | * Emmanuel Bernard: Distinguished Engineer and Chief Architect Data at 42 | Red Hat (middleware). His work is Open Source. He is most well known for his 43 | contributions and lead of the Hibernate projects as well as his contribution 44 | to Java standards. His most recent endeavour is Quarkus (A Kubernetes Native 45 | Java stack tailored for GraalVM and OpenJDK HotSpot, crafted from the best of 46 | breed Java libraries and standards). 47 | 48 | 49 | * Chad Arimura, vice president of Java Developer Relations at Oracle. 50 | 51 | * https://community.oracle.com/community/groundbreakers/java/java-champions 52 | * https://blogs.oracle.com/java/new-java-champions-in-2017 53 | 54 | * A N M Bazlur Rahman works at Contrast Security as a Sr. Software Engineer. 55 | * Recently (2023) he was named a Java Champion. 56 | mentor, writer, speaker at conferences, and open-source contributor. 57 | * founder and current moderator of the Java User Group in Bangladesh. 58 | * Named Most Valuable Blogger (MVP) at DZone. 59 | * editor for the Java Queue at InfoQ 60 | * editor at Foojay.io: a place for friends of OpenJDK. 61 | * He has published four books about the Java programming language in Bengali. 62 | * bachelor's degree from the Institute of Information Technology, University of 63 | Dhaka, Bangladesh, in Information Technology, majoring in Software 64 | Engineering. He currently (2023) lives in Toronto, Canada 65 | 66 | * Sven Reimers 67 | https://qconlondon.com/speakers/svenreimers 68 | * System Engineer @Airbus Defence & Space 69 | * over 25 years of experience in designing and implementing 70 | distributed systems which have a focus in the engineering domain. 71 | * lead software architect for a system and network management 72 | solution for satellite communication networks, which received the 73 | Duke'c Choice award. 74 | * He currently works on ground segment software for earth observation satellite systems. 75 | * author of several scientific articles, co-author of some Java 76 | community driven books and is a long term member of the program 77 | committee of the JavaOne conference. He was chosen a Java Champion by 78 | his peers. 79 | 80 | * Ron Pressler is the technical lead for Project Loom, which aims to add delimited 81 | continuations, fibers and tail-calls to the JVM. 82 | 83 | * David Delabassée 84 | * Java Platform Group - Oracle 85 | * 86 | * 87 | 88 | 89 | * Johannes Bechberger 90 | * JVM developer working on profilers in SapMachine team at SAP. 91 | including improvements to async-profiler and its ecosystem, 92 | a website to view the different JFR event types, and 93 | improvements to the FirefoxProfiler, making it usable in the 94 | Java world. 95 | * He started at SAP last year after two years of research studies 96 | at the KIT in the field of Java security analyses. His work today is 97 | comprised of many open-source contributions and his blog, where he 98 | writes regularly on in-depth profiling and debugging topics. 99 | 100 | * Gerrit Grunwald, JUG Leader, CRaC project, JReleaser, Azul VM, ... 101 | 102 | [[}]] 103 | --------------------------------------------------------------------------------