├── .gitignore ├── README.md ├── _config.yml ├── pom.xml └── src └── test ├── java └── com │ └── balamaci │ └── rx │ ├── BaseTestObservables.java │ ├── Part01CreateFlowable.java │ ├── Part02SimpleOperators.java │ ├── Part03MergingStreams.java │ ├── Part04HotPublishers.java │ ├── Part05AdvancedOperators.java │ ├── Part06Schedulers.java │ ├── Part07FlatMapOperator.java │ ├── Part08ErrorHandling.java │ ├── Part09BackpressureHandling.java │ └── util │ ├── Helpers.java │ └── Pair.java └── resources └── simplelogger.properties /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | ### Java template 3 | *.class 4 | 5 | # Package Files # 6 | *.jar 7 | *.war 8 | *.ear 9 | 10 | # virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml 11 | hs_err_pid* 12 | 13 | target 14 | 15 | .idea 16 | *.iml -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # RxJava 2.x 2 | 3 | View at [Github page](https://balamaci.github.io/rxjava-walkthrough/) 4 | 5 | also available for [reactor-core](https://github.com/balamaci/reactor-core-playground) 6 | 7 | ## Contents 8 | 9 | - [Flowable, Single and Observable](#flowable) 10 | - [Simple Operators](#simple-operators) 11 | - [Merging Streams](#merging-streams) 12 | - [Hot Publishers](#hot-publishers) 13 | - [Schedulers](#schedulers) 14 | - [FlatMap Operator](#flatmap-operator) 15 | - [Error Handling](#error-handling) 16 | - [Backpressure](#backpressure) 17 | - [Articles and books](#articles) 18 | 19 | ## Reactive Streams 20 | Reactive Streams is a programming concept for handling asynchronous 21 | data streams in a non-blocking manner while providing backpressure to stream publishers. 22 | It has evolved into a [specification](https://github.com/reactive-streams/reactive-streams-jvm) that is based on the concept of **Publisher<T>** and **Subscriber<T>**. 23 | A **Publisher** is the source of events **T** in the stream, and a **Subscriber** is the consumer for those events. 24 | A **Subscriber** subscribes to a **Publisher** by invoking a "factory method" in the Publisher that will push 25 | the stream items **<T>** starting a new **Subscription**: 26 | 27 | ```java 28 | public interface Publisher { 29 | public void subscribe(Subscriber s); 30 | } 31 | ``` 32 | 33 | When the Subscriber is ready to start handling events, it signals this via a **request** to that **Subscription** 34 | 35 | ```java 36 | public interface Subscription { 37 | public void request(long n); //request n items 38 | public void cancel(); 39 | } 40 | ``` 41 | 42 | Upon receiving this signal, the Publisher begins to invoke **Subscriber::onNext(T)** for each event **T**. 43 | This continues until either completion of the stream (**Subscriber::onComplete()**) 44 | or an error occurs during processing (**Subscriber::onError(Throwable)**). 45 | 46 | ```java 47 | public interface Subscriber { 48 | //signals to the Publisher to start sending events 49 | public void onSubscribe(Subscription s); 50 | 51 | public void onNext(T t); 52 | public void onError(Throwable t); 53 | public void onComplete(); 54 | } 55 | ``` 56 | 57 | ## Flowable and Observable 58 | RxJava provides more types of event publishers: 59 | - **Flowable** Publisher that emits 0..N elements, and then completes successfully or with an error 60 | - **Observable** like Flowables but without a backpressure strategy. They were introduced in RxJava 1.x 61 | 62 | - **Single** a specialized emitter that completes with a value successfully either an error.(doesn't have onComplete callback, instead onSuccess(val)) 63 | - **Maybe** a specialized emitter that can complete with / without a value or complete with an error. 64 | - **Completable** a specialized emitter that just signals if it completed successfully or with an error. 65 | 66 | Code is available at [Part01CreateFlowable.java](https://github.com/balamaci/rxjava-playground/blob/master/src/test/java/com/balamaci/rx/Part01CreateFlowable.java) 67 | 68 | ### Simple operators to create Streams 69 | ```java 70 | Flowable flowable = Flowable.just(1, 5, 10); 71 | Flowable flowable = Flowable.range(1, 10); 72 | Flowable flowable = Flowable.fromArray(new String[] {"red", "green", "blue"}); 73 | Flowable flowable = Flowable.fromIterable(List.of("red", "green", "blue")); 74 | ``` 75 | 76 | 77 | ### Flowable from Future 78 | 79 | ```java 80 | CompletableFuture completableFuture = CompletableFuture 81 | .supplyAsync(() -> { //starts a background thread the ForkJoin common pool 82 | log.info("CompletableFuture work starts"); 83 | Helpers.sleepMillis(100); 84 | return "red"; 85 | }); 86 | 87 | Single single = Single.from(completableFuture); 88 | single.subscribe(val -> log.info("Stream completed successfully : {}", val)); 89 | ``` 90 | 91 | 92 | ### Creating your own stream 93 | 94 | We can use **Flowable.create(...)** to implement the emissions of events by calling **onNext(val)**, **onComplete()**, **onError(throwable)** 95 | 96 | When subscribing to the Observable / Flowable with flowable.subscribe(...) the lambda code inside **create(...)** gets executed. 97 | Flowable.subscribe(...) can take 3 handlers for each type of event - onNext, onError and onCompleted. 98 | 99 | When using **Observable.create(...)** you need to be aware of [backpressure](#backpressure) and that Observables created with 'create' are not BackPressure aware 100 | 101 | ```java 102 | Observable stream = Observable.create(subscriber -> { 103 | log.info("Started emitting"); 104 | 105 | log.info("Emitting 1st"); 106 | subscriber.onNext(1); 107 | 108 | log.info("Emitting 2nd"); 109 | subscriber.onNext(2); 110 | 111 | subscriber.onComplete(); 112 | }); 113 | 114 | //Flowable version same Observable but with a BackpressureStrategy 115 | //that will be discussed separately. 116 | Flowable stream = Flowable.create(subscriber -> { 117 | log.info("Started emitting"); 118 | 119 | log.info("Emitting 1st"); 120 | subscriber.onNext(1); 121 | 122 | log.info("Emitting 2nd"); 123 | subscriber.onNext(2); 124 | 125 | subscriber.onComplete(); 126 | }, BackpressureStrategy.MISSING); 127 | 128 | stream.subscribe( 129 | val -> log.info("Subscriber received: {}", val), 130 | err -> log.error("Subscriber received error", err), 131 | () -> log.info("Subscriber got Completed event") 132 | ); 133 | ``` 134 | 135 | ### Streams are lazy 136 | Streams are lazy meaning that the code inside create() doesn't get executed without subscribing to the stream. 137 | So event if we sleep for a long time inside create() method(to simulate a costly operation), 138 | without subscribing to this Observable, the code is not executed and the method returns immediately. 139 | 140 | ```java 141 | public void observablesAreLazy() { 142 | Observable observable = Observable.create(subscriber -> { 143 | log.info("Started emitting but sleeping for 5 secs"); //this is not executed 144 | Helpers.sleepMillis(5000); 145 | subscriber.onNext(1); 146 | }); 147 | log.info("Finished"); 148 | } 149 | =========== 150 | [main] - Finished 151 | ``` 152 | 153 | ### Multiple subscriptions to the same Observable / Flowable 154 | When subscribing to an Observable/Flowable, the create() method gets executed for each Subscriber, the events 155 | inside **create(..)** are re-emitted to each subscriber independently. 156 | 157 | So every subscriber will get the same events and will not lose any events - this behavior is named **'cold observable'** 158 | See [Hot Publishers](#hot-publisher) to understand sharing a subscription and multicasting events. 159 | 160 | ```java 161 | Observable observable = Observable.create(subscriber -> { 162 | log.info("Started emitting"); 163 | 164 | log.info("Emitting 1st event"); 165 | subscriber.onNext(1); 166 | 167 | log.info("Emitting 2nd event"); 168 | subscriber.onNext(2); 169 | 170 | subscriber.onComplete(); 171 | }); 172 | 173 | log.info("Subscribing 1st subscriber"); 174 | observable.subscribe(val -> log.info("First Subscriber received: {}", val)); 175 | 176 | log.info("======================="); 177 | 178 | log.info("Subscribing 2nd subscriber"); 179 | observable.subscribe(val -> log.info("Second Subscriber received: {}", val)); 180 | ``` 181 | 182 | will output 183 | 184 | ``` 185 | [main] - Subscribing 1st subscriber 186 | [main] - Started emitting 187 | [main] - Emitting 1st event 188 | [main] - First Subscriber received: 1 189 | [main] - Emitting 2nd event 190 | [main] - First Subscriber received: 2 191 | [main] - ======================= 192 | [main] - Subscribing 2nd subscriber 193 | [main] - Started emitting 194 | [main] - Emitting 1st event 195 | [main] - Second Subscriber received: 1 196 | [main] - Emitting 2nd event 197 | [main] - Second Subscriber received: 2 198 | ``` 199 | 200 | ## Observable / Flowable lifecycle 201 | 202 | ### Operators 203 | Between the source Observable / Flowable and the Subscriber there can be a wide range of operators and RxJava provides 204 | lots of operators to chose from. Probably you are already familiar with functional operations like **filter** and **map**. 205 | so let's use them as example: 206 | 207 | ```java 208 | Flowable stream = Flowable.create(subscriber -> { 209 | subscriber.onNext(1); 210 | subscriber.onNext(2); 211 | .... 212 | subscriber.onComplete(); 213 | }, BackpressureStrategy.MISSING); 214 | .filter(val -> val < 10) 215 | .map(val -> val * 10) 216 | .subscribe(val -> log.info("Received: {}", val)); 217 | ``` 218 | 219 | When we call _Flowable.create()_ you might think that we're calling onNext(..), onComplete(..) on the Subscriber at the end of the chain, 220 | not the operators between them. 221 | 222 | This is not true because **the operators themselves are decorators for their source** wrapping it with the operator behavior 223 | like an onion's layers. 224 | When we call **.subscribe()** at the end of the chain, **Subscription propagates through the layers back to the source, 225 | each operator subscribing itself to it's wrapped source Observable / Flowable and so on to the original source, 226 | triggering it to start producing/emitting items**. 227 | 228 | **Flowable.create** calls **---> filterOperator.onNext(val)** which if val > 10 calls **---> 229 | mapOperator.onNext(val)** does val = val * 10 and calls **---> subscriber.onNext(val)**. 230 | 231 | [Found](https://tomstechnicalblog.blogspot.ro/2015_10_01_archive.html) a nice analogy with a team of house movers, with every mover doing it's thing before passing it to the next in line 232 | until it reaches the final subscriber. 233 | 234 | ![Movers](https://1.bp.blogspot.com/-1RuGVz4-U9Q/VjT0AsfiiUI/AAAAAAAAAKQ/xWQaOwNtS7o/s1600/animation_2.gif) 235 | 236 | ### Canceling subscription 237 | Inside the create() method, we can check is there are still active subscribers to our Flowable/Observable. 238 | 239 | There are operators that also unsubscribe from the stream so the source knows to stop producing events. 240 | It's a way to prevent to do extra work(like for ex. querying a datasource for entries) if no one is listening 241 | In the following example we'd expect to have an infinite stream, but because we stop if there are no active subscribers, we stop producing events. 242 | 243 | **take(limit)** is a simple operator. It's role is to count the number of events and then unsubscribes from it's source 244 | once it received the specified amount and calls onComplete() to it's subscriber. 245 | 246 | ```java 247 | Observable observable = Observable.create(subscriber -> { 248 | 249 | int i = 1; 250 | while(true) { 251 | if(subscriber.isDisposed()) { 252 | break; 253 | } 254 | 255 | subscriber.onNext(i++); 256 | 257 | //registering a callback when the downstream subscriber unsubscribes 258 | subscriber.setCancellable(() -> log.info("Subscription canceled")); 259 | } 260 | }); 261 | 262 | observable 263 | .take(5) //unsubscribes after the 5th event 264 | .subscribe(val -> log.info("Subscriber received: {}", val), 265 | err -> log.error("Subscriber received error", err), 266 | () -> log.info("Subscriber got Completed event") //The Complete event 267 | //is triggered by 'take()' operator 268 | 269 | ================== 270 | [main] - Subscriber received: *1* 271 | [main] - Subscriber received: *2* 272 | [main] - Subscriber received: *3* 273 | [main] - Subscriber received: *4* 274 | [main] - Subscriber received: *5* 275 | [main] - Subscriber got Completed event 276 | [main] - Subscription canceled 277 | ``` 278 | 279 | 280 | ## Simple Operators 281 | Code is available at [Part02SimpleOperators.java](https://github.com/balamaci/rxjava-playground/blob/master/src/test/java/com/balamaci/rx/Part02SimpleOperators.java) 282 | 283 | ### delay 284 | Delay operator - the Thread.sleep of the reactive world, it's pausing each emission for a particular increment of time. 285 | 286 | ```java 287 | CountDownLatch latch = new CountDownLatch(1); 288 | Flowable.range(0, 2) 289 | .doOnNext(val -> log.info("Emitted {}", val)) 290 | .delay(5, TimeUnit.SECONDS) 291 | .subscribe(tick -> log.info("Tick {}", tick), 292 | (ex) -> log.info("Error emitted"), 293 | () -> { 294 | log.info("Completed"); 295 | latch.countDown(); 296 | }); 297 | latch.await(); 298 | 299 | ============== 300 | 14:27:44 [main] - Starting 301 | 14:27:45 [main] - Emitted 0 302 | 14:27:45 [main] - Emitted 1 303 | 14:27:50 [RxComputationThreadPool-1] - Tick 0 304 | 14:27:50 [RxComputationThreadPool-1] - Tick 1 305 | 14:27:50 [RxComputationThreadPool-1] - Completed 306 | ``` 307 | 308 | The **.delay()**, **.interval()** operators uses a [Scheduler](#schedulers) by default which is why we see it executing 309 | on a different thread _RxComputationThreadPool-1_ which actually means it's running the operators and the subscribe operations 310 | on another thread and so the test method will terminate before we see the text from the log unless we wait for the completion of the stream. 311 | This is the role of the **CountdownLatch**. 312 | 313 | ### interval 314 | Periodically emits a number starting from 0 and then increasing the value on each emission. 315 | 316 | ```java 317 | log.info("Starting"); 318 | Flowable.interval(5, TimeUnit.SECONDS) 319 | .take(4) 320 | .subscribe(tick -> log.info("Subscriber received {}", tick), 321 | (ex) -> log.info("Error emitted"), 322 | () -> log.info("Subscriber got Completed event")); 323 | 324 | ========== 325 | 12:17:56 [main] - Starting 326 | 12:18:01 [RxComputationThreadPool-1] - Subscriber received: 0 327 | 12:18:06 [RxComputationThreadPool-1] - Subscriber received: 1 328 | 12:18:11 [RxComputationThreadPool-1] - Subscriber received: 2 329 | 12:18:16 [RxComputationThreadPool-1] - Subscriber received: 3 330 | 12:18:21 [RxComputationThreadPool-1] - Subscriber received: 4 331 | 12:18:21 [RxComputationThreadPool-1] - Subscriber got Completed event 332 | ``` 333 | 334 | 335 | 336 | ### scan 337 | Takes an **initial value** and a **function(accumulator, currentValue)**. It goes through the events 338 | sequence and combines the current event value with the previous result(accumulator) emitting downstream the function's 339 | result for each event(the initial value is used for the first event) 340 | 341 | ```java 342 | Flowable numbers = 343 | Flowable.just(3, 5, -2, 9) 344 | .scan(0, (totalSoFar, currentValue) -> { 345 | log.info("TotalSoFar={}, currentValue={}", 346 | totalSoFar, currentValue); 347 | return totalSoFar + currentValue; 348 | }); 349 | 350 | ============= 351 | 16:09:17 [main] - Subscriber received: 0 352 | 16:09:17 [main] - TotalSoFar=0, currentValue=3 353 | 16:09:17 [main] - Subscriber received: 3 354 | 16:09:17 [main] - TotalSoFar=3, currentValue=5 355 | 16:09:17 [main] - Subscriber received: 8 356 | 16:09:17 [main] - TotalSoFar=8, currentValue=-2 357 | 16:09:17 [main] - Subscriber received: 6 358 | 16:09:17 [main] - TotalSoFar=6, currentValue=9 359 | 16:09:17 [main] - Subscriber received: 15 360 | 16:09:17 [main] - Subscriber got Completed event 361 | ``` 362 | 363 | ### reduce 364 | reduce operator acts like the scan operator but it only passes downstream the final result 365 | (doesn't pass the intermediate results downstream) so the subscriber receives just one event 366 | 367 | ```java 368 | Flowable numbers = Flowable.just(3, 5, -2, 9) 369 | .reduce(0, (totalSoFar, val) -> { 370 | log.info("totalSoFar={}, emitted={}", 371 | totalSoFar, val); 372 | return totalSoFar + val; 373 | }); 374 | 375 | ============= 376 | 17:08:29 [main] - totalSoFar=0, emitted=3 377 | 17:08:29 [main] - totalSoFar=3, emitted=5 378 | 17:08:29 [main] - totalSoFar=8, emitted=-2 379 | 17:08:29 [main] - totalSoFar=6, emitted=9 380 | 17:08:29 [main] - Subscriber received: 15 381 | 17:08:29 [main] - Subscriber got Completed event 382 | ``` 383 | 384 | ### collect 385 | collect operator acts similar to the _reduce_ operator, but while the _reduce_ operator uses a reduce function 386 | which returns a value, the _collect_ operator takes a container supplier and a function which doesn't return 387 | anything(a consumer). The mutable container is passed for every event and thus you get a chance to modify it 388 | in this collect consumer function. 389 | 390 | ```java 391 | Flowable> numbers = Flowable.just(3, 5, -2, 9) 392 | .collect(ArrayList::new, (container, value) -> { 393 | log.info("Adding {} to container", value); 394 | container.add(value); 395 | //notice we don't need to return anything 396 | }); 397 | ========= 398 | 17:40:18 [main] - Adding 3 to container 399 | 17:40:18 [main] - Adding 5 to container 400 | 17:40:18 [main] - Adding -2 to container 401 | 17:40:18 [main] - Adding 9 to container 402 | 17:40:18 [main] - Subscriber received: [3, 5, -2, 9] 403 | 17:40:18 [main] - Subscriber got Completed event 404 | ``` 405 | 406 | because the usecase to store to a List container is so common, there is a **.toList()** operator that is just a collector adding to a List. 407 | 408 | ### defer 409 | An easy way to switch from a blocking method to a reactive Single/Flowable is to use **.defer(() -> blockingOp())**. 410 | 411 | Simply using **Flowable.just(blockingOp())** would still block, as Java needs to resolve the parameter when invoking 412 | **Flux.just(param)** method, so _blockingOp()_ method would still be invoked(and block). 413 | 414 | ```java 415 | //NOT OK 416 | Flowable flowableBlocked = Flowable.just((blockingOp())); //blocks on this line 417 | ``` 418 | 419 | In order to get around this problem, we can use **Flowable.defer(() -> blockingOp())** and wrap the _blockingOp()_ call inside a lambda which 420 | will be invoked lazy **at subscribe time**. 421 | 422 | ```java 423 | Flowable stream = Flowable.defer(() -> Flowable.just(blockingOperation())); 424 | stream.subscribe(val -> log.info("Val " + val)); //only now the code inside defer() is executed 425 | ``` 426 | 427 | 428 | ## Merging Streams 429 | Operators for working with multiple streams 430 | Code at [Part03MergingStreams.java](https://github.com/balamaci/rxjava-playground/blob/master/src/test/java/com/balamaci/rx/Part03MergingStreams.java) 431 | 432 | ### zip 433 | Zip operator operates sort of like a zipper in the sense that it takes an event from one stream and waits 434 | for an event from another other stream. Once an event for the other stream arrives, it uses the zip function 435 | to merge the two events. 436 | 437 | This is an useful scenario when for example you want to make requests to remote services in parallel and 438 | wait for both responses before continuing. It also takes a function which will produce the combined result 439 | of the zipped streams once each has emitted a value. 440 | 441 | ![Zip](https://raw.githubusercontent.com/reactor/projectreactor.io/master/src/main/static/assets/img/marble/zip.png) 442 | 443 | Zip operator besides the streams to zip, also takes as parameter a function which will produce the 444 | combined result of the zipped streams once each stream emitted its value 445 | 446 | ```java 447 | Single isUserBlockedStream = 448 | Single.fromFuture(CompletableFuture.supplyAsync(() -> { 449 | Helpers.sleepMillis(200); 450 | return Boolean.FALSE; 451 | })); 452 | 453 | Single userCreditScoreStream = 454 | Single.fromFuture(CompletableFuture.supplyAsync(() -> { 455 | Helpers.sleepMillis(2300); 456 | return 5; 457 | })); 458 | 459 | Single> userCheckStream = Single.zip(isUserBlockedStream, userCreditScoreStream, 460 | (blocked, creditScore) -> new Pair(blocked, creditScore)); 461 | 462 | userCheckStream.subscribe(pair -> log.info("Received " + pair)); 463 | ``` 464 | 465 | Even if the 'isUserBlockedStream' finishes after 200ms, 'userCreditScoreStream' is slow at 2.3secs, 466 | the 'zip' method applies the combining function(new Pair(x,y)) after it received both values and passes it 467 | to the subscriber. 468 | 469 | 470 | Another good example of 'zip' is to slow down a stream by another basically **implementing a periodic emitter of events**: 471 | 472 | ```java 473 | Flowable colors = Flowable.just("red", "green", "blue"); 474 | Flowable timer = Flowable.interval(2, TimeUnit.SECONDS); 475 | 476 | Flowable periodicEmitter = Flowable.zip(colors, timer, (key, val) -> key); 477 | ``` 478 | 479 | Since the zip operator needs a pair of events, the slow stream will work like a timer by periodically emitting 480 | with zip setting the pace of emissions downstream every 2 seconds. 481 | 482 | **Zip is not limited to just two streams**, it can merge 2,3,4,.. streams and wait for groups of 2,3,4 'pairs' of 483 | events which it combines with the zip function and sends downstream. 484 | 485 | ### merge 486 | Merge operator combines one or more stream and passes events downstream as soon as they appear. 487 | 488 | ![merge](https://raw.githubusercontent.com/reactor/projectreactor.io/master/src/main/static/assets/img/marble/merge.png) 489 | 490 | ``` 491 | Flowable colors = periodicEmitter("red", "green", "blue", 2, TimeUnit.SECONDS); 492 | 493 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS) 494 | .take(5); 495 | 496 | //notice we can't say Flowable or Flowable as the return stream o the merge operator since 497 | //it can emit either a color or number. 498 | Flowable flowable = Flowable.merge(colors, numbers); 499 | 500 | ============ 501 | 21:32:15 - Subscriber received: 0 502 | 21:32:16 - Subscriber received: red 503 | 21:32:16 - Subscriber received: 1 504 | 21:32:17 - Subscriber received: 2 505 | 21:32:18 - Subscriber received: green 506 | 21:32:18 - Subscriber received: 3 507 | 21:32:19 - Subscriber received: 4 508 | 21:32:20 - Subscriber received: blue 509 | ``` 510 | 511 | ### concat 512 | Concat operator appends another streams at the end of another 513 | ![concat](https://raw.githubusercontent.com/reactor/projectreactor.io/master/src/main/static/assets/img/marble/concat.png) 514 | 515 | ```java 516 | Flowable colors = periodicEmitter("red", "green", "blue", 2, TimeUnit.SECONDS); 517 | 518 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS) 519 | .take(4); 520 | 521 | Flowable events = Flowable.concat(colors, numbers); 522 | 523 | ========== 524 | 22:48:23 - Subscriber received: red 525 | 22:48:25 - Subscriber received: green 526 | 22:48:27 - Subscriber received: blue 527 | 22:48:28 - Subscriber received: 0 528 | 22:48:29 - Subscriber received: 1 529 | 22:48:30 - Subscriber received: 2 530 | 22:48:31 - Subscriber received: 3 531 | ``` 532 | 533 | Even if the 'numbers' streams should start early, the 'colors' stream emits fully its events 534 | before we see any 'numbers'. 535 | This is because 'numbers' stream is actually subscribed only after the 'colors' complete. 536 | Should the second stream be a 'hot' emitter, its events would be lost until the first one finishes 537 | and the seconds stream is subscribed. 538 | 539 | ## Hot Publishers 540 | We've seen that with 'cold publishers', whenever a subscriber subscribes, each subscriber will get 541 | it's version of emitted values independently, the exact set of data indifferently when they subscribe. 542 | But cold publishers only produce data when the subscribers subscribes, however there are cases where 543 | the events happen independently from the consumers regardless if someone is 544 | listening or not and we don't have control to request more. So you could say we have 'cold publishers' for pull 545 | scenarios and 'hot publishers' which push. 546 | 547 | ### Subjects 548 | Subjects are one way to handle hot observables. Subjects keep reference to their subscribers and allow 'multicasting' 549 | an event to them. 550 | 551 | ```java 552 | for (Disposable s : subscribers.get()) { 553 | s.onNext(t); 554 | } 555 | ``` 556 | 557 | Subjects besides being traditional Observables you can use the same operators and subscribe to them, 558 | are also an **Observer**(interface like **Subscriber** from [reactive-streams](#reactive-streams), implementing the 3 methods **onNext, onError, onComplete**), 559 | meaning you can invoke subject.onNext(value) from different parts in the code, 560 | which means that you publish events which the Subject will pass on to their subscribers. 561 | 562 | ```java 563 | Subject subject = ReplaySubject.create() 564 | .map(...); 565 | .subscribe(); // 566 | 567 | ... 568 | subject.onNext(val); 569 | ... 570 | subject.onNext(val2); 571 | ``` 572 | remember for 573 | ```java 574 | Observable.create(subscriber -> { 575 | subscriber.onNext(val); 576 | }) 577 | ``` 578 | 579 | ### ReplaySubject 580 | ReplaySubject keeps a buffer of events that it 'replays' to each new subscriber, first he receives a batch of missed 581 | and only later events in real-time. 582 | 583 | ```java 584 | Subject subject = ReplaySubject.createWithSize(50); 585 | 586 | log.info("Pushing 0"); 587 | subject.onNext(0); 588 | log.info("Pushing 1"); 589 | subject.onNext(1); 590 | 591 | log.info("Subscribing 1st"); 592 | subject.subscribe(val -> log.info("Subscriber1 received {}", val), 593 | logError(), logComplete()); 594 | 595 | log.info("Pushing 2"); 596 | subject.onNext(2); 597 | 598 | log.info("Subscribing 2nd"); 599 | subject.subscribe(val -> log.info("Subscriber2 received {}", val), 600 | logError(), logComplete()); 601 | 602 | log.info("Pushing 3"); 603 | subject.onNext(3); 604 | 605 | subject.onComplete(); 606 | 607 | ================== 608 | [main] - Pushing 0 609 | [main] - Pushing 1 610 | [main] - Subscribing 1st 611 | [main] - Subscriber1 received 0 612 | [main] - Subscriber1 received 1 613 | [main] - Pushing 2 614 | [main] - Subscriber1 received 2 615 | [main] - Subscribing 2nd 616 | [main] - Subscriber2 received 0 617 | [main] - Subscriber2 received 1 618 | [main] - Subscriber2 received 2 619 | [main] - Pushing 3 620 | [main] - Subscriber1 received 3 621 | [main] - Subscriber2 received 3 622 | [main] - Subscriber got Completed event 623 | [main] - Subscriber got Completed event 624 | ``` 625 | 626 | ### ConnectableObservable / ConnectableFlowable and resource sharing 627 | There are cases when we want to share a single subscription between subscribers, meaning while the code that executes 628 | on subscribing should be executed once, the events should be published to all subscribers. 629 | 630 | For ex. when we want to share a connection between multiple Observables / Flowables. 631 | Using a plain Observable would just reexecute the code inside _.create()_ and opening / closing a new connection for each 632 | new subscriber when it subscribes / cancels its subscription. 633 | 634 | **ConnectableObservable** are a special kind of **Observable**. No matter how many Subscribers subscribe to ConnectableObservable, 635 | it opens just one subscription to the Observable from which it was created. 636 | 637 | Anyone who subscribes to **ConnectableObservable** is placed in a set of Subscribers(it doesn't trigger 638 | the _.create()_ code a normal Observable would when .subscribe() is called). A **.connect()** method is available for ConnectableObservable. 639 | **As long as connect() is not called, these Subscribers are put on hold, they never directly subscribe to upstream Observable** 640 | 641 | ```java 642 | ConnectableObservable connectableObservable = 643 | Observable.create(subscriber -> { 644 | log.info("Inside create()"); 645 | 646 | /* A JMS connection listener example 647 | Just an example of a costly operation that is better to be shared **/ 648 | 649 | /* Connection connection = connectionFactory.createConnection(); 650 | Session session = connection.createSession(true, AUTO_ACKNOWLEDGE); 651 | MessageConsumer consumer = session.createConsumer(orders); 652 | consumer.setMessageListener(subscriber::onNext); */ 653 | 654 | subscriber.setCancellable(() -> log.info("Subscription cancelled")); 655 | 656 | log.info("Emitting 1"); 657 | subscriber.onNext(1); 658 | 659 | log.info("Emitting 2"); 660 | subscriber.onNext(2); 661 | 662 | subscriber.onComplete(); 663 | }).publish(); 664 | 665 | connectableObservable 666 | .take(1) 667 | .subscribe((val) -> log.info("Subscriber1 received: {}", val), 668 | logError(), logComplete()); 669 | 670 | connectableObservable 671 | .subscribe((val) -> log.info("Subscriber2 received: {}", val), 672 | logError(), logComplete()); 673 | 674 | log.info("Now connecting to the ConnectableObservable"); 675 | connectableObservable.connect(); 676 | 677 | =================== 678 | 679 | ``` 680 | 681 | ### share() operator 682 | Another operator of the ConnectableObservable **.refCount()** allows to do away with having to manually call **.connect()**, 683 | instead it invokes the .create() code when the first Subscriber subscribes while sharing this single subscription with subsequent Subscribers. 684 | This means that **.refCount()** basically keeps a count of references of it's subscribers and subscribes to upstream Observable 685 | (executes the code inside .create() just for the first subscriber), but multicasts the same event to each active subscriber. 686 | When the last subscriber unsubscribes, the ref counter goes from 1 to 0 and triggers any unsubscribe callback associated. 687 | If another Subscriber subscribes after that, counter goes from 0 to 1 and the process starts over again. 688 | 689 | ```java 690 | ConnectableObservable connectableStream = Observable.create(subscriber -> { 691 | log.info("Inside create()"); 692 | 693 | //Simulated MessageListener emits periodically every 500 milliseconds 694 | ResourceConnectionHandler resourceConnectionHandler = new ResourceConnectionHandler() { 695 | @Override 696 | public void onMessage(Integer message) { 697 | log.info("Emitting {}", message); 698 | subscriber.onNext(message); 699 | } 700 | }; 701 | resourceConnectionHandler.openConnection(); 702 | 703 | //when the last subscriber unsubscribes it will invoke disconnect on the resourceConnectionHandler 704 | subscriber.setCancellable(resourceConnectionHandler::disconnect); 705 | }).publish(); 706 | 707 | //publish().refCount() have been joined together in the .share() operator 708 | Observable observable = connectableObservable.refCount(); 709 | 710 | CountDownLatch latch = new CountDownLatch(2); 711 | connectableStream 712 | .take(5) 713 | .subscribe((val) -> log.info("Subscriber1 received: {}", val), 714 | logError(), logComplete(latch)); 715 | 716 | Helpers.sleepMillis(1000); 717 | 718 | log.info("Subscribing 2nd"); 719 | //we're not seing the code inside .create() reexecuted 720 | connectableStream 721 | .take(2) 722 | .subscribe((val) -> log.info("Subscriber2 received: {}", val), 723 | logError(), logComplete(latch)); 724 | 725 | //waiting for the streams to complete 726 | Helpers.wait(latch); 727 | 728 | //subscribing another after previous Subscribers unsubscribed 729 | latch = new CountDownLatch(1); 730 | log.info("Subscribing 3rd"); 731 | observable 732 | .take(1) 733 | .subscribe((val) -> log.info("Subscriber3 received: {}", val), logError(), logComplete(latch)); 734 | 735 | 736 | private abstract class ResourceConnectionHandler { 737 | 738 | ScheduledExecutorService scheduledExecutorService; 739 | 740 | private int counter; 741 | 742 | public void openConnection() { 743 | log.info("**Opening connection"); 744 | 745 | scheduledExecutorService = periodicEventEmitter(() -> { 746 | counter ++; 747 | onMessage(counter); 748 | }, 500, TimeUnit.MILLISECONDS); 749 | } 750 | 751 | public abstract void onMessage(Integer message); 752 | 753 | public void disconnect() { 754 | log.info("**Shutting down connection"); 755 | scheduledExecutorService.shutdown(); 756 | } 757 | } 758 | 759 | =============== 760 | 14:55:23 [main] INFO BaseTestObservables - Inside create() 761 | 14:55:23 [main] INFO BaseTestObservables - **Opening connection 762 | 14:55:23 [pool-1-thread-1] INFO BaseTestObservables - Emitting 1 763 | 14:55:23 [pool-1-thread-1] INFO BaseTestObservables - Subscriber1 received: 1 764 | 14:55:24 [pool-1-thread-1] INFO BaseTestObservables - Emitting 2 765 | 14:55:24 [pool-1-thread-1] INFO BaseTestObservables - Subscriber1 received: 2 766 | 14:55:24 [pool-1-thread-1] INFO BaseTestObservables - Emitting 3 767 | 14:55:24 [pool-1-thread-1] INFO BaseTestObservables - Subscriber1 received: 3 768 | 14:55:24 [main] INFO BaseTestObservables - Subscribing 2nd 769 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Emitting 4 770 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber1 received: 4 771 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber2 received: 4 772 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Emitting 5 773 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber1 received: 5 774 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber got Completed event 775 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber2 received: 5 776 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - **Shutting down connection 777 | 14:55:25 [pool-1-thread-1] INFO BaseTestObservables - Subscriber got Completed event 778 | 14:55:25 [main] INFO BaseTestObservables - Subscribing 3rd 779 | 14:55:25 [main] INFO BaseTestObservables - Inside create() 780 | 14:55:25 [main] INFO BaseTestObservables - **Opening connection 781 | 14:55:25 [pool-2-thread-1] INFO BaseTestObservables - Emitting 1 782 | 14:55:25 [pool-2-thread-1] INFO BaseTestObservables - Subscriber3 received: 1 783 | 14:55:25 [pool-2-thread-1] INFO BaseTestObservables - **Shutting down connection 784 | 14:55:25 [pool-2-thread-1] INFO BaseTestObservables - Subscriber got Completed event 785 | ``` 786 | The **share()** operator of Observable / Flowable is an operator which basically does **publish().refCount()**. 787 | 788 | ## Schedulers 789 | RxJava provides some high level concepts for concurrent execution, like ExecutorService we're not dealing 790 | with the low level constructs like creating the Threads ourselves. Instead we're using a **Scheduler** which create 791 | Workers who are responsible for scheduling and running code. By default RxJava will not introduce concurrency 792 | and will run the operations on the subscription thread. 793 | 794 | There are two methods through which we can introduce Schedulers into our chain of operations: 795 | 796 | - **subscribeOn** allows to specify which Scheduler invokes the code contained in the lambda code for Observable.create() 797 | - **observeOn** allows control to which Scheduler executes the code in the downstream operators 798 | 799 | RxJava provides some general use Schedulers: 800 | 801 | - **Schedulers.computation()** - to be used for CPU intensive tasks. A threadpool. Should not be used for tasks involving blocking IO. 802 | - **Schedulers.io()** - to be used for IO bound tasks 803 | - **Schedulers.from(Executor)** - custom ExecutorService 804 | - **Schedulers.newThread()** - always creates a new thread when a worker is needed. Since it's not thread pooled and 805 | always creates a new thread instead of reusing one, this scheduler is not very useful 806 | 807 | Although we said by default RxJava doesn't introduce concurrency. Notice how we are not doing anything on another thread 808 | than the subscribing thread 'main' and the Test doesn't end until the complete event is processed: 809 | ```java 810 | @Test 811 | public void byDefaultRxJavaDoesntIntroduceConcurrency() { 812 | log.info("Starting"); 813 | 814 | Observable.create(subscriber -> { 815 | log.info("Someone subscribed"); 816 | subscriber.onNext(1); 817 | subscriber.onNext(2); 818 | 819 | subscriber.onComplete(); 820 | }) 821 | .map(val -> { 822 | log.info("Mapping {}", val); 823 | //what if we do some Thread.sleep here 824 | //Thread.sleep(2000); 825 | return val * 10; 826 | }) 827 | .subscribe(logNext()); 828 | } 829 | =============== 830 | 11:23:49 [main] INFO BaseTestObservables - Starting 831 | 11:23:50 [main] INFO BaseTestObservables - Someone subscribed 832 | 11:23:50 [main] INFO BaseTestObservables - Mapping 1 833 | 11:23:50 [main] INFO BaseTestObservables - Subscriber received: 10 834 | 11:23:50 [main] INFO BaseTestObservables - Mapping 2 835 | 11:23:50 [main] INFO BaseTestObservables - Subscriber received: 20 836 | ``` 837 | now let's enable that _Thread.sleep(2000)_ above. 838 | ``` 839 | 11:42:12 [main] INFO BaseTestObservables - Starting 840 | 11:42:12 [main] INFO BaseTestObservables - Someone subscribed 841 | 11:42:12 [main] INFO BaseTestObservables - Mapping 1 842 | 11:42:14 [main] INFO BaseTestObservables - Subscriber received: 10 843 | 11:42:14 [main] INFO BaseTestObservables - Mapping 2 844 | 11:42:16 [main] INFO BaseTestObservables - Subscriber received: 20 845 | ``` 846 | as expected nothing changes, just that we receive the events in the Subscriber delayed by 2 secs. 847 | To prevent this, lots of RxJava operators that involve waiting as **delay**,**interval**, **zip** run on a Scheduler, otherwise they would just block the subscribing thread. 848 | By default **Schedulers.computation()** is used, but the Scheduler can be passed as a parameter to those methods. 849 | 850 | Ok so how can we provide different threads to run the different parts of the code. 851 | 852 | ### subscribeOn 853 | As stated above **subscribeOn** allows to specify on which Scheduler thread the subscribtion is made - which thread invokes the code contained in the lambda for Observable.create() - 854 | (it's **not** abouth the thread for where the code in **.subscribe((val) -> {...})** gets executed). 855 | Since the operators are lazy and nothing happens until subscription, where the **.subscribeOn()** is called doesn't make any difference. 856 | Also calling **.subscribeOn()** multiple times at different positions doesn't have any effect, only the first **.subscribeOn()** Scheduler is considered. 857 | 858 | 859 | ```java 860 | @Test 861 | public void testSubscribeOn() { 862 | log.info("Starting"); 863 | 864 | Observable observable = Observable.create(subscriber -> { 865 | //code that will execute inside the IO ThreadPool 866 | log.info("Starting slow network op"); 867 | Helpers.sleepMillis(2000); 868 | 869 | log.info("Emitting 1st"); 870 | subscriber.onNext(1); 871 | 872 | subscriber.onComplete(); 873 | }); 874 | 875 | observable = observable 876 | .subscribeOn(Schedulers.io()) //Specify execution on the IO Scheduler 877 | .map(val -> { 878 | int newValue = val * 10; 879 | log.info("Mapping {} to {}", val, newValue); 880 | return newValue; 881 | }); 882 | 883 | /** Since we are switching the subscription thread we now need to wait 884 | * for the Thread to complete so again we are using the CountDownLatch "trick" to do it. 885 | **/ 886 | CountDownLatch latch = new CountDownLatch(1); 887 | 888 | observable.subscribe( 889 | logNext(), 890 | logError(latch), 891 | logComplete(latch) 892 | ); 893 | 894 | Helpers.wait(latch); 895 | } 896 | 897 | =============== 898 | 13:16:31 [main] INFO BaseTestObservables - Starting 899 | 13:16:31 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Starting slow network op 900 | 13:16:33 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Emitting 1st 901 | 13:16:33 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Mapping 1 to 10 902 | 13:16:33 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Subscriber received: 10 903 | 13:16:33 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Subscriber got Completed event 904 | ``` 905 | Notice how the code and also the flow down the operators like **.map()** is switched to this new Scheduler that was specified. 906 | 907 | 908 | ### observeOn 909 | **observeOn** allows control to which Scheduler executes the code in the downstream operators. 910 | So by using **observeOn()** we changed the Scheduler for the **map** operator, but notice how the last **.observeOn(Schedulers.newThread())** 911 | we also influence the code received by the subscriber, while **.subscribeOn()** just had a part on the code executed before we changed with **.observeOn()** 912 | 913 | ```java 914 | log.info("Starting"); 915 | 916 | Observable observable = 917 | Observable.create(subscriber -> { 918 | //code that will execute inside the IO Scheduler 919 | log.info("Emitting 1st"); 920 | subscriber.onNext(1); 921 | 922 | log.info("Emitting 2nd"); 923 | subscriber.onNext(2); 924 | 925 | subscriber.onComplete(); 926 | }) 927 | .subscribeOn(Schedulers.io()) 928 | .observeOn(Schedulers.computation()) 929 | .map(val -> { 930 | int newValue = val * 10; 931 | log.info("Mapping {} to {}", val, newValue); 932 | return newValue; 933 | }) 934 | .observeOn(Schedulers.newThread()); 935 | 936 | CountDownLatch latch = new CountDownLatch(1); 937 | 938 | observable.subscribe( 939 | logNext(), 940 | logError(latch), 941 | logComplete(latch) 942 | ); 943 | 944 | Helpers.wait(latch); 945 | 946 | =============== 947 | 19:35:01 [main] INFO BaseTestObservables - Starting 948 | 19:35:01 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Started emitting 949 | 19:35:01 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Emitting 1st 950 | 19:35:01 [RxCachedThreadScheduler-1] INFO BaseTestObservables - Emitting 2nd 951 | 19:35:01 [RxComputationThreadPool-1] INFO BaseTestObservables - Mapping 1 to 10 952 | 19:35:01 [RxNewThreadScheduler-1] INFO BaseTestObservables - Subscriber received: 10 953 | 19:35:01 [RxComputationThreadPool-1] INFO BaseTestObservables - Mapping 2 to 20 954 | 19:35:01 [RxNewThreadScheduler-1] INFO BaseTestObservables - Subscriber received: 20 955 | 19:35:01 [RxNewThreadScheduler-1] INFO BaseTestObservables - Subscriber got Completed event 956 | ``` 957 | 958 | ### back to blocking world 959 | How about when we want to switch back to a blocking flow. We saw above how we need to explicitly use latching 960 | to keep the [main] thread. Say we're incrementally switching from legacy code and we have a Service method 961 | **Collection\ findUsers()** inside this method we can still be reactive but to the caller of the method we 962 | still need to block until we get all the elements of the Collection. 963 | Using **blockingIterable** will block our Test thread till the Flow completes, waiting 964 | for the events to be emitted(we're sleeping just to show it's not completing by chance). 965 | 966 | ```java 967 | log.info("Starting"); 968 | 969 | Flowable flowable = simpleFlowable() 970 | .subscribeOn(Schedulers.io()) 971 | .subscribeOn(Schedulers.computation()) 972 | .map(val -> { 973 | String newValue = "^^" + val + "^^"; 974 | log.info("Mapping new val {}", newValue); 975 | Helpers.sleepMillis(500); 976 | return newValue; 977 | }); 978 | 979 | Iterable iterable = flowable.blockingIterable(); //this call will block until 980 | //the stream completes 981 | iterable.forEach(val -> log.info("Received {}", val)); 982 | 983 | ========================== 984 | 17:48:13 [RxCachedThreadScheduler-1] - Started emitting 985 | 17:48:13 [RxCachedThreadScheduler-1] - Emitting 1st 986 | 17:48:13 [RxCachedThreadScheduler-1] - Mapping new val ^^1^^ 987 | 17:48:14 [RxCachedThreadScheduler-1] - Emitting 2nd 988 | 17:48:14 [main] - Received ^^1^^ 989 | 17:48:14 [RxCachedThreadScheduler-1] - Mapping new val ^^2^^ 990 | 17:48:14 [main] - Received ^^2^^ 991 | 17:48:14 [main] - Finished blockingIterable 992 | ``` 993 | we can see the events being received back on the **\[main\]** thread. 994 | 995 | ```java 996 | //block until the stream completes or throws an error. 997 | flowable.blockingSubscribe(val -> log.info("Subscriber received {}", val)); 998 | ``` 999 | 1000 | 1001 | ## Flatmap operator 1002 | The flatMap operator is so important and has so many different uses it deserves it's own category to explain it. 1003 | Code at [Part06FlatMapOperator.java](https://github.com/balamaci/rxjava-playground/blob/master/src/test/java/com/balamaci/rx/Part06FlatMapOperator.java) 1004 | 1005 | I like to think of it as a sort of **fork-join** operation because what flatMap does is it takes individual stream items 1006 | and maps each of them to an Observable(so it creates new Streams from each object) and then 'flattens' the events from 1007 | these Streams back as coming from a single stream. 1008 | 1009 | Why this looks like fork-join because for each element you can fork some jobs that keeps emitting results, 1010 | and these results are emitted back as elements to the subscribers downstream 1011 | 1012 | **Rules of thumb** to consider before getting comfortable with flatMap: 1013 | 1014 | - When you have an 'item' **T** and a method **T -< Flowable<X>**, you need flatMap. Most common example is when you want 1015 | to make a remote call that returns an Observable / Flowable . For ex if you have a stream of customerIds, and downstream you 1016 | want to work with actual Customer objects: 1017 | 1018 | - When you have Observable<Observable<T>>(aka stream of streams) you probably need flatMap. Because flatMap means you are subscribing 1019 | to each substream. 1020 | 1021 | We use a simulated remote call that returns asynchronous events. This is a most common scenario to make a remote call for each stream element, 1022 | (although in non reactive world we're more likely familiar with remote operations returning Lists **T -> List<X>**). 1023 | Our simulated remote operation produces as many events as the length of the color string received as parameter every 200ms, 1024 | so for example **red : red0, red1, red2** 1025 | 1026 | ```java 1027 | private Flowable simulateRemoteOperation(String color) { 1028 | return Flowable.intervalRange(1, color.length(), 0, 200, TimeUnit.MILLISECONDS) 1029 | .map(iteration -> color + iteration); 1030 | } 1031 | ``` 1032 | 1033 | If we have a stream of color names: 1034 | 1035 | ```java 1036 | Flowable colors = Flowable.just("orange", "red", "green") 1037 | ``` 1038 | 1039 | to invoke the remote operation: 1040 | 1041 | ```java 1042 | Flowable colors = Flowable.just("orange", "red", "green") 1043 | .flatMap(colorName -> simulatedRemoteOperation(colorName)); 1044 | 1045 | colors.subscribe(val -> log.info("Subscriber received: {}", val)); 1046 | 1047 | ==== 1048 | 16:44:15 [Thread-0]- Subscriber received: orange0 1049 | 16:44:15 [Thread-2]- Subscriber received: green0 1050 | 16:44:15 [Thread-1]- Subscriber received: red0 1051 | 16:44:15 [Thread-0]- Subscriber received: orange1 1052 | 16:44:15 [Thread-2]- Subscriber received: green1 1053 | 16:44:15 [Thread-1]- Subscriber received: red1 1054 | 16:44:15 [Thread-0]- Subscriber received: orange2 1055 | 16:44:15 [Thread-2]- Subscriber received: green2 1056 | 16:44:15 [Thread-1]- Subscriber received: red2 1057 | 16:44:15 [Thread-0]- Subscriber received: orange3 1058 | 16:44:15 [Thread-2]- Subscriber received: green3 1059 | 16:44:16 [Thread-0]- Subscriber received: orange4 1060 | 16:44:16 [Thread-2]- Subscriber received: green4 1061 | 16:44:16 [Thread-0]- Subscriber received: orange5 1062 | ``` 1063 | 1064 | Notice how the results are coming intertwined(mixed) and it might not be as you expected it.This is because flatMap actually subscribes to it's inner Observables 1065 | returned from 'simulateRemoteOperation'. You can specify the **concurrency level of flatMap** as a parameter. Meaning 1066 | you can say how many of the substreams should be subscribed "concurrently" - after **onComplete** is triggered on the substreams, 1067 | a new substream is subscribed-. 1068 | 1069 | By setting the concurrency to **1** we don't subscribe to other substreams until the current one finishes: 1070 | 1071 | ``` 1072 | Flowable colors = Flowable.just("orange", "red", "green") 1073 | .flatMap(val -> simulateRemoteOperation(val), 1); // 1074 | 1075 | ``` 1076 | 1077 | Notice now there is a sequence from each color before the next one appears 1078 | 1079 | ``` 1080 | 17:15:24 [Thread-0]- Subscriber received: orange0 1081 | 17:15:24 [Thread-0]- Subscriber received: orange1 1082 | 17:15:25 [Thread-0]- Subscriber received: orange2 1083 | 17:15:25 [Thread-0]- Subscriber received: orange3 1084 | 17:15:25 [Thread-0]- Subscriber received: orange4 1085 | 17:15:25 [Thread-0]- Subscriber received: orange5 1086 | 17:15:25 [Thread-1]- Subscriber received: red0 1087 | 17:15:26 [Thread-1]- Subscriber received: red1 1088 | 17:15:26 [Thread-1]- Subscriber received: red2 1089 | 17:15:26 [Thread-2]- Subscriber received: green0 1090 | 17:15:26 [Thread-2]- Subscriber received: green1 1091 | 17:15:26 [Thread-2]- Subscriber received: green2 1092 | 17:15:27 [Thread-2]- Subscriber received: green3 1093 | 17:15:27 [Thread-2]- Subscriber received: green4 1094 | ``` 1095 | 1096 | There is actually an operator which is basically this **flatMap with 1 concurrency called concatMap**. 1097 | 1098 | 1099 | Inside the flatMap we can operate on the substream with the same stream operators 1100 | 1101 | ```java 1102 | Observable> colorsCounted = colors 1103 | .flatMap(colorName -> { 1104 | Observable timer = Observable.interval(2, TimeUnit.SECONDS); 1105 | 1106 | return simulateRemoteOperation(colorName) // <- Still a stream 1107 | .zipWith(timer, (val, timerVal) -> val) 1108 | .count() 1109 | .map(counter -> new Pair<>(colorName, counter)); 1110 | } 1111 | ); 1112 | ``` 1113 | 1114 | We can also use **switchIfEmpty** to provide some values when the original Publisher doesn't return anything, just completes. 1115 | ```java 1116 | Flowable colors = Flowable.just("red", "", "blue") 1117 | .flatMap(colorName -> simulateRemoteOperation(colorName) 1118 | .switchIfEmpty(Flowable.just("NONE"))); 1119 | 1120 | 13:11:02 Subscriber received: red0 1121 | 13:11:02 Subscriber received: red1 1122 | 13:11:02 Subscriber received: red2 1123 | 13:11:03 Subscriber received: NONE 1124 | 13:11:03 Subscriber received: blue0 1125 | 13:11:03 Subscriber received: blue1 1126 | 13:11:03 Subscriber received: blue2 1127 | 13:11:03 Subscriber received: blue3 1128 | 13:11:03 Subscriber got Completed event 1129 | ``` 1130 | 1131 | **flatMapIterable** is just an easy way to pass each of the elements of a collection 1132 | as a stream 1133 | ``` 1134 | Flowable colors = Flowable.just(1) 1135 | .flatMapIterable(it -> generateColors()); 1136 | 1137 | 1138 | private List generateColors() { 1139 | return Arrays.asList("red", "green", "blue"); 1140 | } 1141 | 1142 | ``` 1143 | 1144 | **switchMap** operator also prevents inter-leavings as only one of stream is subscribed at a time, 1145 | but this is controlled from upstream. If a new value comes from upstream, the current subscribed inner-stream 1146 | gets canceled and a new subscription is made for the new value. 1147 | The current stream will remain subscribed as long as there are no new values from upstream. 1148 | ```java 1149 | Flowable colors = Flowable.interval(0,400, TimeUnit.MILLISECONDS) 1150 | .zipWith(Arrays.asList("EUR", "USD", "GBP"), (it, currency) -> currency) 1151 | .doOnNext(ev -> log.info("Emitting {}", ev)) 1152 | .switchMap(currency -> simulateRemoteOperation(currency) 1153 | .doOnSubscribe((subscription) -> log.info("Subscribed new")) 1154 | .doOnCancel(() -> log.info("Unsubscribed {}", currency)) 1155 | ); 1156 | ``` 1157 | 1158 | 17:45:16 [RxComputationThreadPool-1] INFO BaseTestObservables - Emitting EUR 1159 | 17:45:16 [RxComputationThreadPool-1] INFO BaseTestObservables - Subscribed new 1160 | 17:45:16 [RxComputationThreadPool-2] INFO BaseTestObservables - Subscriber received: EUR1 1161 | 17:45:16 [RxComputationThreadPool-2] INFO BaseTestObservables - Subscriber received: EUR2 1162 | 17:45:16 [RxComputationThreadPool-1] INFO BaseTestObservables - Emitting USD 1163 | 17:45:16 [RxComputationThreadPool-1] INFO BaseTestObservables - Unsubscribed EUR 1164 | 17:45:16 [RxComputationThreadPool-1] INFO BaseTestObservables - Subscribed new 1165 | 17:45:16 [RxComputationThreadPool-3] INFO BaseTestObservables - Subscriber received: USD1 1166 | 17:45:16 [RxComputationThreadPool-3] INFO BaseTestObservables - Subscriber received: USD2 1167 | 17:45:17 [RxComputationThreadPool-1] INFO BaseTestObservables - Emitting GBP 1168 | 17:45:17 [RxComputationThreadPool-1] INFO BaseTestObservables - Unsubscribed USD 1169 | 17:45:17 [RxComputationThreadPool-1] INFO BaseTestObservables - Subscribed new 1170 | 17:45:17 [RxComputationThreadPool-3] INFO BaseTestObservables - Subscriber received: USD3 1171 | 17:45:17 [RxComputationThreadPool-4] INFO BaseTestObservables - Subscriber received: GBP1 1172 | 17:45:17 [RxComputationThreadPool-4] INFO BaseTestObservables - Subscriber received: GBP2 1173 | 17:45:17 [RxComputationThreadPool-4] INFO BaseTestObservables - Subscriber received: GBP3 1174 | 17:45:17 [RxComputationThreadPool-4] INFO BaseTestObservables - Subscriber got Completed event 1175 | 1176 | ## Error handling 1177 | Code at [Part08ErrorHandling.java](https://github.com/balamaci/rxjava-playground/blob/master/src/test/java/com/balamaci/rx/Part08ErrorHandling.java) 1178 | 1179 | Exceptions are for exceptional situations. 1180 | The Reactive Streams specification says that **exceptions are terminal operations**. 1181 | That means in case an error occurs, it triggers an unsubscription upstream and the error travels downstream to the Subscriber, invoking the 'onError' handler: 1182 | 1183 | ```java 1184 | Observable colors = Observable.just("green", "blue", "red", "yellow") 1185 | .map(color -> { 1186 | if ("red".equals(color)) { 1187 | throw new RuntimeException("Encountered red"); 1188 | } 1189 | return color + "*"; 1190 | }) 1191 | .map(val -> val + "XXX"); 1192 | 1193 | colors.subscribe( 1194 | val -> log.info("Subscriber received: {}", val), 1195 | exception -> log.error("Subscriber received error '{}'", exception.getMessage()), 1196 | () -> log.info("Subscriber completed") 1197 | ); 1198 | ``` 1199 | 1200 | returns: 1201 | ``` 1202 | 23:30:17 [main] INFO - Subscriber received: green*XXX 1203 | 23:30:17 [main] INFO - Subscriber received: blue*XXX 1204 | 23:30:17 [main] ERROR - Subscriber received error 'Encountered red' 1205 | ``` 1206 | After the map() operator encounters an error it unsubscribes(cancels the subscription) from upstream 1207 | (therefore 'yellow' is not even emitted). The error travels downstream and triggers the error handler in the Subscriber. 1208 | 1209 | 1210 | There are operators to deal with error flow control: 1211 | 1212 | ### onErrorReturn 1213 | 1214 | The 'onErrorReturn' operator replaces an exception with a value: 1215 | 1216 | ```java 1217 | Flowable numbers = Flowable.just("1", "3", "a", "4", "5", "c") 1218 | .doOnCancel(() -> log.info("Subscription canceled")) 1219 | .map(Integer::parseInt) 1220 | .onErrorReturn(0); 1221 | subscribeWithLog(numbers); 1222 | 1223 | ====================== 1224 | Subscriber received: 1 1225 | Subscriber received: 3 1226 | Subscription canceled 1227 | Subscriber received: 0 1228 | Subscriber got Completed event 1229 | ``` 1230 | 1231 | Notice though how **it didn't prevent map() operator from unsubscribing from the Flowable**, but it did 1232 | trigger the normal **onNext** callback instead of **onError** in the subscriber. 1233 | 1234 | 1235 | Let's introduce a more realcase scenario of a simulated remote request that fails whenever it's invoked 1236 | with "red" and "black" color parameters otherwise just add some \*s. 1237 | 1238 | 1239 | ```java 1240 | private Observable simulateRemoteOperation(String color) { 1241 | return Observable.create(subscriber -> { 1242 | if ("red".equals(color)) { 1243 | log.info("Emitting RuntimeException for {}", color); 1244 | throw new RuntimeException("Color red raises exception"); 1245 | } 1246 | if ("black".equals(color)) { 1247 | log.info("Emitting IllegalArgumentException for {}", color); 1248 | throw new IllegalArgumentException("Black is not a color"); 1249 | } 1250 | 1251 | String value = "**" + color + "**"; 1252 | 1253 | log.info("Emitting {}", value); 1254 | subscriber.onNext(value); 1255 | subscriber.onCompleted(); 1256 | }); 1257 | } 1258 | 1259 | Flowable colors = Flowable.just("green", "blue", "red", "white", "blue") 1260 | .flatMap(color -> simulateRemoteOperation(color)) 1261 | .onErrorReturn(throwable -> "-blank-"); 1262 | 1263 | subscribeWithLog(colors); 1264 | 1265 | ============ 1266 | 1267 | 22:15:51 [main] INFO - Emitting **green** 1268 | 22:15:51 [main] INFO - Subscriber received: **green** 1269 | 22:15:51 [main] INFO - Emitting **blue** 1270 | 22:15:51 [main] INFO - Subscriber received: **blue** 1271 | 22:15:51 [main] INFO - Emitting RuntimeException for red 1272 | 22:15:51 [main] INFO - Subscriber received: -blank- 1273 | 22:15:51 [main] INFO - Subscriber got Completed event 1274 | ``` 1275 | flatMap encounters an error when it subscribes to 'red' substreams and thus still unsubscribe from 'colors' 1276 | stream and the remaining colors are not longer emitted 1277 | 1278 | 1279 | ```java 1280 | Flowable colors = Flowable.just("green", "blue", "red", "white", "blue") 1281 | .flatMap(color -> simulateRemoteOperation(color) 1282 | .onErrorReturn(throwable -> "-blank-") 1283 | ); 1284 | ``` 1285 | onErrorReturn() is applied to the flatMap substream and thus translates the exception to a value and so flatMap 1286 | continues on with the other colors after red 1287 | 1288 | returns: 1289 | ``` 1290 | 22:15:51 [main] INFO - Emitting **green** 1291 | 22:15:51 [main] INFO - Subscriber received: **green** 1292 | 22:15:51 [main] INFO - Emitting **blue** 1293 | 22:15:51 [main] INFO - Subscriber received: **blue** 1294 | 22:15:51 [main] INFO - Emitting RuntimeException for red 1295 | 22:15:51 [main] INFO - Subscriber received: -blank- 1296 | 22:15:51 [main] INFO - Emitting **white** 1297 | 22:15:51 [main] INFO - Subscriber received: **white** 1298 | 22:15:51 [main] INFO - Emitting **blue** 1299 | 22:15:51 [main] INFO - Subscriber received: **blue** 1300 | 22:15:51 [main] INFO - Subscriber got Completed event 1301 | ``` 1302 | 1303 | ### onErrorResumeNext 1304 | onErrorResumeNext() returns a stream instead of an exception, useful for example to invoke a fallback 1305 | method that returns an alternate Stream 1306 | 1307 | ```java 1308 | Observable colors = Observable.just("green", "blue", "red", "white", "blue") 1309 | .flatMap(color -> simulateRemoteOperation(color) 1310 | .onErrorResumeNext(th -> { 1311 | if (th instanceof IllegalArgumentException) { 1312 | return Observable.error(new RuntimeException("Fatal, wrong arguments")); 1313 | } 1314 | return fallbackRemoteOperation(); 1315 | }) 1316 | ); 1317 | 1318 | private Observable fallbackRemoteOperation() { 1319 | return Observable.just("blank"); 1320 | } 1321 | ``` 1322 | 1323 | 1324 | 1325 | ## Retrying 1326 | 1327 | ### timeout() 1328 | Timeout operator raises exception when there are no events incoming before it's predecessor in the specified time limit. 1329 | 1330 | ### retry() 1331 | **retry()** - resubscribes in case of exception to the Observable 1332 | 1333 | ```java 1334 | Flowable colors = Flowable.just("red", "blue", "green", "yellow") 1335 | .concatMap(color -> delayedByLengthEmitter(TimeUnit.SECONDS, color) 1336 | //if there are no events flowing in the timeframe 1337 | .timeout(6, TimeUnit.SECONDS) 1338 | .retry(2) 1339 | .onErrorResumeNext(Observable.just("blank")) 1340 | ); 1341 | 1342 | subscribeWithLog(colors.toBlocking()); 1343 | ``` 1344 | 1345 | returns 1346 | ``` 1347 | 12:40:16 [main] INFO - Received red delaying for 3 1348 | 12:40:19 [main] INFO - Subscriber received: red 1349 | 12:40:19 [RxComputationScheduler-2] INFO - Received blue delaying for 4 1350 | 12:40:23 [main] INFO - Subscriber received: blue 1351 | 12:40:23 [RxComputationScheduler-4] INFO - Received green delaying for 5 1352 | 12:40:28 [main] INFO - Subscriber received: green 1353 | 12:40:28 [RxComputationScheduler-6] INFO - Received yellow delaying for 6 1354 | 12:40:34 [RxComputationScheduler-7] INFO - Received yellow delaying for 6 1355 | 12:40:40 [RxComputationScheduler-1] INFO - Received yellow delaying for 6 1356 | 12:40:46 [main] INFO - Subscriber received: blank 1357 | 12:40:46 [main] INFO - Subscriber got Completed event 1358 | ``` 1359 | 1360 | When you want to retry considering the thrown exception type: 1361 | 1362 | ```java 1363 | Observable colors = Observable.just("blue", "red", "black", "yellow") 1364 | .flatMap(colorName -> simulateRemoteOperation(colorName) 1365 | .retry((retryAttempt, exception) -> { 1366 | if (exception instanceof IllegalArgumentException) { 1367 | log.error("{} encountered non retry exception ", colorName); 1368 | return false; 1369 | } 1370 | log.info("Retry attempt {} for {}", retryAttempt, colorName); 1371 | return retryAttempt <= 2; 1372 | }) 1373 | .onErrorResumeNext(Observable.just("generic color")) 1374 | ); 1375 | ``` 1376 | 1377 | ``` 1378 | 13:21:37 [main] INFO - Emitting **blue** 1379 | 13:21:37 [main] INFO - Emitting RuntimeException for red 1380 | 13:21:37 [main] INFO - Retry attempt 1 for red 1381 | 13:21:37 [main] INFO - Emitting RuntimeException for red 1382 | 13:21:37 [main] INFO - Retry attempt 2 for red 1383 | 13:21:37 [main] INFO - Emitting RuntimeException for red 1384 | 13:21:37 [main] INFO - Retry attempt 3 for red 1385 | 13:21:37 [main] INFO - Emitting IllegalArgumentException for black 1386 | 13:21:37 [main] ERROR - black encountered non retry exception 1387 | 13:21:37 [main] INFO - Emitting **yellow** 1388 | 13:21:37 [main] INFO - Subscriber received: **blue** 1389 | 13:21:37 [main] INFO - Subscriber received: generic color 1390 | 13:21:37 [main] INFO - Subscriber received: generic color 1391 | 13:21:37 [main] INFO - Subscriber received: **yellow** 1392 | 13:21:37 [main] INFO - Subscriber got Completed event 1393 | ``` 1394 | 1395 | ### retryWhen 1396 | A more complex retry logic like implementing a backoff strategy in case of exception 1397 | This can be obtained with **retryWhen**(exceptionObservable -> Observable) 1398 | 1399 | retryWhen resubscribes when an event from an Observable is emitted. It receives as parameter an exception stream 1400 | 1401 | we zip the exceptionsStream with a .range() stream to obtain the number of retries, 1402 | however we want to wait a little before retrying so in the zip function we return a delayed event - .timer() 1403 | 1404 | The delay also needs to be subscribed to be effected so we also flatMap 1405 | 1406 | ```java 1407 | Observable colors = Observable.just("blue", "green", "red", "black", "yellow"); 1408 | 1409 | colors.flatMap(colorName -> 1410 | simulateRemoteOperation(colorName) 1411 | .retryWhen(exceptionStream -> exceptionStream 1412 | .zipWith(Observable.range(1, 3), (exc, attempts) -> { 1413 | //don't retry for IllegalArgumentException 1414 | if(exc instanceof IllegalArgumentException) { 1415 | return Observable.error(exc); 1416 | } 1417 | 1418 | if(attempts < 3) { 1419 | return Observable.timer(2 * attempts, TimeUnit.SECONDS); 1420 | } 1421 | return Observable.error(exc); 1422 | }) 1423 | .flatMap(val -> val) 1424 | ) 1425 | .onErrorResumeNext(Observable.just("generic color") 1426 | ) 1427 | ); 1428 | ``` 1429 | 1430 | ``` 1431 | 15:20:23 [main] INFO - Emitting **blue** 1432 | 15:20:23 [main] INFO - Emitting **green** 1433 | 15:20:23 [main] INFO - Emitting RuntimeException for red 1434 | 15:20:23 [main] INFO - Emitting IllegalArgumentException for black 1435 | 15:20:23 [main] INFO - Emitting **yellow** 1436 | 15:20:23 [main] INFO - Subscriber received: **blue** 1437 | 15:20:23 [main] INFO - Subscriber received: **green** 1438 | 15:20:23 [main] INFO - Subscriber received: generic color 1439 | 15:20:23 [main] INFO - Subscriber received: **yellow** 1440 | 15:20:25 [RxComputationScheduler-1] INFO - Emitting RuntimeException for red 1441 | 15:20:29 [RxComputationScheduler-2] INFO - Emitting RuntimeException for red 1442 | 15:20:29 [main] INFO - Subscriber received: generic color 1443 | 15:20:29 [main] INFO - Subscriber got Completed event 1444 | ``` 1445 | 1446 | **retryWhen vs repeatWhen** 1447 | With similar names it worth noting the difference. 1448 | 1449 | - repeat() resubscribes when it receives onCompleted(). 1450 | - retry() resubscribes when it receives onError(). 1451 | 1452 | Example using repeatWhen() to implement periodic polling 1453 | ```java 1454 | remoteOperation.repeatWhen(completed -> completed 1455 | .delay(2, TimeUnit.SECONDS)) 1456 | ``` 1457 | 1458 | ## Backpressure 1459 | 1460 | It can be the case of a slow consumer that cannot keep up with the producer that is producing too many events 1461 | that the subscriber cannot process. 1462 | 1463 | Backpressure relates to a feedback mechanism through which the subscriber can signal to the producer how much data 1464 | it can consume and so to produce only that amount. 1465 | 1466 | The [reactive-streams](https://github.com/reactive-streams/reactive-streams-jvm) section above we saw that besides the 1467 | **onNext, onError** and **onComplete** handlers, the Subscriber 1468 | has an **onSubscribe(Subscription)**, Subscription through which it can signal upstream it's ready to receive a number 1469 | of items and after it processes the items request another batch. 1470 | 1471 | 1472 | ```java 1473 | public interface Subscriber { 1474 | //signals to the Publisher to start sending events 1475 | public void onSubscribe(Subscription s); 1476 | 1477 | public void onNext(T t); 1478 | public void onError(Throwable t); 1479 | public void onComplete(); 1480 | } 1481 | ``` 1482 | 1483 | The methods exposed by **Subscription** through which the subscriber comunicates with the upstream: 1484 | 1485 | ```java 1486 | public interface Subscription { 1487 | public void request(long n); //request n items 1488 | public void cancel(); 1489 | } 1490 | ``` 1491 | 1492 | So in theory the Subscriber can prevent being overloaded by requesting an initial number of items. The Publisher would 1493 | send those items downstream and not produce any more, until the Subscriber would request more. We say in theory because 1494 | until now we did not see a custom **onSubscribe(Subscription)** request being implemented. This is because if not specified explicitly, 1495 | there is a default implementation which requests of **Long.MAX_VALUE** which basically means "send all you have". 1496 | 1497 | Neither did we see the code in the producer that takes consideration of the number of items requested by the subscriber. 1498 | 1499 | ```java 1500 | Flowable.create(subscriber -> { 1501 | log.info("Started emitting"); 1502 | 1503 | for(int i=0; i < 300; i++) { 1504 | if(subscriber.isCanceled()) { 1505 | return; 1506 | } 1507 | log.info("Emitting {}", i); 1508 | subscriber.next(i); 1509 | } 1510 | 1511 | subscriber.complete(); 1512 | }, BackpressureStrategy.BUFFER); //BackpressureStrategy will be explained further bellow 1513 | ``` 1514 | Looks like it's not possible to slow down production based on request(as there is no reference to the requested items), 1515 | we can at most stop production if the subscriber canceled subscription. 1516 | 1517 | This can be done if we extend Flowable so we can pass our custom Subscription type to the downstream subscriber: 1518 | 1519 | ```java 1520 | private class CustomRangeFlowable extends Flowable { 1521 | 1522 | private int startFrom; 1523 | private int count; 1524 | 1525 | CustomRangeFlowable(int startFrom, int count) { 1526 | this.startFrom = startFrom; 1527 | this.count = count; 1528 | } 1529 | 1530 | @Override 1531 | public void subscribeActual(Subscriber subscriber) { 1532 | subscriber.onSubscribe(new CustomRangeSubscription(startFrom, count, subscriber)); 1533 | } 1534 | 1535 | class CustomRangeSubscription implements Subscription { 1536 | 1537 | volatile boolean cancelled; 1538 | boolean completed = false; 1539 | 1540 | private int count; 1541 | private int currentCount; 1542 | private int startFrom; 1543 | 1544 | private Subscriber actualSubscriber; 1545 | 1546 | CustomRangeSubscription(int startFrom, int count, Subscriber actualSubscriber) { 1547 | this.count = count; 1548 | this.startFrom = startFrom; 1549 | this.actualSubscriber = actualSubscriber; 1550 | } 1551 | 1552 | @Override 1553 | public void request(long items) { 1554 | log.info("Downstream requests {} items", items); 1555 | for(int i=0; i < items; i++) { 1556 | if(cancelled || completed) { 1557 | return; 1558 | } 1559 | 1560 | if(currentCount == count) { 1561 | completed = true; 1562 | if(cancelled) { 1563 | return; 1564 | } 1565 | 1566 | actualSubscriber.onComplete(); 1567 | return; 1568 | } 1569 | 1570 | int emitVal = startFrom + currentCount; 1571 | currentCount++; 1572 | actualSubscriber.onNext(emitVal); 1573 | } 1574 | } 1575 | 1576 | @Override 1577 | public void cancel() { 1578 | cancelled = true; 1579 | } 1580 | } 1581 | } 1582 | ``` 1583 | Now lets see how we can custom control how many items we request from upstream, to simulate an initial big request, 1584 | and then a request for other smaller batches of items as soon as the subscriber finishes and is ready for another batch. 1585 | 1586 | ```java 1587 | Flowable flowable = new CustomRangeFlowable(5, 10); 1588 | 1589 | flowable.subscribe(new Subscriber() { 1590 | 1591 | private Subscription subscription; 1592 | private int backlogItems; 1593 | 1594 | private final int BATCH = 2; 1595 | private final int INITIAL_REQ = 5; 1596 | 1597 | @Override 1598 | public void onSubscribe(Subscription subscription) { 1599 | this.subscription = subscription; 1600 | backlogItems = INITIAL_REQ; 1601 | 1602 | log.info("Initial request {}", backlogItems); 1603 | subscription.request(backlogItems); 1604 | } 1605 | 1606 | @Override 1607 | public void onNext(Integer val) { 1608 | log.info("Subscriber received {}", val); 1609 | backlogItems --; 1610 | 1611 | if(backlogItems == 0) { 1612 | backlogItems = BATCH; 1613 | subscription.request(BATCH); 1614 | } 1615 | } 1616 | 1617 | @Override 1618 | public void onError(Throwable throwable) { 1619 | log.info("Subscriber encountered error"); 1620 | } 1621 | 1622 | @Override 1623 | public void onComplete() { 1624 | log.info("Subscriber completed"); 1625 | } 1626 | }); 1627 | ===================== 1628 | Initial request 5 1629 | Downstream requests 5 items 1630 | Subscriber received 5 1631 | Subscriber received 6 1632 | Subscriber received 7 1633 | Subscriber received 8 1634 | Subscriber received 9 1635 | Downstream requests 2 items 1636 | Subscriber received 10 1637 | Subscriber received 11 1638 | Downstream requests 2 items 1639 | Subscriber received 12 1640 | Subscriber received 13 1641 | Downstream requests 2 items 1642 | Subscriber received 14 1643 | Subscriber completed 1644 | ``` 1645 | 1646 | Returning to the _Flowable.create()_ example since it's not taking any account of the requested 1647 | items by the subscriber, does it mean it might overwhelm a slow Subscriber? 1648 | 1649 | ```java 1650 | private Flowable createFlowable(int items, 1651 | BackpressureStrategy backpressureStrategy) { 1652 | 1653 | return Flowable.create(subscriber -> { 1654 | log.info("Started emitting"); 1655 | 1656 | for (int i = 0; i < items; i++) { 1657 | if(subscriber.isCancelled()) { 1658 | return; 1659 | } 1660 | 1661 | log.info("Emitting {}", i); 1662 | subscriber.onNext(i); 1663 | } 1664 | 1665 | subscriber.onComplete(); 1666 | }, backpressureStrategy); //can be BackpressureStrategy.DROP, BUFFER, LATEST,.. 1667 | ``` 1668 | This is where the 2nd parameter _BackpressureStrategy_ comes in that allows you to specify what to do 1669 | in the case. 1670 | 1671 | - BackpressureStrategy.BUFFER buffer in memory the events that overflow. Of course is we don't drop over some threshold, it might lead to OufOfMemory. 1672 | - BackpressureStrategy.DROP just drop the overflowing events 1673 | - BackpressureStrategy.LATEST keep only recent event and discards previous unconsumed events. 1674 | - BackpressureStrategy.ERROR we get an error in the subscriber immediately 1675 | - BackpressureStrategy.MISSING means we don't care about backpressure(we let one of the downstream operators 1676 | onBackpressureXXX handle it -explained further down-) 1677 | 1678 | 1679 | Still what does it mean to 'overwhelm' the subscriber? 1680 | It means to emit more items than requested by downstream subscriber. 1681 | But we said that by default the subscriber requests Long.MAX_VALUE since the code 1682 | **flowable.subscribe(onNext(), onError, onComplete)** uses a default **onSubscribe**: 1683 | ``` 1684 | (subscription) -> subscription.request(Long.MAX_VALUE); 1685 | ``` 1686 | 1687 | so unless we override it like in our custom Subscriber above, it means it would never overflow. But between the Publisher and the Subscriber you'd have a series of operators. 1688 | When we subscribe, a Subscriber travels up through all operators to the original Publisher and some operators override 1689 | the requested items upstream. One such operator is **observeOn**() which makes it's own request to the upstream Publisher(256 by default), 1690 | but can take a parameter to specify the request size. 1691 | 1692 | ``` 1693 | Flowable flowable = createFlowable(5, BackpressureStrategy.DROP) 1694 | .observeOn(Schedulers.io(), false, 3); 1695 | flowable.subscribe((val) -> { 1696 | log.info("Subscriber received: {}", val); 1697 | Helpers.sleepMillis(millis); 1698 | }, logError(), logComplete()); 1699 | ====== 1700 | [main] - Started emitting 1701 | [main] - Emitting 0 1702 | [main] - Emitting 1 1703 | [main] - Emitting 2 1704 | [main] - Emitting 3 1705 | [main] - Emitting 4 1706 | [RxCachedThreadScheduler-1] - Subscriber received: 0 1707 | [RxCachedThreadScheduler-1] - Subscriber received: 1 1708 | [RxCachedThreadScheduler-1] - Subscriber received: 2 1709 | [RxCachedThreadScheduler-1] - Subscriber got Completed event 1710 | ``` 1711 | This is expected, as the subscription travels upstream through the operators to the source Flowable, while initially 1712 | the Subscriber requesting Long.MAX_VALUE from the upstream operator **observeOn**, which in turn subscribes to the source and it requests just 3 items from the source instead. 1713 | Since we used **BackpressureStrategy.DROP** all the items emitted outside the expected 3, get discarded and thus never reach our subscriber. 1714 | 1715 | You may wonder what would have happened if we didn't use **observeOn**. We had to use it if we wanted to be able 1716 | to produce faster than the subscriber(it wasn't just to show a limited request operator), because we'd need a 1717 | separate thread to produce events faster than the subscriber processes them. 1718 | 1719 | Also you can transform an Observable to Flowable by specifying a BackpressureStrategy, otherwise Observables 1720 | just throw exception on overflowing(same as using BackpressureStrategy.DROP in Flowable.create()). 1721 | ``` 1722 | Flowable flowable = observable.toFlowable(BackpressureStrategy.DROP) 1723 | ``` 1724 | so can a hot Publisher be converted to a Flowable: 1725 | ``` 1726 | PublishSubject subject = PublishSubject.create(); 1727 | 1728 | Flowable flowable = subject 1729 | .toFlowable(BackpressureStrategy.DROP) 1730 | ``` 1731 | 1732 | There are also specialized operators to handle backpressure the onBackpressureXXX operators: **onBackpressureBuffer**, 1733 | **onBackpressureDrop**, **onBackpressureLatest** 1734 | 1735 | These operators request Long.MAX_VALUE(unbounded amount) from upstream and then take it upon themselves to manage the 1736 | requests from downstream. 1737 | In the case of _onBackpressureBuffer_ it adds in an internal queue and send downstream the events as requested, 1738 | _onBackpressureDrop_ just discards events that are received from upstream more than requested from downstream, 1739 | _onBackpressureLatest_ also drops emitted events excluding the last emitted event(most recent). 1740 | 1741 | ```java 1742 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 1743 | .onBackpressureBuffer(5, () -> log.info("Buffer has overflown")); 1744 | 1745 | flowable = flowable 1746 | .observeOn(Schedulers.io(), false, 3); 1747 | subscribeWithSlowSubscriber(flowable); 1748 | 1749 | ===== 1750 | [main] - Started emitting 1751 | [main] - Emitting 0 1752 | [main] - Emitting 1 1753 | [RxCachedThreadScheduler-1] - Subscriber received: 0 1754 | [main] - Emitting 2 1755 | [main] - Emitting 3 1756 | [main] - Emitting 4 1757 | [main] - Emitting 5 1758 | [main] - Emitting 6 1759 | [main] - Emitting 7 1760 | [main] - Emitting 8 1761 | [main] - Emitting 9 1762 | [main] - Buffer has overflown 1763 | [RxCachedThreadScheduler-1] ERROR - Subscriber received error 'Buffer is full' 1764 | ``` 1765 | 1766 | We create the Flowable with _BackpressureStrategy.MISSING_ saying we don't care about backpressure 1767 | but let one of the onBackpressureXXX operators handle it. 1768 | Notice however 1769 | 1770 | 1771 | 1772 | Chaining together multiple onBackpressureXXX operators doesn't actually make sense 1773 | Using something like 1774 | 1775 | ```java 1776 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 1777 | .onBackpressureBuffer(5) 1778 | .onBackpressureDrop((val) -> log.info("Dropping {}", val)) 1779 | flowable = flowable 1780 | .observeOn(Schedulers.io(), false, 3); 1781 | 1782 | subscribeWithSlowSubscriber(flowable); 1783 | ``` 1784 | 1785 | is not behaving as probably you'd expected - buffer 5 values, and then dropping overflowing events-. 1786 | Because _onBackpressureDrop_ subscribes to the previous _onBackpressureBuffer_ operator 1787 | signaling it's requesting **Long.MAX_VALUE**(unbounded amount) from it. 1788 | Thus **onBackpressureBuffer** will never feel its subscriber is overwhelmed and never "trigger", meaning that the last 1789 | onBackpressureXXX operator overrides the previous one if they are chained. 1790 | 1791 | Of course for implementing an event dropping strategy after a full buffer, there is the special overrided 1792 | version of **onBackpressureBuffer** that takes a **BackpressureOverflowStrategy**. 1793 | 1794 | ```java 1795 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 1796 | .onBackpressureBuffer(5, () -> log.info("Buffer has overflown"), 1797 | BackpressureOverflowStrategy.DROP_OLDEST); 1798 | 1799 | flowable = flowable 1800 | .observeOn(Schedulers.io(), false, 3); 1801 | 1802 | subscribeWithSlowSubscriber(flowable); 1803 | 1804 | =============== 1805 | [main] - Started emitting 1806 | [main] - Emitting 0 1807 | [main] - Emitting 1 1808 | [RxCachedThreadScheduler-1] - Subscriber received: 0 1809 | [main] - Emitting 2 1810 | [main] - Emitting 3 1811 | [main] - Emitting 4 1812 | [main] - Emitting 5 1813 | [main] - Emitting 6 1814 | [main] - Emitting 7 1815 | [main] - Emitting 8 1816 | [main] - Buffer has overflown 1817 | [main] - Emitting 9 1818 | [main] - Buffer has overflown 1819 | [RxCachedThreadScheduler-1] - Subscriber received: 1 1820 | [RxCachedThreadScheduler-1] - Subscriber received: 2 1821 | [RxCachedThreadScheduler-1] - Subscriber received: 5 1822 | [RxCachedThreadScheduler-1] - Subscriber received: 6 1823 | [RxCachedThreadScheduler-1] - Subscriber received: 7 1824 | [RxCachedThreadScheduler-1] - Subscriber received: 8 1825 | [RxCachedThreadScheduler-1] - Subscriber received: 9 1826 | [RxCachedThreadScheduler-1] - Subscriber got Completed event 1827 | ``` 1828 | 1829 | onBackpressureXXX operators can be added whenever necessary and it's not limited to cold publishers and we can use them 1830 | on hot publishers also. 1831 | 1832 | ## Articles and books for further reading 1833 | [Reactive Programming with RxJava](http://shop.oreilly.com/product/0636920042228.do) 1834 | 1835 | 1836 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-cayman -------------------------------------------------------------------------------- /pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 5 | 4.0.0 6 | 7 | com.balamaci 8 | rxjava-playground 9 | 1.0-SNAPSHOT 10 | RxJava Playground 11 | RxJava Playground - Test scenarios describing RxJava functionality 12 | 13 | 14 | 2.2.3 15 | 1.7.25 16 | 17 | 18 | 19 | 20 | 21 | io.reactivex.rxjava2 22 | rxjava 23 | ${rxjava.version} 24 | 25 | 26 | 27 | 28 | org.slf4j 29 | slf4j-api 30 | ${slf4j.version} 31 | 32 | 33 | 34 | org.slf4j 35 | slf4j-simple 36 | ${slf4j.version} 37 | 38 | 39 | 40 | junit 41 | junit 42 | 4.12 43 | test 44 | 45 | 46 | 47 | 48 | io.vavr 49 | vavr 50 | 0.9.2 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | true 59 | org.apache.maven.plugins 60 | maven-compiler-plugin 61 | 3.1 62 | 63 | 1.8 64 | 1.8 65 | UTF-8 66 | true 67 | true 68 | 69 | 70 | 71 | 72 | 73 | 74 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/BaseTestObservables.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.BackpressureStrategy; 5 | import io.reactivex.Flowable; 6 | import io.reactivex.Observable; 7 | import io.reactivex.Single; 8 | import io.reactivex.functions.Action; 9 | import io.reactivex.functions.Consumer; 10 | import org.slf4j.Logger; 11 | import org.slf4j.LoggerFactory; 12 | 13 | import java.util.concurrent.CountDownLatch; 14 | import java.util.concurrent.TimeUnit; 15 | 16 | /** 17 | * @author sbalamaci 18 | */ 19 | public interface BaseTestObservables { 20 | 21 | Logger log = LoggerFactory.getLogger(BaseTestObservables.class); 22 | 23 | default Flowable simpleFlowable() { 24 | return Flowable.create(subscriber -> { 25 | log.info("Started emitting"); 26 | 27 | log.info("Emitting 1st"); 28 | subscriber.onNext(1); 29 | 30 | log.info("Emitting 2nd"); 31 | subscriber.onNext(2); 32 | 33 | subscriber.onComplete(); 34 | }, BackpressureStrategy.BUFFER); 35 | } 36 | 37 | default void subscribeWithLog(Flowable flowable) { 38 | flowable.subscribe( 39 | logNext(), 40 | logError(), 41 | logComplete() 42 | ); 43 | } 44 | 45 | default void subscribeWithLog(Observable observable) { 46 | observable.subscribe( 47 | logNext(), 48 | logError(), 49 | logComplete() 50 | ); 51 | } 52 | 53 | default void subscribeWithLogOutputWaitingForComplete(Observable observable) { 54 | CountDownLatch latch = new CountDownLatch(1); 55 | 56 | observable.subscribe( 57 | logNext(), 58 | logError(latch), 59 | logComplete(latch) 60 | ); 61 | 62 | Helpers.wait(latch); 63 | } 64 | 65 | default void subscribeWithLog(Single single) { 66 | single.subscribe( 67 | val -> log.info("Subscriber received: {}", val), 68 | logError() 69 | ); 70 | } 71 | 72 | default void subscribeWithLogOutputWaitingForComplete(Flowable flowable) { 73 | CountDownLatch latch = new CountDownLatch(1); 74 | 75 | flowable.subscribe( 76 | logNext(), 77 | logError(latch), 78 | logComplete(latch) 79 | ); 80 | 81 | Helpers.wait(latch); 82 | } 83 | 84 | default void subscribeWithLogOutputWaitingForComplete(Single single) { 85 | CountDownLatch latch = new CountDownLatch(1); 86 | 87 | single.subscribe( 88 | val -> { 89 | log.info("Subscriber received: {} and completed", val); 90 | latch.countDown(); 91 | }, 92 | logError(latch) 93 | ); 94 | 95 | Helpers.wait(latch); 96 | } 97 | 98 | default Flowable periodicEmitter(T t1, T t2, T t3, int interval, TimeUnit unit) { 99 | return periodicEmitter(t1, t2, t3, interval, unit, interval); 100 | } 101 | 102 | default Flowable periodicEmitter(T t1, T t2, T t3, int interval, 103 | TimeUnit unit, int initialDelay) { 104 | Flowable itemsStream = Flowable.just(t1, t2, t3); 105 | Flowable timer = Flowable.interval(initialDelay, interval, unit); 106 | 107 | return Flowable.zip(itemsStream, timer, (key, val) -> key); 108 | } 109 | 110 | default Observable periodicEmitter(T[] items, int interval, 111 | TimeUnit unit, int initialDelay) { 112 | Observable itemsStream = Observable.fromArray(items); 113 | Observable timer = Observable.interval(initialDelay, interval, unit); 114 | 115 | return Observable.zip(itemsStream, timer, (key, val) -> key); 116 | } 117 | 118 | default Observable periodicEmitter(T[] items, int interval, 119 | TimeUnit unit) { 120 | return periodicEmitter(items, interval, unit); 121 | } 122 | 123 | default Flowable delayedByLengthEmitter(TimeUnit unit, String...items) { 124 | Flowable itemsStream = Flowable.fromArray(items); 125 | 126 | return itemsStream.concatMap(item -> Flowable.just(item) 127 | .doOnNext(val -> log.info("Received {} delaying for {} ", val, val.length())) 128 | .delay(item.length(), unit) 129 | ); 130 | } 131 | 132 | default Consumer logNext() { 133 | return (Consumer) val -> log.info("Subscriber received: {}", val); 134 | } 135 | 136 | default Consumer logNextAndSlowByMillis(int millis) { 137 | return (Consumer) val -> { 138 | log.info("Subscriber received: {}", val); 139 | Helpers.sleepMillis(millis); 140 | }; 141 | } 142 | 143 | default Consumer logError() { 144 | return err -> log.error("Subscriber received error '{}'", err.getMessage()); 145 | } 146 | 147 | default Consumer logError(CountDownLatch latch) { 148 | return err -> { 149 | log.error("Subscriber received error '{}'", err.getMessage()); 150 | latch.countDown(); 151 | }; 152 | } 153 | 154 | default Action logComplete() { 155 | return () -> log.info("Subscriber got Completed event"); 156 | } 157 | 158 | default Action logComplete(CountDownLatch latch) { 159 | return () -> { 160 | log.info("Subscriber got Completed event"); 161 | latch.countDown(); 162 | }; 163 | } 164 | 165 | } 166 | 167 | 168 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part01CreateFlowable.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.BackpressureStrategy; 5 | import io.reactivex.Flowable; 6 | import io.reactivex.Observable; 7 | import io.reactivex.Single; 8 | import io.reactivex.disposables.Disposable; 9 | import org.junit.Test; 10 | 11 | import java.util.Arrays; 12 | import java.util.concurrent.CompletableFuture; 13 | 14 | /** 15 | * 16 | * 17 | * @author sbalamaci 18 | */ 19 | public class Part01CreateFlowable implements BaseTestObservables { 20 | 21 | 22 | @Test 23 | public void just() { 24 | Flowable flowable = Flowable.just(1, 5, 10); 25 | 26 | flowable.subscribe( 27 | val -> log.info("Subscriber received: {}", val)); 28 | } 29 | 30 | @Test 31 | public void range() { 32 | Flowable flowable = Flowable.range(1, 10); 33 | 34 | flowable.subscribe( 35 | val -> log.info("Subscriber received: {}", val)); 36 | } 37 | 38 | @Test 39 | public void fromArray() { 40 | Flowable flowable = Flowable.fromArray(new String[]{"red", "green", "blue", "black"}); 41 | 42 | flowable.subscribe( 43 | val -> log.info("Subscriber received: {}")); 44 | } 45 | 46 | @Test 47 | public void fromIterable() { 48 | Flowable flowable = Flowable.fromIterable(Arrays.asList("red", "green", "blue")); 49 | 50 | flowable.subscribe( 51 | val -> log.info("Subscriber received: {}")); 52 | } 53 | 54 | /** 55 | * We can also create a stream from Future, making easier to switch from legacy code to reactive 56 | */ 57 | @Test 58 | public void fromFuture() { 59 | CompletableFuture completableFuture = CompletableFuture. 60 | supplyAsync(() -> { //starts a background thread the ForkJoin common pool 61 | Helpers.sleepMillis(100); 62 | return "red"; 63 | }); 64 | 65 | Single single = Single.fromFuture(completableFuture); 66 | single.subscribe(val -> log.info("Stream completed successfully : {}", val)); 67 | } 68 | 69 | 70 | 71 | /** 72 | * Using Flowable.create to handle the actual emissions of events with the events like onNext, onComplete, onError 73 | *

74 | * When subscribing to the Flowable / Observable with flowable.subscribe(), the lambda code inside create() gets executed. 75 | * Flowable.subscribe can take 3 handlers for each type of event - onNext, onError and onComplete 76 | *

77 | * When using Observable.create you need to be aware of Backpressure and that Observables based on 'create' method 78 | * are not Backpressure aware {@see Part09BackpressureHandling}. 79 | */ 80 | @Test 81 | public void createSimpleObservable() { 82 | Flowable flowable = Flowable.create(subscriber -> { 83 | log.info("Started emitting"); 84 | 85 | log.info("Emitting 1st"); 86 | subscriber.onNext(1); 87 | 88 | log.info("Emitting 2nd"); 89 | subscriber.onNext(2); 90 | 91 | subscriber.onComplete(); 92 | }, BackpressureStrategy.BUFFER); 93 | 94 | log.info("Subscribing"); 95 | Disposable subscription = flowable.subscribe( 96 | val -> log.info("Subscriber received: {}", val), 97 | err -> log.error("Subscriber received error", err), 98 | () -> log.info("Subscriber got Completed event")); 99 | } 100 | 101 | 102 | /** 103 | * Observable emits an Error event which is a terminal operation and the subscriber is no longer executing 104 | * it's onNext callback. We're actually breaking the the Observable contract that we're still emitting events 105 | * after onComplete or onError have fired. 106 | */ 107 | @Test 108 | public void createSimpleObservableThatEmitsError() { 109 | Observable observable = Observable.create(subscriber -> { 110 | log.info("Started emitting"); 111 | 112 | log.info("Emitting 1st"); 113 | subscriber.onNext(1); 114 | 115 | subscriber.onError(new RuntimeException("Test exception")); 116 | 117 | log.info("Emitting 2nd"); 118 | subscriber.onNext(2); 119 | }); 120 | 121 | Disposable disposable = observable.subscribe( 122 | val -> log.info("Subscriber received: {}", val), 123 | err -> log.error("Subscriber received error", err), 124 | () -> log.info("Subscriber got Completed event") 125 | ); 126 | } 127 | 128 | /** 129 | * Observables are lazy, meaning that the code inside create() doesn't get executed without subscribing to the Observable 130 | * So even if we sleep for a long time inside create() method(to simulate a costly operation), 131 | * without subscribing to this Observable the code is not executed and the method returns immediately. 132 | */ 133 | @Test 134 | public void flowablesAreLazy() { 135 | Observable flowable = Observable.create(subscriber -> { 136 | log.info("Started emitting but sleeping for 5 secs"); //this is not executed 137 | Helpers.sleepMillis(5000); 138 | subscriber.onNext(1); 139 | }); 140 | log.info("Finished"); 141 | } 142 | 143 | /** 144 | * When subscribing to an Observable, the create() method gets executed for each subscription 145 | * this means that the events inside create are re-emitted to each subscriber. So every subscriber will get the 146 | * same events and will not lose any events. 147 | */ 148 | @Test 149 | public void multipleSubscriptionsToSameObservable() { 150 | Observable flowable = Observable.create(subscriber -> { 151 | log.info("Started emitting"); 152 | 153 | log.info("Emitting 1st event"); 154 | subscriber.onNext(1); 155 | 156 | log.info("Emitting 2nd event"); 157 | subscriber.onNext(2); 158 | 159 | subscriber.onComplete(); 160 | }); 161 | 162 | log.info("Subscribing 1st subscriber"); 163 | flowable.subscribe(val -> log.info("First Subscriber received: {}", val)); 164 | 165 | log.info("======================="); 166 | 167 | log.info("Subscribing 2nd subscriber"); 168 | flowable.subscribe(val -> log.info("Second Subscriber received: {}", val)); 169 | } 170 | 171 | /** 172 | * Inside the create() method, we can check is there are still active subscribers to our Observable. 173 | * It's a way to prevent to do extra work(like for ex. querying a datasource for entries) if no one is listening 174 | * In the following example we'd expect to have an infinite stream, but because we stop if there are no active 175 | * subscribers we stop producing events. 176 | * The take() operator unsubscribes from the Observable after it's received the specified amount of events 177 | * while in the same time calling onComplete() downstream. 178 | */ 179 | @Test 180 | public void showUnsubscribeObservable() { 181 | Observable observable = Observable.create(subscriber -> { 182 | int i = 1; 183 | while(true) { 184 | if(subscriber.isDisposed()) { 185 | break; 186 | } 187 | 188 | subscriber.onNext(i++); 189 | } 190 | //subscriber.onCompleted(); too late to emit Complete event since subscriber already unsubscribed 191 | 192 | subscriber.setCancellable(() -> log.info("Subscription canceled")); 193 | }); 194 | 195 | observable 196 | .take(5) 197 | .map(val -> "*" + val + "*") 198 | .subscribe( 199 | val -> log.info("Subscriber received: {}", val), 200 | err -> log.error("Subscriber received error", err), 201 | () -> log.info("Subscriber got Completed event") //The Complete event is triggered by 'take()' operator 202 | ); 203 | } 204 | 205 | 206 | /** 207 | * .defer acts as a factory of Flowables, just when subscribed it actually invokes the logic to create the 208 | * Flowable to be emitted. 209 | * It's an easy way to switch from a blocking method to a reactive Single/Flowable. 210 | * Simply using Flowable.just(blockingOp()) would still block, as Java needs to resolve the parameter when invoking 211 | * Flux.just(param) method, so blockingOp() method would still be invoked(and block). 212 | * 213 | * The solution is to wrap the blockingOp() method inside a lambda that gets passed to .defer(() -> blockingOp()) 214 | * 215 | */ 216 | @Test 217 | public void deferCreateObservable() { 218 | log.info("Starting blocking Flowable"); 219 | Flowable flowableBlocked = Flowable.just((blockingOperation())); 220 | log.info("After blocking Flowable"); 221 | 222 | log.info("Starting defered op"); 223 | Flowable stream = Flowable.defer(() -> Flowable.just(blockingOperation())); 224 | log.info("After defered op"); 225 | 226 | log.info("Sleeping a little before subscribing and executing the defered code"); 227 | Helpers.sleepMillis(2000); 228 | 229 | log.info("Subscribing"); 230 | subscribeWithLogOutputWaitingForComplete(stream); 231 | } 232 | 233 | private String blockingOperation() { 234 | log.info("Blocking 1sec..."); 235 | Helpers.sleepMillis(1000); 236 | log.info("Ended blocking"); 237 | 238 | return "Hello"; 239 | } 240 | 241 | } 242 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part02SimpleOperators.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.Flowable; 5 | import io.reactivex.Single; 6 | import org.junit.Test; 7 | 8 | import java.util.ArrayList; 9 | import java.util.List; 10 | import java.util.Random; 11 | import java.util.concurrent.CountDownLatch; 12 | import java.util.concurrent.TimeUnit; 13 | 14 | /** 15 | * @author sbalamaci 16 | */ 17 | public class Part02SimpleOperators implements BaseTestObservables { 18 | 19 | /** 20 | * Delay operator - the Thread.sleep of the reactive world, it's pausing for a particular increment of time 21 | * before emitting the whole range events which are thus shifted by the specified time amount. 22 | * 23 | * The delay operator uses a Scheduler {@see Part06Schedulers} by default, which actually means it's 24 | * running the operators and the subscribe operations on a different thread, which means the test method 25 | * will terminate before we see the text from the log. That is why we use the CountDownLatch waiting for the 26 | * completion of the stream. 27 | * 28 | */ 29 | @Test 30 | public void delayOperator() { 31 | log.info("Starting"); 32 | 33 | CountDownLatch latch = new CountDownLatch(1); 34 | Flowable.range(0, 2) 35 | .doOnNext(val -> log.info("Emitted {}", val)) 36 | .delay(5, TimeUnit.SECONDS) 37 | .subscribe( 38 | tick -> log.info("Tick {}", tick), 39 | (ex) -> log.info("Error emitted"), 40 | () -> { 41 | log.info("Completed"); 42 | latch.countDown(); 43 | }); 44 | 45 | Helpers.wait(latch); 46 | } 47 | 48 | /** 49 | * Timer operator waits for a specific amount of time before it emits an event and then completes 50 | */ 51 | @Test 52 | public void timerOperator() { 53 | log.info("Starting"); 54 | Flowable flowable = Flowable.timer(5, TimeUnit.SECONDS); 55 | subscribeWithLogOutputWaitingForComplete(flowable); 56 | } 57 | 58 | 59 | 60 | @Test 61 | public void delayOperatorWithVariableDelay() { 62 | log.info("Starting"); 63 | Flowable flowable = Flowable.range(0, 5) 64 | .doOnNext(val -> log.info("Emitted {}", val)) 65 | .delay(val -> Flowable.timer(val * 2, TimeUnit.SECONDS)); 66 | subscribeWithLogOutputWaitingForComplete(flowable); 67 | } 68 | 69 | /** 70 | * Periodically emits a number starting from 0 and then increasing the value on each emission 71 | */ 72 | @Test 73 | public void intervalOperator() { 74 | log.info("Starting"); 75 | Flowable flowable = Flowable.interval(1, TimeUnit.SECONDS) 76 | .take(5); 77 | 78 | subscribeWithLogOutputWaitingForComplete(flowable); 79 | } 80 | 81 | /** 82 | * scan operator - takes an initial value and a function(accumulator, currentValue). It goes through the events 83 | * sequence and combines the current event value with the previous result(accumulator) emitting downstream the 84 | * the function's result for each event(the initial value is used for the first event). 85 | */ 86 | @Test 87 | public void scanOperator() { 88 | Flowable numbers = Flowable.just(3, 5, -2, 9) 89 | .scan(0, (totalSoFar, currentValue) -> { 90 | log.info("totalSoFar={}, emitted={}", totalSoFar, currentValue); 91 | return totalSoFar + currentValue; 92 | }); 93 | 94 | subscribeWithLog(numbers); 95 | } 96 | 97 | /** 98 | * reduce operator acts like the scan operator but it only passes downstream the final result 99 | * (doesn't pass the intermediate results downstream) so the subscriber receives just one event 100 | */ 101 | @Test 102 | public void reduceOperator() { 103 | Single numbers = Flowable.just(3, 5, -2, 9) 104 | .reduce(0, (totalSoFar, val) -> { 105 | log.info("totalSoFar={}, emitted={}", totalSoFar, val); 106 | return totalSoFar + val; 107 | }); 108 | subscribeWithLog(numbers); 109 | } 110 | 111 | /** 112 | * collect operator acts similar to the reduce() operator, but while the reduce() operator uses a reduce function 113 | * which returns a value, the collect() operator takes a container supplie and a function which doesn't return 114 | * anything(a consumer). The mutable container is passed for every event and thus you get a chance to modify it 115 | * in this collect consumer function 116 | */ 117 | @Test 118 | public void collectOperator() { 119 | Single> numbers = Flowable.just(3, 5, -2, 9) 120 | .collect(ArrayList::new, (container, value) -> { 121 | log.info("Adding {} to container", value); 122 | container.add(value); 123 | //notice we don't need to return anything 124 | }); 125 | subscribeWithLog(numbers); 126 | } 127 | 128 | /** 129 | * repeat resubscribes to the observable after it receives onComplete 130 | */ 131 | @Test 132 | public void repeat() { 133 | Flowable random = Flowable.defer(() -> { 134 | Random rand = new Random(); 135 | return Flowable.just(rand.nextInt(20)); 136 | }) 137 | .repeat(5); 138 | 139 | subscribeWithLogOutputWaitingForComplete(random); 140 | } 141 | 142 | } 143 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part03MergingStreams.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import com.balamaci.rx.util.Pair; 5 | import io.reactivex.Flowable; 6 | import io.reactivex.Single; 7 | import org.junit.Test; 8 | 9 | import java.util.concurrent.CompletableFuture; 10 | import java.util.concurrent.TimeUnit; 11 | 12 | /** 13 | * Operators for working with multiple streams 14 | * 15 | * 16 | */ 17 | public class Part03MergingStreams implements BaseTestObservables { 18 | 19 | /** 20 | * Zip operator operates sort of like a zipper in the sense that it takes an event from one stream and waits 21 | * for an event from another other stream. Once an event for the other stream arrives, it uses the zip function 22 | * to merge the two events. 23 | *

24 | * This is an useful scenario when for example you want to make requests to remote services in parallel and 25 | * wait for both responses before continuing. 26 | *

27 | * Zip operator besides the streams to zip, also takes as parameter a function which will produce the 28 | * combined result of the zipped streams once each stream emitted it's value 29 | */ 30 | @Test 31 | public void zipUsedForTakingTheResultOfCombinedAsyncOperations() { 32 | Single isUserBlockedStream = Single.fromFuture(CompletableFuture.supplyAsync(() -> { 33 | Helpers.sleepMillis(200); 34 | return Boolean.FALSE; 35 | })); 36 | Single userCreditScoreStream = Single.fromFuture(CompletableFuture.supplyAsync(() -> { 37 | Helpers.sleepMillis(2300); 38 | return 200; 39 | })); 40 | 41 | Single> userCheckStream = Single.zip(isUserBlockedStream, userCreditScoreStream, 42 | Pair::new); 43 | subscribeWithLogOutputWaitingForComplete(userCheckStream); 44 | } 45 | 46 | /** 47 | * Implementing a periodic emitter, by waiting for a slower stream to emit periodically. 48 | * Since the zip operator need a pair of events, the slow stream will work like a timer by periodically emitting 49 | * with zip setting the pace of emissions downstream. 50 | */ 51 | @Test 52 | public void zipUsedToSlowDownAnotherStream() { 53 | Flowable colors = Flowable.just("red", "green", "blue"); 54 | Flowable timer = Flowable.interval(2, TimeUnit.SECONDS); 55 | 56 | Flowable periodicEmitter = Flowable.zip(colors, timer, (key, val) -> key); 57 | 58 | subscribeWithLogOutputWaitingForComplete(periodicEmitter); 59 | } 60 | 61 | 62 | /** 63 | * Merge operator combines one or more stream and passes events downstream as soon 64 | * as they appear 65 | *

66 | * The subscriber will receive both color strings and numbers from the Observable.interval 67 | * as soon as they are emitted 68 | */ 69 | @Test 70 | public void mergeOperator() { 71 | log.info("Starting"); 72 | 73 | Flowable colors = periodicEmitter("red", "green", "blue", 2, TimeUnit.SECONDS); 74 | 75 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS) 76 | .take(5); 77 | 78 | Flowable flowable = Flowable.merge(colors, numbers); 79 | subscribeWithLogOutputWaitingForComplete(flowable); 80 | } 81 | 82 | /** 83 | * Concat operator appends another streams at the end of another 84 | * The ex. shows that even the 'numbers' streams should start early, the 'colors' stream emits fully its events 85 | * before we see any 'numbers'. 86 | * This is because 'numbers' stream is actually subscribed only after the 'colors' complete. 87 | * Should the second stream be a 'hot' emitter, its events would be lost until the first one finishes 88 | * and the seconds stream is subscribed. 89 | */ 90 | @Test 91 | public void concatStreams() { 92 | log.info("Starting"); 93 | Flowable colors = periodicEmitter("red", "green", "blue", 2, TimeUnit.SECONDS); 94 | 95 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS) 96 | .take(4); 97 | 98 | Flowable observable = Flowable.concat(colors, numbers); 99 | subscribeWithLogOutputWaitingForComplete(observable); 100 | } 101 | 102 | /** 103 | * combineLatest pairs events from multiple streams, but instead of waiting for an event 104 | * from other streams, it uses the last emitted event from that stream 105 | */ 106 | @Test 107 | public void combineLatest() { 108 | log.info("Starting"); 109 | 110 | Flowable colors = periodicEmitter("red", "green", "blue", 3, TimeUnit.SECONDS); 111 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS) 112 | .take(4); 113 | Flowable combinedFlowables = Flowable.combineLatest(colors, numbers, Pair::new); 114 | 115 | subscribeWithLogOutputWaitingForComplete(combinedFlowables); 116 | } 117 | } 118 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part04HotPublishers.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.Observable; 5 | import io.reactivex.observables.ConnectableObservable; 6 | import io.reactivex.subjects.PublishSubject; 7 | import io.reactivex.subjects.ReplaySubject; 8 | import io.reactivex.subjects.Subject; 9 | import org.junit.Test; 10 | 11 | import java.util.concurrent.CountDownLatch; 12 | import java.util.concurrent.Executors; 13 | import java.util.concurrent.ScheduledExecutorService; 14 | import java.util.concurrent.TimeUnit; 15 | 16 | /** 17 | * @author sbalamaci 18 | */ 19 | public class Part04HotPublishers implements BaseTestObservables { 20 | 21 | /** 22 | * We've seen that with 'cold publishers', whenever a subscriber subscribes, each subscriber will get 23 | * it's version of emitted values independently 24 | * 25 | * Subjects keep reference to their subscribers and allow 'multicasting' to them. 26 | * 27 | * for (Disposable s : subscribers.get()) { 28 | * s.onNext(t); 29 | * } 30 | 31 | * 32 | * Subjects besides being traditional Observables you can use the same operators and subscribe to them, 33 | * are also an Observer, meaning you can invoke subject.onNext(value) from different parts in the code, 34 | * which means that you publish events which the Subject will pass on to their subscribers. 35 | * 36 | * Observable.create(subscriber -> { 37 | * subscriber.onNext(val); 38 | * }) 39 | * .map(...) 40 | * .subscribe(...); 41 | * 42 | * With Subjects you can call onNext from different parts of the code: 43 | * Subject subject = ReplaySubject.create() 44 | * .map(...); 45 | * .subscribe(); // 46 | * 47 | * ... 48 | * subject.onNext(val); 49 | * ... 50 | * subject.onNext(val2); 51 | * 52 | * ReplaySubject keeps a buffer of events that it 'replays' to each new subscriber, first he receives a batch of missed 53 | * and only later events in real-time. 54 | * 55 | * PublishSubject - doesn't keep a buffer but instead, meaning if another subscriber subscribes later, it's going to loose events 56 | */ 57 | 58 | @Test 59 | public void replaySubject() { 60 | Subject subject = ReplaySubject.createWithSize(50); 61 | 62 | // Runnable pushAction = pushEventsToSubjectAction(subject, 10); 63 | // periodicEventEmitter(pushAction, 500, TimeUnit.MILLISECONDS); 64 | 65 | pushToSubject(subject, 0); 66 | pushToSubject(subject, 1); 67 | 68 | CountDownLatch latch = new CountDownLatch(2); 69 | log.info("Subscribing 1st"); 70 | subject.subscribe(val -> log.info("Subscriber1 received {}", val), logError(), logComplete(latch)); 71 | 72 | pushToSubject(subject, 2); 73 | 74 | log.info("Subscribing 2nd"); 75 | subject.subscribe(val -> log.info("Subscriber2 received {}", val), logError(), logComplete(latch)); 76 | pushToSubject(subject, 3); 77 | 78 | subject.onComplete(); 79 | 80 | Helpers.wait(latch); 81 | } 82 | 83 | private void pushToSubject(Subject subject, int val) { 84 | log.info("Pushing {}", val); 85 | subject.onNext(val); 86 | } 87 | 88 | @Test 89 | public void publishSubject() { 90 | Subject subject = PublishSubject.create(); 91 | 92 | Helpers.sleepMillis(1000); 93 | log.info("Subscribing 1st"); 94 | 95 | CountDownLatch latch = new CountDownLatch(2); 96 | subject 97 | .subscribe(val -> log.info("Subscriber1 received {}", val), logError(), logComplete(latch)); 98 | 99 | Helpers.sleepMillis(1000); 100 | log.info("Subscribing 2nd"); 101 | subject.subscribe(val -> log.info("Subscriber2 received {}", val), logError(), logComplete(latch)); 102 | Helpers.wait(latch); 103 | } 104 | 105 | 106 | /** 107 | * Because reactive stream specs mandates that events should be ordered(cannot emit downstream two events simultaneously) 108 | * it means that it's illegal to call onNext,onComplete,onError from different threads. 109 | * 110 | * To make this easy there is the .toSerialized() operator that wraps the Subject inside a SerializedSubject 111 | * which basically just calls the onNext,.. methods of the wrapped Subject inside a synchronized block. 112 | */ 113 | @Test 114 | public void callsToSubjectMethodsMustHappenOnSameThread() { 115 | Subject subject = PublishSubject.create(); 116 | 117 | CountDownLatch latch = new CountDownLatch(1); 118 | 119 | Subject serializedSubject = subject.toSerialized(); 120 | subject.subscribe(logNext(), logError(), logComplete(latch)); 121 | 122 | new Thread(() -> serializedSubject.onNext(1), "thread1").start(); 123 | new Thread(() -> serializedSubject.onNext(2), "thread2").start(); 124 | new Thread(() -> serializedSubject.onComplete(), "thread3").start(); 125 | 126 | Helpers.wait(latch); 127 | } 128 | 129 | 130 | /** 131 | * ConnectableObservable is a special kind of Observable that when calling .subscribe() 132 | * it just keeps a reference to its subscribers, it only subscribes once the .connect() method is called 133 | */ 134 | @Test 135 | public void sharingResourcesBetweenSubscriptions() { 136 | ConnectableObservable connectableObservable = Observable.create(subscriber -> { 137 | log.info("Inside create()"); 138 | 139 | /* A JMS connection listener example 140 | Just an example of a costly operation that is better to be shared **/ 141 | 142 | /* Connection connection = connectionFactory.createConnection(); 143 | Session session = connection.createSession(true, AUTO_ACKNOWLEDGE); 144 | MessageConsumer consumer = session.createConsumer(orders); 145 | consumer.setMessageListener(subscriber::onNext); */ 146 | 147 | subscriber.setCancellable(() -> log.info("Subscription cancelled")); 148 | 149 | log.info("Emitting 1"); 150 | subscriber.onNext(1); 151 | 152 | log.info("Emitting 2"); 153 | subscriber.onNext(2); 154 | 155 | subscriber.onComplete(); 156 | }).publish(); //calling .publish makes an Observable a ConnectableObservable 157 | 158 | log.info("Before subscribing"); 159 | CountDownLatch latch = new CountDownLatch(2); 160 | 161 | /* calling .subscribe() bellow doesn't actually subscribe, but puts them in a list to actually subscribe 162 | when calling .connect() */ 163 | connectableObservable 164 | .take(1) 165 | .subscribe((val) -> log.info("Subscriber1 received: {}", val), logError(), logComplete(latch)); 166 | 167 | connectableObservable 168 | .subscribe((val) -> log.info("Subscriber2 received: {}", val), logError(), logComplete(latch)); 169 | 170 | 171 | //we need to call .connect() to trigger the real subscription 172 | log.info("Now connecting to the ConnectableObservable"); 173 | connectableObservable.connect(); 174 | 175 | Helpers.wait(latch); 176 | } 177 | 178 | /** 179 | * We can get away with having to call ourselves .connect(), by using 180 | */ 181 | @Test 182 | public void autoConnectingWithFirstSubscriber() { 183 | ConnectableObservable connectableObservable = Observable.create(subscriber -> { 184 | log.info("Inside create()"); 185 | 186 | //simulating some listener that produces events after 187 | //connection is initialized 188 | ResourceConnectionHandler resourceConnectionHandler = new ResourceConnectionHandler() { 189 | @Override 190 | public void onMessage(Integer message) { 191 | log.info("Emitting {}", message); 192 | subscriber.onNext(message); 193 | } 194 | }; 195 | 196 | resourceConnectionHandler.openConnection(); 197 | 198 | subscriber.setCancellable(resourceConnectionHandler::disconnect); 199 | }).publish(); 200 | 201 | Observable observable = connectableObservable.autoConnect(); 202 | 203 | CountDownLatch latch = new CountDownLatch(2); 204 | observable 205 | .take(5) 206 | .subscribe((val) -> log.info("Subscriber1 received: {}", val), logError(), logComplete(latch)); 207 | Helpers.sleepMillis(1000); 208 | 209 | observable 210 | .take(2) 211 | .subscribe((val) -> log.info("Subscriber2 received: {}", val), logError(), logComplete(latch)); 212 | 213 | Helpers.wait(latch); 214 | } 215 | 216 | /** 217 | * Even the above .autoConnect() can be improved 218 | */ 219 | @Test 220 | public void refCountTheConnectableObservableAutomaticSubscriptionOperator() { 221 | ConnectableObservable connectableObservable = Observable.create(subscriber -> { 222 | log.info("Inside create()"); 223 | 224 | //simulating some listener that produces events after 225 | //connection is initialized 226 | ResourceConnectionHandler resourceConnectionHandler = new ResourceConnectionHandler() { 227 | @Override 228 | public void onMessage(Integer message) { 229 | log.info("Emitting {}", message); 230 | subscriber.onNext(message); 231 | } 232 | }; 233 | resourceConnectionHandler.openConnection(); 234 | 235 | subscriber.setCancellable(resourceConnectionHandler::disconnect); 236 | }).publish(); 237 | 238 | Observable observable = connectableObservable.refCount(); 239 | //publish().refCount() equals share() 240 | 241 | CountDownLatch latch = new CountDownLatch(2); 242 | observable 243 | .take(5) 244 | .subscribe((val) -> log.info("Subscriber1 received: {}", val), logError(), logComplete(latch)); 245 | 246 | Helpers.sleepMillis(1000); 247 | 248 | log.info("Subscribing 2nd"); 249 | //we're not seeing the code inside .create() re-executed 250 | observable 251 | .take(2) 252 | .subscribe((val) -> log.info("Subscriber2 received: {}", val), logError(), logComplete(latch)); 253 | 254 | Helpers.wait(latch); 255 | 256 | //Previous Subscribers all unsubscribed, subscribing another will trigger the execution of the code 257 | //inside .create() 258 | latch = new CountDownLatch(1); 259 | log.info("Subscribing 3rd"); 260 | observable 261 | .take(1) 262 | .subscribe((val) -> log.info("Subscriber3 received: {}", val), logError(), logComplete(latch)); 263 | Helpers.wait(latch); 264 | } 265 | 266 | 267 | 268 | 269 | private ScheduledExecutorService periodicEventEmitter(Runnable action, 270 | int period, TimeUnit timeUnit) { 271 | ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1); 272 | scheduledExecutorService.scheduleAtFixedRate(action, 0, period, timeUnit); 273 | 274 | return scheduledExecutorService; 275 | } 276 | 277 | 278 | private abstract class ResourceConnectionHandler { 279 | 280 | ScheduledExecutorService scheduledExecutorService; 281 | 282 | private int counter; 283 | 284 | public void openConnection() { 285 | log.info("**Opening connection"); 286 | 287 | scheduledExecutorService = periodicEventEmitter(() -> { 288 | counter ++; 289 | onMessage(counter); 290 | }, 500, TimeUnit.MILLISECONDS); 291 | } 292 | 293 | public abstract void onMessage(Integer message); 294 | 295 | public void disconnect() { 296 | log.info("**Shutting down connection"); 297 | scheduledExecutorService.shutdown(); 298 | } 299 | } 300 | 301 | /* private Runnable pushEventsToSubjectAction(Subject subject, int maxEvents) { 302 | return () -> { 303 | if(counter == maxEvents) { 304 | subject.onComplete(); 305 | return; 306 | } 307 | 308 | counter ++; 309 | 310 | log.info("Emitted {}", counter); 311 | subject.onNext(counter); 312 | }; 313 | }*/ 314 | } 315 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part05AdvancedOperators.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Pair; 4 | import io.reactivex.Flowable; 5 | import io.reactivex.Observable; 6 | import io.reactivex.flowables.GroupedFlowable; 7 | import org.junit.Test; 8 | 9 | import java.util.List; 10 | import java.util.concurrent.TimeUnit; 11 | 12 | /** 13 | * @author sbalamaci 14 | */ 15 | public class Part05AdvancedOperators implements BaseTestObservables { 16 | 17 | @Test 18 | public void buffer() { 19 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS); 20 | 21 | Flowable> delayedNumbersWindow = numbers 22 | .buffer(5); 23 | 24 | subscribeWithLog(delayedNumbersWindow); 25 | } 26 | 27 | @Test 28 | public void simpleWindow() { 29 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS); 30 | 31 | Flowable delayedNumbersWindow = numbers 32 | .window(5) 33 | .flatMap(window -> window.doOnComplete(() -> log.info("Window completed"))); 34 | 35 | subscribeWithLog(delayedNumbersWindow); 36 | } 37 | 38 | 39 | @Test 40 | public void window() { 41 | Flowable numbers = Flowable.interval(1, TimeUnit.SECONDS); 42 | 43 | Flowable delayedNumbersWindow = numbers 44 | .window(10, 5, TimeUnit.SECONDS) 45 | .flatMap(window -> window.doOnComplete(() -> log.info("Window completed"))); 46 | 47 | subscribeWithLog(delayedNumbersWindow); 48 | } 49 | 50 | /** 51 | * groupBy splits the stream into multiple streams with the key generated by the function passed as 52 | * parameter to groupBy 53 | */ 54 | @Test 55 | public void groupBy() { 56 | Flowable colors = Flowable.fromArray("red", "green", "blue", 57 | "red", "yellow", "green", "green"); 58 | 59 | Flowable> groupedColorsStream = colors 60 | .groupBy(val -> val); //identity function 61 | // .groupBy(val -> "length" + val.length()); 62 | 63 | Flowable> colorCountStream = groupedColorsStream 64 | .flatMap(groupedColor -> groupedColor 65 | .count() 66 | .map(count -> new Pair<>(groupedColor.getKey(), count)) 67 | .toFlowable() 68 | ); 69 | 70 | subscribeWithLog(colorCountStream); 71 | } 72 | 73 | @Test 74 | public void bufferWithLimitTriggeredByObservable() { 75 | Observable colors = Observable.fromArray("red", "green", "blue", 76 | "red", "yellow", "#", "green", "green"); 77 | 78 | 79 | colors.publish(p -> p.filter(val -> ! val.equals("#")) 80 | .buffer(() -> p.filter(val -> val.equals("#"))) 81 | ) 82 | .subscribe(list -> { 83 | String listCommaSeparated = String.join(",", list); 84 | 85 | log.info("List {}", listCommaSeparated); 86 | }); 87 | } 88 | } 89 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part06Schedulers.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.Flowable; 5 | import io.reactivex.Observable; 6 | import io.reactivex.Single; 7 | import io.reactivex.schedulers.Schedulers; 8 | import org.junit.Test; 9 | 10 | import java.util.concurrent.CountDownLatch; 11 | import java.util.concurrent.ExecutorService; 12 | import java.util.concurrent.Executors; 13 | 14 | /** 15 | * RxJava provides some high level concepts for concurrent execution, like ExecutorService we're not dealing 16 | * with the low level constructs like creating the Threads ourselves. Instead we're using a {@see rx.Scheduler} which create 17 | * Workers who are responsible for scheduling and running code. By default RxJava will not introduce concurrency 18 | * and will run the operations on the subscription thread. 19 | * 20 | * There are two methods through which we can introduce Schedulers into our chain of operations: 21 | * - subscribeOn allows to specify which Scheduler invokes the code contained in the lambda code for Observable.create() 22 | * - observeOn allows control to which Scheduler executes the code in the downstream operators 23 | * 24 | * RxJava provides some general use Schedulers already implemented: 25 | * - Schedulers.computation() - to be used for CPU intensive tasks. A threadpool equal to the numbers of available CPUs 26 | * - Schedulers.io() - to be used for IO bound tasks 27 | * - Schedulers.from(Executor) - custom ExecutorService 28 | * - Schedulers.newThread() - always creates a new thread when a worker is needed. Since it's not thread pooled 29 | * and always creates a new thread instead of reusing one, this scheduler is not very useful 30 | * 31 | * Although we said by default RxJava doesn't introduce concurrency, some operators that involve waiting like 'delay', 32 | * 'interval' need to run on a Scheduler, otherwise they would just block the subscribing thread. 33 | * By default **Schedulers.computation()** is used, but the Scheduler can be passed as a parameter. 34 | * 35 | * @author sbalamaci 36 | */ 37 | public class Part06Schedulers implements BaseTestObservables { 38 | 39 | @Test 40 | public void byDefaultRxJavaDoesntIntroduceConcurrency() { 41 | log.info("Starting"); 42 | 43 | Observable.create(subscriber -> { 44 | log.info("Someone subscribed"); 45 | subscriber.onNext(1); 46 | subscriber.onNext(2); 47 | 48 | subscriber.onComplete(); 49 | }) 50 | .map(val -> { 51 | int newValue = val * 10; 52 | log.info("Mapping {} to {}", val, newValue); 53 | // Helpers.sleepMillis(2000); 54 | return newValue; 55 | }) 56 | .subscribe(logNext()); 57 | } 58 | 59 | @Test 60 | public void subscribingThread() { 61 | CountDownLatch latch = new CountDownLatch(1); 62 | 63 | Observable observable = Observable.create(subscriber -> { 64 | log.info("Someone subscribed"); 65 | new Thread(() -> { 66 | log.info("Emitting.."); 67 | subscriber.onNext(1); 68 | subscriber.onComplete(); 69 | }, "custom-thread").start(); 70 | }) 71 | .map(val -> { 72 | int newValue = val * 10; 73 | log.info("Mapping {} to {}", val, newValue); 74 | 75 | return newValue; 76 | }); 77 | 78 | observable.subscribe(logNext(), logError(), logComplete(latch)); 79 | Helpers.wait(latch); 80 | 81 | log.info("Blocking Subscribe"); 82 | observable.blockingSubscribe(logNext(), logError(), logComplete()); 83 | observable.observeOn(Schedulers.trampoline()); 84 | log.info("Got"); 85 | } 86 | 87 | /** 88 | * subscribeOn allows to specify which Scheduler invokes the code contained in the lambda code for Observable.create() 89 | */ 90 | @Test 91 | public void testSubscribeOn() { 92 | log.info("Starting"); 93 | 94 | Observable observable = Observable.create(subscriber -> { //code that will execute inside the IO ThreadPool 95 | log.info("Starting slow network op"); 96 | Helpers.sleepMillis(2000); 97 | 98 | log.info("Emitting 1st"); 99 | subscriber.onNext(1); 100 | 101 | subscriber.onComplete(); 102 | }); 103 | 104 | observable = observable 105 | .subscribeOn(Schedulers.io()) //Specify execution on the IO Scheduler 106 | .map(val -> { 107 | int newValue = val * 10; 108 | log.info("Mapping {} to {}", val, newValue); 109 | return newValue; 110 | }); 111 | 112 | subscribeWithLogOutputWaitingForComplete(observable); 113 | } 114 | 115 | 116 | /** 117 | * observeOn switches the thread that is used for the subscribers downstream. 118 | * If we initially subscribedOn the IoScheduler we and we 119 | * further make another . 120 | */ 121 | @Test 122 | public void testObserveOn() { 123 | log.info("Starting"); 124 | 125 | Flowable observable = simpleFlowable() 126 | .subscribeOn(Schedulers.io()) 127 | .observeOn(Schedulers.computation()) 128 | .map(val -> { 129 | int newValue = val * 10; 130 | log.info("Mapping {} to {}", val, newValue); 131 | return newValue; 132 | }) 133 | .observeOn(Schedulers.newThread()); 134 | 135 | subscribeWithLogOutputWaitingForComplete(observable); 136 | } 137 | 138 | /** 139 | * Multiple calls to subscribeOn have no effect, just the first one will take effect, so we'll see the code 140 | * execute on an IoScheduler thread. 141 | */ 142 | @Test 143 | public void multipleCallsToSubscribeOn() { 144 | log.info("Starting"); 145 | 146 | Flowable observable = simpleFlowable() 147 | .subscribeOn(Schedulers.io()) 148 | .subscribeOn(Schedulers.computation()) 149 | .map(val -> { 150 | int newValue = val * 2; 151 | log.info("Mapping new val {}", newValue); 152 | return newValue; 153 | }); 154 | 155 | subscribeWithLogOutputWaitingForComplete(observable); 156 | } 157 | 158 | @Test 159 | public void blocking() { 160 | log.info("Starting"); 161 | 162 | Flowable flowable = simpleFlowable() 163 | .subscribeOn(Schedulers.io()) 164 | .subscribeOn(Schedulers.computation()) 165 | .map(val -> { 166 | String newValue = "^^" + val + "^^"; 167 | log.info("Mapping new val {}", newValue); 168 | Helpers.sleepMillis(500); 169 | return newValue; 170 | }); 171 | flowable.blockingSubscribe(val -> log.info("Subscriber received {}", val)); 172 | log.info("Finished blocking subscribe"); 173 | 174 | Iterable iterable = flowable.blockingIterable(); 175 | iterable.forEach(val -> log.info("Received {}", val)); 176 | log.info("Finished blockingIterable"); 177 | } 178 | 179 | /** 180 | * Controlling concurrency in flatMap 181 | * 182 | * By using subscribeOn in flatMap you can control the thread on which flapMap subscribes to the particular 183 | * stream. By using a scheduler from a custom executor to which we allow a limited number of threads, 184 | * we can also control how many concurrent threads are handling stream operations inside the flatMap 185 | */ 186 | @Test 187 | public void flatMapSubscribesToSubstream() { 188 | ExecutorService fixedThreadPool = Executors.newFixedThreadPool(2); 189 | 190 | Flowable observable = Flowable.range(1, 5) 191 | .observeOn(Schedulers.io()) //Scheduler for multiply 192 | .map(val -> { 193 | log.info("Multiplying {}", val); 194 | return val * 10; 195 | }) 196 | .flatMap(val -> simulateRemoteOp(val) 197 | .subscribeOn(Schedulers.from(fixedThreadPool)) 198 | ); 199 | 200 | subscribeWithLogOutputWaitingForComplete(observable); 201 | } 202 | 203 | private Flowable simulateRemoteOp(Integer val) { 204 | return Single.create(subscriber -> { 205 | log.info("Simulate remote call {}", val); 206 | Helpers.sleepMillis(3000); 207 | subscriber.onSuccess("***" + val + "***"); 208 | }).toFlowable(); 209 | } 210 | 211 | } 212 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part07FlatMapOperator.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Pair; 4 | import io.reactivex.Flowable; 5 | import io.reactivex.flowables.GroupedFlowable; 6 | import io.reactivex.schedulers.Schedulers; 7 | import org.junit.Test; 8 | 9 | import java.util.Arrays; 10 | import java.util.List; 11 | import java.util.concurrent.TimeUnit; 12 | 13 | /** 14 | * The flatMap operator is so important and has so many different uses it deserves it's own category to explain 15 | * I like to think of it as a sort of fork-join operation because what flatMap does is it takes individual items 16 | * and maps each of them to an Observable(so it creates new Streams from each object) and then 'flattens' the 17 | * events from these Streams back into a single Stream. 18 | * Why this looks like fork-join because for each element you can fork some async jobs that emit some items before completing, 19 | * and these items are sent downstream to the subscribers as coming from a single stream 20 | * 21 | * RuleOfThumb 1: When you have an 'item' as parameter and you need to invoke something that returns an 22 | * Observable instead of , you need flatMap 23 | * RuleOfThumb 2: When you have Observable> you probably need flatMap. 24 | * 25 | * @author sbalamaci 26 | */ 27 | public class Part07FlatMapOperator implements BaseTestObservables { 28 | 29 | /** 30 | * Common usecase when for each item you make an async remote call that returns a stream of items (an Observable) 31 | * 32 | * The thing to notice that it's not clear upfront that events from the flatMaps 'substreams' don't arrive 33 | * in a guaranteed order and events from a substream might get interleaved with the events from other substreams. 34 | * 35 | */ 36 | @Test 37 | public void flatMap() { 38 | Flowable colors = Flowable.just("orange", "red", "green") 39 | .flatMap(colorName -> simulateRemoteOperation(colorName)); 40 | 41 | subscribeWithLogOutputWaitingForComplete(colors); 42 | } 43 | 44 | /** 45 | * Inside the flatMap we can operate on the substream with the same stream operators like for ex count 46 | */ 47 | @Test 48 | public void flatMapSubstreamOperations() { 49 | Flowable colors = Flowable.just("orange", "red", "green", "blue"); 50 | 51 | Flowable> colorsCounted = colors 52 | .flatMap(colorName -> { 53 | Flowable timer = Flowable.interval(2, TimeUnit.SECONDS); 54 | 55 | return simulateRemoteOperation(colorName) // <- Still a stream 56 | .zipWith(timer, (val, timerVal) -> val) 57 | .count() 58 | .map(counter -> new Pair<>(colorName, counter)) 59 | .toFlowable(); 60 | } 61 | ); 62 | 63 | subscribeWithLogOutputWaitingForComplete(colorsCounted); 64 | } 65 | 66 | 67 | /** 68 | * Controlling the level of concurrency of the substreams. 69 | * In the ex. below, only one of the substreams(the Observables returned by simulateRemoteOperation) 70 | * is subscribed. As soon the substream completes, another substream is subscribed. 71 | * Since only one substream is subscribed at any time, this way we don't see any values interleaved 72 | */ 73 | @Test 74 | public void flatMapConcurrency() { 75 | Flowable colors = Flowable.just("orange", "red", "green") 76 | .flatMap(colorName -> simulateRemoteOperation(colorName), 1); 77 | 78 | subscribeWithLogOutputWaitingForComplete(colors); 79 | } 80 | 81 | /** 82 | * As seen above flatMap might mean that events emitted by multiple streams might get interleaved 83 | * 84 | * concatMap operator acts as a flatMap with 1 level of concurrency which means only one of the created 85 | * substreams(Observable) is subscribed and thus only one emits events so it's just this substream which 86 | * emits events until it finishes and a new one will be subscribed and so on 87 | */ 88 | @Test 89 | public void concatMap() { 90 | Flowable colors = Flowable.just("orange", "red", "green", "blue") 91 | .subscribeOn(Schedulers.io()) 92 | .concatMap(val -> simulateRemoteOperation(val) 93 | .subscribeOn(Schedulers.io()) 94 | ); 95 | 96 | subscribeWithLogOutputWaitingForComplete(colors); 97 | } 98 | 99 | /** 100 | * When you have a Stream of Streams - Observable> 101 | */ 102 | @Test 103 | public void flatMapForProcessingAStreamOfStreams() { 104 | Flowable colors = Flowable.just("red", "green", "blue", 105 | "red", "yellow", "green", "green"); 106 | 107 | Flowable> groupedColorsStream = colors 108 | .groupBy(val -> val);//grouping key 109 | // is the String itself, the color 110 | 111 | Flowable> 112 | countedColors = groupedColorsStream 113 | .flatMap(groupedFlow -> groupedFlow 114 | .count() 115 | .map(countVal -> new Pair<>(groupedFlow.getKey(), countVal)) 116 | .toFlowable() 117 | ); 118 | 119 | subscribeWithLogOutputWaitingForComplete(countedColors); 120 | } 121 | 122 | /** 123 | * 'switchIfEmpty' push some value(s) when the original stream just completes without 'returning' anything 124 | */ 125 | @Test 126 | public void flatMapSubstituteEmptyStream() { 127 | Flowable colors = Flowable.just("red", "", "blue") 128 | .flatMap(colorName -> simulateRemoteOperation(colorName) 129 | .switchIfEmpty(Flowable.just("NONE"))); 130 | 131 | subscribeWithLogOutputWaitingForComplete(colors); 132 | } 133 | 134 | /** 135 | * flatMapIterable just takes as List and emits each of the elements 136 | * as a stream. 137 | */ 138 | @Test 139 | public void flatMapIterable() { 140 | Flowable colors = Flowable.just(1) 141 | .flatMapIterable(it -> generateColors()); 142 | 143 | subscribeWithLogOutputWaitingForComplete(colors); 144 | } 145 | 146 | private List generateColors() { 147 | return Arrays.asList("red", "green", "blue"); 148 | } 149 | 150 | @Test 151 | public void switchMap() { 152 | Flowable colors = Flowable.interval(0,400, TimeUnit.MILLISECONDS) 153 | .zipWith(Arrays.asList("EUR", "USD", "GBP"), (it, currency) -> currency) 154 | .doOnNext(ev -> log.info("Emitting {}", ev)) 155 | .switchMap(currency -> simulateRemoteOperation(currency) 156 | .doOnSubscribe((subscription) -> log.info("Subscribed new")) 157 | .doOnCancel(() -> log.info("Unsubscribed {}", currency)) 158 | ); 159 | 160 | subscribeWithLogOutputWaitingForComplete(colors); 161 | } 162 | 163 | 164 | /** 165 | * Simulated remote operation that emits as many events as the length of the color string 166 | * @param color color 167 | * @return stream of events 168 | */ 169 | private Flowable simulateRemoteOperation(String color) { 170 | return Flowable.intervalRange(1, color.length(), 0, 200, TimeUnit.MILLISECONDS) 171 | .map(iteration -> color + iteration); 172 | } 173 | 174 | } 175 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part08ErrorHandling.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import io.reactivex.BackpressureStrategy; 4 | import io.reactivex.Flowable; 5 | import org.junit.Test; 6 | 7 | import java.util.Random; 8 | import java.util.concurrent.ConcurrentHashMap; 9 | import java.util.concurrent.TimeUnit; 10 | import java.util.concurrent.atomic.AtomicInteger; 11 | 12 | /** 13 | * Exceptions are for exceptional situations. 14 | * The Observable contract specifies that exceptions are terminal operations. 15 | * There are however operator available for error flow control 16 | */ 17 | public class Part08ErrorHandling implements BaseTestObservables { 18 | 19 | private static final ConcurrentHashMap attemptsMap = new ConcurrentHashMap<>(); 20 | 21 | /** 22 | * After the map() operator encounters an error, it triggers the error handler 23 | * in the subscriber which also unsubscribes from the stream, 24 | * therefore 'yellow' is not even sent downstream. 25 | */ 26 | @Test 27 | public void errorIsTerminalOperation() { 28 | Flowable colors = Flowable.just("green", "blue", "red", "yellow") 29 | .map(color -> { 30 | if ("red".equals(color)) { 31 | throw new RuntimeException("Encountered red"); 32 | } 33 | return color + "*"; 34 | }) 35 | .map(val -> val + "XXX"); 36 | 37 | subscribeWithLog(colors); 38 | } 39 | 40 | 41 | /** 42 | * The 'onErrorReturn' operator doesn't prevent the unsubscription from the 'colors' 43 | * but it does translate the exception for the downstream operators and the final Subscriber 44 | * receives it in the 'onNext()' instead in 'onError()' 45 | */ 46 | @Test 47 | public void onErrorReturn() { 48 | Flowable colors = Flowable.just("green", "blue", "red", "yellow") 49 | .map(color -> { 50 | if ("red".equals(color)) { 51 | throw new RuntimeException("Encountered red"); 52 | } 53 | return color + "*"; 54 | }) 55 | .onErrorReturn(th -> "-blank-") 56 | .map(val -> val + "XXX"); 57 | 58 | subscribeWithLog(colors); 59 | } 60 | 61 | 62 | @Test 63 | public void onErrorReturnWithFlatMap() { 64 | //flatMap encounters an error when it subscribes to 'red' substreams and thus unsubscribe from 65 | // 'colors' stream and the remaining colors still are not longer emitted 66 | Flowable colors = Flowable.just("green", "blue", "red", "white", "blue") 67 | .flatMap(color -> simulateRemoteOperation(color)) 68 | .onErrorReturn(throwable -> "-blank-"); //onErrorReturn just has the effect of translating 69 | 70 | subscribeWithLog(colors); 71 | 72 | log.info("*****************"); 73 | 74 | //bellow onErrorReturn() is applied to the flatMap substream and thus translates the exception to 75 | //a value and so flatMap continues on with the other colors after red 76 | colors = Flowable.just("green", "blue", "red", "white", "blue") 77 | .flatMap(color -> simulateRemoteOperation(color) 78 | .onErrorReturn(throwable -> "-blank-") //onErrorReturn doesn't trigger 79 | // the onError() inside flatMap so it doesn't unsubscribe from 'colors' 80 | ); 81 | 82 | subscribeWithLog(colors); 83 | } 84 | 85 | 86 | /** 87 | * onErrorResumeNext() returns a stream instead of an exception and subscribes to that stream instead, 88 | * useful for example to invoke a fallback method that returns also a stream 89 | */ 90 | @Test 91 | public void onErrorResumeNext() { 92 | Flowable colors = Flowable.just("green", "blue", "red", "white", "blue") 93 | .flatMap(color -> simulateRemoteOperation(color) 94 | .onErrorResumeNext(th -> { 95 | if (th instanceof IllegalArgumentException) { 96 | return Flowable.error(new RuntimeException("Fatal, wrong arguments")); 97 | } 98 | return fallbackRemoteOperation(); 99 | }) 100 | ); 101 | 102 | subscribeWithLog(colors); 103 | } 104 | 105 | private Flowable fallbackRemoteOperation() { 106 | return Flowable.just("blank"); 107 | } 108 | 109 | 110 | /** 111 | ************* Retry Logic **************** 112 | ****************************************** */ 113 | 114 | /** 115 | * timeout operator raises exception when there are no events incoming before it's predecessor in the specified 116 | * time limit 117 | *

118 | * retry() resubscribes in case of exception to the Observable 119 | */ 120 | @Test 121 | public void timeoutWithRetry() { 122 | Flowable colors = Flowable.just("red", "blue", "green", "yellow") 123 | .concatMap(color -> delayedByLengthEmitter(TimeUnit.SECONDS, color) 124 | .timeout(6, TimeUnit.SECONDS) 125 | .retry(2) 126 | .onErrorResumeNext(Flowable.just("blank")) 127 | ); 128 | 129 | subscribeWithLog(colors); 130 | //there is also 131 | } 132 | 133 | /** 134 | * When you want to retry based on the number considering the thrown exception type 135 | */ 136 | @Test 137 | public void retryBasedOnAttemptsAndExceptionType() { 138 | Flowable colors = Flowable.just("blue", "red", "black", "yellow"); 139 | 140 | colors = colors 141 | .flatMap(colorName -> simulateRemoteOperation(colorName, 2) 142 | .retry((retryAttempt, exception) -> { 143 | if (exception instanceof IllegalArgumentException) { 144 | log.error("{} encountered non retry exception ", colorName); 145 | return false; 146 | } 147 | log.info("Retry attempt {} for {}", retryAttempt, colorName); 148 | return retryAttempt <= 3; 149 | }) 150 | .onErrorResumeNext(Flowable.just("generic color")) 151 | ); 152 | 153 | subscribeWithLog(colors); 154 | } 155 | 156 | /** 157 | * A more complex retry logic like implementing a backoff strategy in case of exception 158 | * This can be obtained with retryWhen(exceptionObservable -> Observable) 159 | *

160 | * retryWhen resubscribes when an event from an Observable is emitted. It receives as parameter an exception stream 161 | *

162 | * we zip the exceptionsStream with a .range() stream to obtain the number of retries, 163 | * however we want to wait a little before retrying so in the zip function we return a delayed event - .timer() 164 | *

165 | * The delay also needs to be subscribed to be effected so we also need flatMap 166 | */ 167 | @Test 168 | public void retryWhenUsedForRetryWithBackoff() { 169 | Flowable colors = Flowable.just("blue", "green", "red", "black", "yellow"); 170 | 171 | colors = colors.flatMap(colorName -> 172 | simulateRemoteOperation(colorName, 3) 173 | .retryWhen(exceptionStream -> exceptionStream 174 | .zipWith(Flowable.range(1, 3), (exc, attempts) -> { 175 | //don't retry for IllegalArgumentException 176 | if (exc instanceof IllegalArgumentException) { 177 | return Flowable.error(exc); 178 | } 179 | 180 | if (attempts < 3) { 181 | log.info("Attempt {}, waiting before retry", attempts); 182 | return Flowable.timer(2 * attempts, TimeUnit.SECONDS); 183 | } 184 | return Flowable.error(exc); 185 | }) 186 | .flatMap(val -> val) 187 | ) 188 | .onErrorResumeNext(Flowable.just("generic color")) 189 | ); 190 | 191 | subscribeWithLog(colors); 192 | } 193 | 194 | /** 195 | * repeatWhen is identical to retryWhen only it responds to 'onCompleted' instead of 'onError' 196 | */ 197 | @Test 198 | public void testRepeatWhen() { 199 | Flowable remoteOperation = Flowable.defer(() -> { 200 | Random random = new Random(); 201 | return Flowable.just(random.nextInt(10)); 202 | }); 203 | 204 | remoteOperation = remoteOperation.repeatWhen(completed -> completed 205 | .delay(2, TimeUnit.SECONDS) 206 | ) 207 | .take(10); 208 | subscribeWithLogOutputWaitingForComplete(remoteOperation); 209 | } 210 | 211 | private Flowable simulateRemoteOperation(String color) { 212 | return simulateRemoteOperation(color, Integer.MAX_VALUE); 213 | } 214 | 215 | private Flowable simulateRemoteOperation(String color, int workAfterAttempts) { 216 | return Flowable.create(subscriber -> { 217 | AtomicInteger attemptsHolder = attemptsMap.computeIfAbsent(color, (colorKey) -> new AtomicInteger(0)); 218 | int attempts = attemptsHolder.incrementAndGet(); 219 | 220 | if ("red".equals(color)) { 221 | checkAndThrowException(color, attempts, workAfterAttempts, 222 | new RuntimeException("Color red raises exception")); 223 | } 224 | if ("black".equals(color)) { 225 | checkAndThrowException(color, attempts, workAfterAttempts, 226 | new IllegalArgumentException("Black is not a color")); 227 | } 228 | 229 | String value = "**" + color + "**"; 230 | 231 | log.info("Emitting {}", value); 232 | subscriber.onNext(value); 233 | subscriber.onComplete(); 234 | }, BackpressureStrategy.BUFFER); 235 | } 236 | 237 | private void checkAndThrowException(String color, int attempts, int workAfterAttempts, Exception exception) { 238 | if(attempts < workAfterAttempts) { 239 | log.info("Emitting {} for {}", exception.getClass(), color); 240 | throw new IllegalArgumentException("Black is not a color"); 241 | } else { 242 | log.info("After attempt {} we don't throw exception", attempts); 243 | } 244 | } 245 | } 246 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/Part09BackpressureHandling.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx; 2 | 3 | import com.balamaci.rx.util.Helpers; 4 | import io.reactivex.BackpressureOverflowStrategy; 5 | import io.reactivex.BackpressureStrategy; 6 | import io.reactivex.Flowable; 7 | import io.reactivex.schedulers.Schedulers; 8 | import io.reactivex.subjects.PublishSubject; 9 | import org.junit.Test; 10 | import org.reactivestreams.Subscriber; 11 | import org.reactivestreams.Subscription; 12 | 13 | import java.util.concurrent.CountDownLatch; 14 | import java.util.concurrent.TimeUnit; 15 | 16 | /** 17 | * Backpressure is related to preventing overloading the subscriber with too many events. 18 | * It can be the case of a slow consumer that cannot keep up with the producer. 19 | * Backpressure relates to a feedback mechanism through which the subscriber can signal 20 | * to the producer how much data it can consume. 21 | * 22 | * However the producer must be 'backpressure-aware' in order to know how to throttle back. 23 | * 24 | * If the producer is not 'backpressure-aware', in order to prevent an OutOfMemory due to an unbounded increase of events, 25 | * we still can define a BackpressureStrategy to specify how we should deal with piling events. 26 | * If we should buffer(BackpressureStrategy.BUFFER) or drop(BackpressureStrategy.DROP, BackpressureStrategy.LATEST) 27 | * incoming events. 28 | * 29 | * @author sbalamaci 30 | */ 31 | public class Part09BackpressureHandling implements BaseTestObservables { 32 | 33 | 34 | 35 | @Test 36 | public void customBackpressureAwareFlux() { 37 | Flowable flux = new CustomRangeFlowable(5, 10); 38 | 39 | flux.subscribe(new Subscriber() { 40 | 41 | private Subscription subscription; 42 | private int backlogItems; 43 | 44 | private final int BATCH = 2; 45 | private final int INITIAL_REQ = 5; 46 | 47 | @Override 48 | public void onSubscribe(Subscription subscription) { 49 | this.subscription = subscription; 50 | backlogItems = INITIAL_REQ; 51 | 52 | log.info("Initial request {}", backlogItems); 53 | subscription.request(backlogItems); 54 | } 55 | 56 | @Override 57 | public void onNext(Integer val) { 58 | log.info("Subscriber received {}", val); 59 | backlogItems--; 60 | 61 | if (backlogItems == 0) { 62 | backlogItems = BATCH; 63 | subscription.request(BATCH); 64 | } 65 | } 66 | 67 | @Override 68 | public void onError(Throwable throwable) { 69 | log.info("Subscriber encountered error"); 70 | } 71 | 72 | @Override 73 | public void onComplete() { 74 | log.info("Subscriber completed"); 75 | } 76 | }); 77 | } 78 | 79 | /** 80 | * We use BackpressureStrategy.DROP in create() to handle events that are emitted outside 81 | * the request amount from the downstream subscriber. 82 | * 83 | * We see that events that reach the subscriber are those only 3 requested by the 84 | * observeOn operator, the events produced outside of the requested amount 85 | * are discarded (BackpressureStrategy.DROP). 86 | * 87 | * observeOn has a default request size from upstream of 128(system parameter {@code rx2.buffer-size}) 88 | * 89 | */ 90 | @Test 91 | public void createFlowableWithBackpressureStrategy() { 92 | BackpressureStrategy backpressureStrategy = 93 | BackpressureStrategy.DROP 94 | // BackpressureStrategy.BUFFER 95 | // BackpressureStrategy.LATEST 96 | // BackpressureStrategy.ERROR 97 | ; 98 | 99 | Flowable flowable = createFlowable(5, backpressureStrategy); 100 | 101 | //we need to switch threads to not run the producer in the same thread as the subscriber(which waits some time 102 | // to simulate a slow subscriber) 103 | flowable = flowable 104 | .observeOn(Schedulers.io(), false, 3); 105 | 106 | subscribeWithSlowSubscriberAndWait(flowable); 107 | } 108 | 109 | 110 | /** 111 | * There are operators for specifying backpressure strategy anywhere in the operators chain, 112 | * not just at Flowable.create(). 113 | * - onBackpressureBuffer 114 | * - onBackpressureDrop 115 | * - onBackpressureLatest 116 | * These operators request from upstream the Long.MAX_VALUE(unbounded amount) and then they buffer(onBackpressureBuffer) 117 | * the events for downstream and send the events as requested. 118 | * 119 | * In the example we specify a buffering strategy in the example, however since the buffer is not very large, 120 | * we still get an exception after the 8th value - 3(requested) + 5(buffer) 121 | * 122 | * We create the Flowable with BackpressureStrategy.MISSING saying we don't care about backpressure 123 | * but let one of the onBackpressureXXX operators handle it. 124 | * 125 | */ 126 | @Test 127 | public void bufferingBackpressureOperator() { 128 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 129 | .onBackpressureBuffer(5, () -> log.info("Buffer has overflown")); 130 | 131 | flowable = flowable 132 | .observeOn(Schedulers.io(), false, 3); 133 | 134 | subscribeWithSlowSubscriberAndWait(flowable); 135 | } 136 | 137 | /** 138 | * We can opt for a variant of the onBackpressureBuffer, to drop events that do not fit 139 | * inside the buffer 140 | */ 141 | @Test 142 | public void bufferingThenDroppingEvents() { 143 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 144 | .onBackpressureBuffer(5, () -> log.info("Buffer has overflown"), 145 | BackpressureOverflowStrategy.DROP_OLDEST); 146 | 147 | //we need to switch threads to not run the producer in the same thread as the subscriber(which waits some time 148 | // to simulate a slow subscriber) 149 | flowable = flowable 150 | .observeOn(Schedulers.io(), false, 3); 151 | 152 | subscribeWithSlowSubscriberAndWait(flowable); 153 | } 154 | 155 | 156 | /** 157 | * Not only a slow subscriber triggers backpressure, but also a slow operator 158 | * that slows down the handling of events and new request calls for new items 159 | */ 160 | @Test 161 | public void throwingBackpressureNotSupportedSlowOperator() { 162 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 163 | .onBackpressureDrop((val) -> log.info("Dropping {}", val)) 164 | .observeOn(Schedulers.io(), false, 3) 165 | .map(val -> { 166 | Helpers.sleepMillis(50); 167 | return "*" + val + "*"; 168 | }); 169 | 170 | subscribeWithLogOutputWaitingForComplete(flowable); //notice it's not the slowSubscribe method used 171 | } 172 | 173 | /** 174 | * Backpressure operators can be added whenever necessary and it's not limited to 175 | * cold publishers and we can use them on hot publishers also 176 | */ 177 | @Test 178 | public void backpressureWithHotPublisher() { 179 | CountDownLatch latch = new CountDownLatch(1); 180 | 181 | PublishSubject subject = PublishSubject.create(); 182 | 183 | Flowable flowable = subject 184 | .toFlowable(BackpressureStrategy.MISSING) 185 | .onBackpressureDrop(val -> log.info("Dropped {}", val)); 186 | 187 | flowable = flowable.observeOn(Schedulers.io(), false, 3); 188 | 189 | subscribeWithSlowSubscriber(flowable, latch); 190 | 191 | for (int i = 1; i <= 10; i++) { 192 | log.info("Emitting {}", i); 193 | subject.onNext(i); 194 | } 195 | subject.onComplete(); 196 | 197 | Helpers.wait(latch); 198 | } 199 | 200 | /** 201 | * Chaining together multiple onBackpressureXXX operators doesn't actually make sense 202 | * Using 203 | * .onBackpressureBuffer(5) 204 | * .onBackpressureDrop((val) -> log.info("Dropping {}", val)) 205 | * is not behaving as maybe expected - buffer 5 values, and then dropping overflowing events-. 206 | * 207 | * Because onBackpressureDrop subscribes to the previous onBackpressureBuffer operator 208 | * signaling its requesting Long.MAX_VALUE(unbounded amount) from it, the onBackpressureBuffer will never feel 209 | * its subscriber is overwhelmed and never "trigger" meaning that the last onBackpressureXXX operator overrides 210 | * the previous one. 211 | * 212 | * Of course for implementing an event dropping strategy after a full buffer, there is the special overrided 213 | * version of onBackpressureBuffer that takes a BackpressureOverflowStrategy. 214 | */ 215 | @Test 216 | public void cascadingOnBackpressureXXXOperators() { 217 | Flowable flowable = createFlowable(10, BackpressureStrategy.MISSING) 218 | .onBackpressureBuffer(5) 219 | .onBackpressureDrop((val) -> log.info("Dropping {}", val)) 220 | .observeOn(Schedulers.io(), false, 3); 221 | 222 | subscribeWithSlowSubscriberAndWait(flowable); 223 | } 224 | 225 | 226 | /** 227 | * Zipping a slow stream with a faster one also can cause a backpressure problem 228 | */ 229 | @Test 230 | public void zipOperatorHasALimit() { 231 | Flowable fast = createFlowable(200, BackpressureStrategy.MISSING); 232 | Flowable slowStream = Flowable.interval(100, TimeUnit.MILLISECONDS); 233 | 234 | Flowable observable = Flowable.zip(fast, slowStream, 235 | (val1, val2) -> val1 + " " + val2); 236 | 237 | subscribeWithSlowSubscriberAndWait(observable); 238 | } 239 | 240 | @Test 241 | public void backpressureAwareObservable() { 242 | Flowable flowable = Flowable.range(0, 10); 243 | 244 | flowable = flowable 245 | .observeOn(Schedulers.io(), false, 3); 246 | 247 | subscribeWithSlowSubscriberAndWait(flowable); 248 | } 249 | 250 | 251 | private Flowable createFlowable(int items, 252 | BackpressureStrategy backpressureStrategy) { 253 | return Flowable.create(subscriber -> { 254 | log.info("Started emitting"); 255 | 256 | for (int i = 0; i < items; i++) { 257 | if(subscriber.isCancelled()) { 258 | return; 259 | } 260 | 261 | log.info("Emitting {}", i); 262 | subscriber.onNext(i); 263 | } 264 | 265 | subscriber.onComplete(); 266 | }, backpressureStrategy); 267 | } 268 | 269 | 270 | private void subscribeWithSlowSubscriberAndWait(Flowable flowable) { 271 | CountDownLatch latch = new CountDownLatch(1); 272 | flowable.subscribe(logNextAndSlowByMillis(50), logError(latch), logComplete(latch)); 273 | 274 | Helpers.wait(latch); 275 | } 276 | 277 | private void subscribeWithSlowSubscriber(Flowable flowable, CountDownLatch latch) { 278 | flowable.subscribe(logNextAndSlowByMillis(50), logError(latch), logComplete(latch)); 279 | } 280 | 281 | private class CustomRangeFlowable extends Flowable { 282 | 283 | private int startFrom; 284 | private int count; 285 | 286 | CustomRangeFlowable(int startFrom, int count) { 287 | this.startFrom = startFrom; 288 | this.count = count; 289 | } 290 | 291 | @Override 292 | public void subscribeActual(Subscriber subscriber) { 293 | subscriber.onSubscribe(new CustomRangeSubscription(startFrom, count, subscriber)); 294 | } 295 | 296 | class CustomRangeSubscription implements Subscription { 297 | 298 | volatile boolean cancelled; 299 | boolean completed = false; 300 | private int count; 301 | private int currentCount; 302 | private int startFrom; 303 | 304 | private Subscriber actualSubscriber; 305 | 306 | CustomRangeSubscription(int startFrom, int count, Subscriber actualSubscriber) { 307 | this.count = count; 308 | this.startFrom = startFrom; 309 | this.actualSubscriber = actualSubscriber; 310 | } 311 | 312 | @Override 313 | public void request(long items) { 314 | log.info("Downstream requests {} items", items); 315 | for(int i=0; i < items; i++) { 316 | if(cancelled || completed) { 317 | return; 318 | } 319 | 320 | if(currentCount == count) { 321 | completed = true; 322 | if(cancelled) { 323 | return; 324 | } 325 | 326 | actualSubscriber.onComplete(); 327 | return; 328 | } 329 | 330 | int emitVal = startFrom + currentCount; 331 | currentCount++; 332 | actualSubscriber.onNext(emitVal); 333 | } 334 | } 335 | 336 | @Override 337 | public void cancel() { 338 | cancelled = true; 339 | } 340 | } 341 | } 342 | 343 | 344 | } 345 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/util/Helpers.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx.util; 2 | 3 | import org.slf4j.Logger; 4 | import org.slf4j.LoggerFactory; 5 | 6 | import java.util.concurrent.CountDownLatch; 7 | 8 | /** 9 | * @author sbalamaci 10 | */ 11 | public class Helpers { 12 | 13 | private static final Logger log = LoggerFactory.getLogger(Helpers.class); 14 | 15 | public static void sleepMillis(int millis) { 16 | try { 17 | Thread.sleep(millis); 18 | } catch (InterruptedException e) { 19 | log.error("Interrupted Thread"); 20 | throw new RuntimeException("Interrupted thread"); 21 | } 22 | } 23 | 24 | public static void wait(CountDownLatch waitOn) { 25 | try { 26 | waitOn.await(); 27 | } catch (InterruptedException e) { 28 | log.error("Interrupted waiting on CountDownLatch"); 29 | throw new RuntimeException("Interrupted thread"); 30 | } 31 | } 32 | 33 | } 34 | -------------------------------------------------------------------------------- /src/test/java/com/balamaci/rx/util/Pair.java: -------------------------------------------------------------------------------- 1 | package com.balamaci.rx.util; 2 | 3 | public class Pair { 4 | 5 | private T key; 6 | private V value; 7 | 8 | public Pair(T key, V value) { 9 | this.key = key; 10 | this.value = value; 11 | } 12 | 13 | public T getKey() { 14 | return key; 15 | } 16 | 17 | public V getValue() { 18 | return value; 19 | } 20 | 21 | @Override 22 | public String toString() { 23 | return "Pair{" + 24 | "key=" + key + 25 | ", value=" + value + 26 | '}'; 27 | } 28 | } 29 | -------------------------------------------------------------------------------- /src/test/resources/simplelogger.properties: -------------------------------------------------------------------------------- 1 | # SLF4J's SimpleLogger configuration file 2 | # Simple implementation of Logger that sends all enabled log messages, for all defined loggers, to System.err. 3 | 4 | # Default logging detail level for all instances of SimpleLogger. 5 | # Must be one of ("trace", "debug", "info", "warn", or "error"). 6 | # If not specified, defaults to "info". 7 | org.slf4j.simpleLogger.defaultLogLevel=info 8 | 9 | # Logging detail level for a SimpleLogger instance named "xxxxx". 10 | # Must be one of ("trace", "debug", "info", "warn", or "error"). 11 | # If not specified, the default logging detail level is used. 12 | #org.slf4j.simpleLogger.log.xxxxx= 13 | org.slf4j.simpleLogger.log.ro.fortsoft.monitor=info 14 | 15 | # Set to true if you want the current date and time to be included in output messages. 16 | # Default is false, and will output the number of milliseconds elapsed since startup. 17 | #org.slf4j.simpleLogger.showDateTime=false 18 | org.slf4j.simpleLogger.showDateTime=true 19 | 20 | # The date and time format to be used in the output messages. 21 | # The pattern describing the date and time format is the same that is used in java.text.SimpleDateFormat. 22 | # If the format is not specified or is invalid, the default format is used. 23 | # The default format is yyyy-MM-dd HH:mm:ss:SSS Z. 24 | #org.slf4j.simpleLogger.dateTimeFormat=yyyy-MM-dd HH:mm:ss:SSS Z 25 | org.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss 26 | 27 | # Set to true if you want to output the current thread name. 28 | # Defaults to true. 29 | #org.slf4j.simpleLogger.showThreadName=true 30 | 31 | # Set to true if you want the Logger instance name to be included in output messages. 32 | # Defaults to true. 33 | #org.slf4j.simpleLogger.showLogName=true 34 | 35 | # Set to true if you want the last component of the name to be included in output messages. 36 | # Defaults to false. 37 | org.slf4j.simpleLogger.showShortLogName=true --------------------------------------------------------------------------------