├── .gitignore ├── LICENSE ├── README.md ├── author.jpg ├── build.sbt ├── op-principle-diagram.png ├── project ├── build.properties └── plugins.sbt ├── src ├── main │ ├── java │ │ └── hobby │ │ │ └── wei │ │ │ └── c │ │ │ ├── reflow │ │ │ ├── adapt │ │ │ │ └── task.java │ │ │ └── step │ │ │ │ ├── Step.java │ │ │ │ ├── Tracker.java │ │ │ │ ├── Unit.java │ │ │ │ └── UnitM.java │ │ │ └── tool │ │ │ └── Reflect.java │ └── scala │ │ └── hobby │ │ └── wei │ │ └── c │ │ ├── reflow │ │ ├── Assist.scala │ │ ├── CodeException.scala │ │ ├── Config.scala │ │ ├── Dependency.scala │ │ ├── Env.scala │ │ ├── Feedback.scala │ │ ├── GlobalTrack.scala │ │ ├── Helper.scala │ │ ├── In.scala │ │ ├── KvTpe.scala │ │ ├── Out.scala │ │ ├── Poster.scala │ │ ├── Pulse.scala │ │ ├── Reflow.scala │ │ ├── Scheduler.scala │ │ ├── State.scala │ │ ├── Task.scala │ │ ├── ThreadResetor.scala │ │ ├── Tracker.scala │ │ ├── Trait.scala │ │ ├── Transformer.scala │ │ ├── Worker.scala │ │ ├── implicits.scala │ │ └── lite │ │ │ └── Task.scala │ │ └── tool │ │ └── Snatcher.scala └── test │ ├── java │ ├── StepTest.java │ └── UnitTest.java │ └── scala │ ├── VolatileTest.scala │ ├── reflow │ └── test │ │ ├── LiteSpec.scala │ │ ├── PulseSpec.scala │ │ ├── ReflowSpec.scala │ │ ├── SnatcherSpec.scala │ │ └── tasks.scala │ └── sample │ ├── FunSuitee.scala │ ├── SetSpec.scala │ ├── TVSetActorSpec.scala │ └── TVSetTest.scala └── why-snatcher.md /.gitignore: -------------------------------------------------------------------------------- 1 | target/ 2 | project/project 3 | 4 | .DS_Store 5 | 6 | # Built application files 7 | *.apk 8 | *.ap_ 9 | 10 | # Files for the ART/Dalvik VM 11 | *.dex 12 | 13 | # Java class files 14 | *.class 15 | 16 | # Generated files 17 | bin/ 18 | gen/ 19 | out/ 20 | 21 | # Gradle files 22 | .gradle/ 23 | build/ 24 | 25 | # Local configuration file (sdk path, etc) 26 | local.properties 27 | 28 | # Proguard folder generated by Eclipse 29 | /proguard/ 30 | 31 | # Log Files 32 | *.log 33 | 34 | # Android Studio Navigation editor temp files 35 | .navigation/ 36 | 37 | # Android Studio captures folder 38 | captures/ 39 | 40 | # Intellij 41 | .idea/ 42 | 43 | # Keystore files 44 | *.jks 45 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Reflow 2 | 3 | A light-weight lock-free `series/parallel` combined scheduling framework for tasks. The goal is to maximize parallelism in order to minimize the execution time overall. 4 | 5 | ---- 6 | 7 | * From imperative to `Monad`: **[lite.Task](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/lite/Task.scala) —— Simplified and Type-safe Task Assembly Toolset** 8 | - Simplify the mixed assembly of 'series/parallel' tasks, and each task definition and assembly tool are marked with `I/O` type, using the compiler check, match errors at a glance. 9 | Examples see _[here](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/LiteSpec.scala)_. 10 | 11 | * From single execution to streaming: **[Pulse](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Pulse.scala) —— Step Streaming Data Processor** 12 | - Data flows through a `set of large-scale integration tasks`, can always be entered in the order in which it was entered, can be 'queued' (`FIFO`, using a ingenious scheduling strategy that doesn't actually have queue) into each tasks, 13 | and each task can also retain the mark specially left by the previous data during processing, so as to be used for the next data processing. 14 | It doesn't matter in any subtask of any depth, and it doesn't matter if the previous data stays in a subtask much longer than the latter. 15 | Examples see _[here](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/PulseSpec.scala)_. 16 | - Backpressure. There are two parameters to resolve the problem: `Pulse.inputCapacity/execCapacity`. Note the return value (`true/false`) of `Pulse.input()`. 17 | 18 | ---- 19 | 20 | ## Overview 21 | 22 | "The maximum improvement in system performance is determined by the parts that cannot be parallelized." 23 | —— [Amdahl's law](https://en.wikipedia.org/wiki/Amdahl%27s_law) 24 | 25 | In a large system, there is often a lot of business logic and control logic, which is the embodiment of hundreds of "jobs". The logic is intertwined and, taken as a whole, often intricate. 26 | On the other hand, if this large amount of "jobs" is not organized reasonably and effectively, the execution time overall and performance are also worrying. 27 | So how do we abstract and simplify such a complex problem? 28 | 29 | > Programs = Algorithms + Data Structures 30 | > Algorithm = Logic + Control 31 | 32 | Inspired by this idea, we can separate the business logic from the control logic, abstract the control logic as a framework, and construct the business logic as a **Task**. 33 | The relationship between tasks can also be further classified into two types: _dependent_ and _non-dependent_, that is, **serial** and **parallel** (dependent tasks must be executed sequentially, and non-dependent tasks can be executed in parallel). 34 | The user programmer only needs to focus on writing task one by one and leave the rest to the framework. **Reflow** is designed around the control logic that handles these tasks. 35 | 36 | **Reflow** was developed to simplify the **coding complexity** of data-flow and event-processing between multiple [tasks](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Task.scala) 37 | in _complex business logic_. Through the design of [ I](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Trait.scala#L46) 38 | /[ O](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Trait.scala#L49) 39 | that requires [explicit definition](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Trait.scala#L43) 40 | of tasks, intelligent [dependency management](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Dependency.scala) 41 | based on [keyword](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/KvTpe.scala) 42 | and [value-type](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/KvTpe.scala) 43 | analysis, unified operation [scheduling](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Scheduler.scala), 44 | [event feedback](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Feedback.scala) 45 | and [error handling](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Task.scala#L122) 46 | interface, the set goal is realized: task [_series_](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Dependency.scala#L81) 47 | /[ _parallel_](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Dependency.scala#L55) 48 | combined scheduling. _Data_ is **electricity**, _task_ is **component**. 49 | In addition to simplifying the coding complexity, the established framework can standardize the original chaotic and intricate writing method, 50 | and coding errors can be easily detected, which will greatly enhance the **readability**, **robustness** and **scalability** of the program. 51 | 52 | In Reflow basic logic, a complex business should first be broken down into a series of **single-function**, **no-blocking**, **single-threaded** task sets and packaged in `Trait` that explicitly define the attributes of the task. 53 | Dependencies are then built and committed using `Dependency`, and a working `reflow` object is finally obtained, which is started and the task flow can be executed. 54 | 55 | 56 | ## Features 57 | 58 | - The intelligent dependency analysis algorithm based on **keyword** and **value-type** can analyze errors in task assembly stage, ensuring that there will not be keyword missing or value-type mismatch errors during task execution; 59 | - On-demand task loading mechanism based on **priority** and **estimated duration**. **On-demand** means that the task is put into the _priority-bucket_ to **wait for** to be executed when it is its turn to execute. 60 | Why wait? Because the current tasks to be executed in different task-flows will be put into the same priority-bucket, the existing tasks in the bucket will be sorted according to their priorities; 61 | - A task-flow can be requested to **browse** mode first and then **reinforce** mode. **Browse** mode allows data to be loaded quickly; 62 | - Tasks can be nested and assembled indefinitely; 63 | - The execution of each task is atomic, single-threaded, and you don't have to worry about thread synchronization problem of input and output, whether the task is in serial or parallel, and whether it is in 64 | [`Pulse`](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Pulse.scala) or not; 65 | - Event feedback can be specified to the thread. For example: UI thread; 66 | - If subsequent tasks do not depend on the output of a task, the output is discarded, allowing memory to be used efficiently; 67 | - Convenient **synchronous/asynchronous** mode switching: 68 | > 69 | ```Scala 70 | // Asynchronous by default 71 | reflow.start(input, feedback) 72 | // To change to sync, simply add 'sync()' to the end. 73 | reflow.start(input, feedback).sync() 74 | ``` 75 | - Task execution progress can be strategically feed back (from which we consider the execution progress is important): 76 | > 77 | ```Scala 78 | // `Progress.Strategy.Xxx` e.g. 79 | implicit lazy val strategy: Strategy = Strategy.Depth(3) -> Strategy.Fluent -> Strategy.Interval(600) 80 | ``` 81 | - Deterministic classification of exceptions: 82 | > 83 | ```Scala 84 | // Feedback.scala 85 | def onFailed(trat: Trait, e: Exception): Unit = { 86 | e match { 87 | // There are two categories: 88 | // 1. `RuntimeException` caused by customer code quality issues such as `NullPointerException` are wrapped in `CodeException`, 89 | // The specific exception object can be retrieved through the `CodeException#getCause()` method. 90 | case c: CodeException => c.getCause 91 | // 2. Custom `Exception`, which is explicitly passed to the 'Task#failed(Exception)' method parameter, possible be 92 | // `null` (although Scala considers `null` to be low level). 93 | case e: Exception => e.printStackTrace() 94 | } 95 | } 96 | ``` 97 | - Clever interrupt strategy; 98 | - Global precise tasks monitoring and management: 99 | > 100 | ```Scala 101 | Reflow.GlobalTrack.registerObserver(new GlobalTrackObserver { 102 | override def onUpdate(current: GlobalTrack, items: All): Unit = { 103 | if (!current.isSubReflow && current.scheduler.getState == State.EXECUTING) { 104 | println(s"++++++++++[[[current.state:${current.scheduler.getState}") 105 | items().foreach(println) 106 | println("----------]]]") 107 | } 108 | } 109 | })(Strategy.Interval(600), null) 110 | ``` 111 | - Optimized configurable thread pool: If tasks continues scheduling to the thread pool, increase the size of threads to the `configured maximum`, then enqueue, idle release threads until `core size`; 112 | - Thread implementation without blocking (lock-free) efficient utilization: 113 | > There is no code for `future.get()` mechanism; 114 | > This requires no blocking in the user-defined tasks (if there is a block, you can split the task into multiple dependent but non-blocking tasks, except for network requests). 115 | 116 | These features greatly meet the practical requirements of various projects. 117 | 118 | 119 | ### _1.1 Related_ 120 | 121 | The main features of this framework are similar to [Facebook Bolts](http://github.com/BoltsFramework/Bolts-Android) and [RxJava](https://github.com/ReactiveX/RxJava), can be seen as fine-grained extensions 122 | of their task-combination capabilities, but it's more intuitive to use, more rigorous, high-spec, and closely aligned with actual project needs. 123 | 124 | This framework is implemented based on the **thread-pool** (`java.util.concurrent.ThreadPoolExecutor`) instead of the **Fork-Join** framework (`java.util.concurrent.ForkJoinPool`), and improved the former 125 | (see [Worker](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Worker.scala#L59)) to conform to the basic logic of 126 | _**increase the size of threads to [maximum](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/Config.scala#L27) first, or else enqueue, release thread when idle**_. 127 | The latter is suitable for computationally intensive tasks, but is not suitable for the design objectives of this framework, and is not suitable for resource-constrained devices (e.g. mobile phones, etc). 128 | 129 | 130 | ### _1.2 Instruction_ 131 | 132 | This framework is completely written in Scala language, and all parameters support **[shorthand](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/ReflowSpec.scala#L136)**, will be automatically escaped as 133 | **needed** ([implicit](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/reflow/implicits.scala)), can be used on any platform that adopts **jvm-like** (e.g. Android Runtime). 134 | 135 | This framework is based on a special **anti-blocking** thread synchronization tool [Snatcher](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/tool/Snatcher.scala), see _**[why snatcher](why-snatcher.md)**_ for details. 136 | 137 | * Note: This framework does not use `java.util.concurrent.Future` tools to handle parallel tasks, since it is implemented based on the **thread blocking** model, it does not meet the design goals of this framework: **lock-free**. 138 | 139 | 140 | ## How Reflow works 141 | ![Reflow operation principle diagram](https://github.com/bdo-cash/reflow/blob/master/op-principle-diagram.png "Reflow operation principle diagram") 142 | 143 | 144 | ## Start using Reflow 145 | 146 | ### _3.1 Dependencies_ 147 | 148 | Please click [![Jitpack](https://jitpack.io/v/bdo-cash/reflow.svg)](https://jitpack.io/#bdo-cash/reflow) 149 | 150 | ### _3.2 Example_ 151 | 152 | See _below_ or [LiteSpec](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/LiteSpec.scala), and [~~ReflowSpec~~](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/ReflowSpec.scala). 153 | 154 | * If using on Android platform, please make the following settings first. 155 | 156 | ```Scala 157 | class App extends AbsApp { 158 | override def onCreate(): Unit = { 159 | App.reflow.init() 160 | super.onCreate() 161 | } 162 | } 163 | 164 | object App { 165 | object implicits { 166 | // Feedback strategy for task-flow execution progress 167 | implicit lazy val strategy: Strategy = Strategy.Depth(3) -> Strategy.Fluent -> Strategy.Interval(600) 168 | implicit lazy val poster: Poster = new Poster { 169 | // Post feedback to the UI thread 170 | override def post(runner: Runnable): Unit = getApp.mainHandler.post(runner) 171 | } 172 | } 173 | 174 | object reflow { 175 | private[App] def init(): Unit = { 176 | Reflow.setThreadResetor(new ThreadResetor { 177 | override def reset(thread: Thread, runOnCurrentThread: Boolean): Unit = { 178 | if (runOnCurrentThread) Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND) 179 | } 180 | }) 181 | } 182 | } 183 | } 184 | ``` 185 | 186 | * Below is part of [LiteSpec](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/LiteSpec.scala), which is very concise and recommended: 187 | 188 | ```Scala 189 | // ... 190 | Scenario("`Serial/Parallel` Tasks mixed assembly") { 191 | val pars = 192 | ( 193 | c2d 194 | +>> 195 | c2abc.inPar("name#c2abc", "`Serial` mix in `Parallel`") 196 | +>> 197 | (c2b >>> b2c >>> c2a >>> a2b) //.inPar("xxx") is optional. 198 | +>> 199 | c2a 200 | ) **> { (d, c, b, a, ctx) => 201 | info(a.toString) 202 | info(b.toString) 203 | info(c.toString) 204 | info(d.toString) 205 | d 206 | } 207 | Input[Aaa] >>> a2b >>> b2c >>> pars run(new Aaa) sync() 208 | 209 | assert(true) 210 | } 211 | 212 | // `Pulse`, more complex: 213 | Scenario("Multi-level nested assembly test of `Pulse`") { 214 | val pars = { 215 | def p = (c2d +>> c2b) **> { (d, _, ctx) => ctx.progress(1, 2); d } +|- { (_, d) => d } 216 | @tailrec 217 | def loop(s: () => Serial[Ccc, Ddd], times: Int = 0, limit: Int = 10): Serial[Ccc, Ddd] = 218 | if (times >= limit) s() 219 | else { 220 | def p = 221 | ( 222 | s() +|- { (_, d) => d } 223 | +>> 224 | s() 225 | +>> 226 | s() 227 | ) **> { (d, _, _, ctx) => ctx.progress(1, 2); d } 228 | loop(() => p, times + 1, limit) 229 | } 230 | loop(() => p, limit = 3) 231 | } 232 | 233 | @volatile var callableOut: Int = 0 234 | val future = new FutureTask[Int](() => callableOut) 235 | 236 | val repeatCount = 1000 // Int.MaxValue 237 | 238 | val pulse = (Input[Aaa] >>> a2b >>> b2c >>> pars) pulse (new reflow.Pulse.Feedback.Lite[Ddd] { 239 | override def onStart(serialNum: Long): Unit = {} 240 | // ... 241 | } 242 | // ... 243 | ``` 244 | 245 | * Additional, refer to [ReflowSpec](https://github.com/bdo-cash/reflow/blob/master/src/test/scala/reflow/test/ReflowSpec.scala), which is verbose and no longer recommended. 246 | 247 | -------------------------------------------------------------------------------- /author.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bdo-cash/reflow/6cb1c43a7322bf3d6489b3c36d4d86314b72136e/author.jpg -------------------------------------------------------------------------------- /build.sbt: -------------------------------------------------------------------------------- 1 | name := baseDirectory.value.getName 2 | 3 | organization := "hobby.wei.c" 4 | 5 | version := "3.0.7" 6 | 7 | scalaVersion := "2.11.12" 8 | 9 | crossScalaVersions := Seq( 10 | "2.11.12", 11 | "2.12.17" 12 | ) 13 | 14 | javacOptions ++= Seq("-source", "1.7", "-target", "1.7") 15 | // 启用对 java8 lambda 语法的支持。 16 | scalacOptions += "-Xexperimental" 17 | 18 | // 解决生成文档报错导致 jitpack.io 出错的问题。 19 | publishArtifact in packageDoc := false 20 | 21 | // 独立使用本库的话，应该启用下面的设置。 22 | //lazy val scalaSettings = Seq( 23 | // scalaVersion := "2.11.12" 24 | //) 25 | // 26 | //lazy val root = Project(id = "reflow", base = file(".")) 27 | // .dependsOn(/*lang*/) 28 | // .settings(scalaSettings, 29 | // aggregate in update := false 30 | // ) 31 | 32 | exportJars := true 33 | 34 | offline := true 35 | 36 | // 如果要用 jitpack 打包的话就加上，打完了再注掉。 37 | resolvers += "jitpack" at "https://jitpack.io" 38 | 39 | libraryDependencies ++= Seq( 40 | // 如果要用 jitpack 打包的话就加上，打完了再注掉。 41 | // 如果独立使用本库，应该启用本依赖。 42 | "com.github.bdo-cash" % "annoguard" % "v1.0.5-beta", 43 | // 如果用 jitpack 打包 2.12.12, 这个包的引入也必须是 2.12.12。 44 | "com.github.bdo-cash" % "scala-lang" % "138bff0c11", 45 | // ScalaTest 的标准引用 46 | "junit" % "junit" % "[4.12,)" % Test, 47 | // `3.2.0-SNAP10`会导致`scala.ScalaReflectionException: object org.scalatest.prop.Configuration$ not found`. 48 | "org.scalatest" %% "scalatest" % "3.2.0-SNAP7" % Test 49 | ) 50 | 51 | // 如果项目要独立编译，请同时启用这部分。 52 | // Macro Settings 53 | ///* 54 | resolvers += Resolver.sonatypeRepo("releases") 55 | addCompilerPlugin("org.scalamacros" % "paradise" % "[2.1.0,)" cross CrossVersion.full) 56 | // https://mvnrepository.com/artifact/org.scala-lang/scala-compiler 57 | libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value 58 | //*/ 59 | -------------------------------------------------------------------------------- /op-principle-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bdo-cash/reflow/6cb1c43a7322bf3d6489b3c36d4d86314b72136e/op-principle-diagram.png -------------------------------------------------------------------------------- /project/build.properties: -------------------------------------------------------------------------------- 1 | sbt.version=0.13.18 2 | -------------------------------------------------------------------------------- /project/plugins.sbt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bdo-cash/reflow/6cb1c43a7322bf3d6489b3c36d4d86314b72136e/project/plugins.sbt -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/reflow/adapt/task.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2022-present, Chenai Nakam(chenai.nakam@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow.adapt; 18 | 19 | import hobby.wei.c.reflow.Task.Context; 20 | import hobby.wei.c.reflow.implicits; 21 | import hobby.wei.c.reflow.lite.Input; 22 | import hobby.wei.c.reflow.lite.Lite; 23 | import hobby.wei.c.reflow.lite.Serial; 24 | import hobby.wei.c.reflow.lite.Task; 25 | import scala.reflect.ClassTag; 26 | import scala.reflect.ClassTag$; 27 | import scala.runtime.AbstractFunction1; 28 | import scala.runtime.AbstractFunction2; 29 | import scala.runtime.AbstractFunction3; 30 | 31 | /** 32 | * @author Chenai Nakam(chenai.nakam@gmail.com) 33 | * @version 1.0, 05/06/2022 34 | */ 35 | public class task { 36 | 37 | // TODO: 38 | // Java 调用示例： 39 | Serial e_g_() { 40 | Input input = new Input(ClassTag$.MODULE$.apply(String.class)); 41 | 42 | Lite cut = implicits.$plus$bar$minus(new AbstractFunction1() { 43 | @Override 44 | public Integer apply(String v1) { 45 | return Integer.parseInt(v1); 46 | } 47 | }, ClassTag$.MODULE$.apply(String.class), ClassTag$.MODULE$.apply(Integer.class)); 48 | 49 | Lite lite = Task.apply(implicits.TRANSIENT(), implicits.P_HIGH(), "", "", true, new AbstractFunction2() { 50 | @Override 51 | public Integer apply(String v1, Context v2) { 52 | return null; 53 | } 54 | }, ClassTag$.MODULE$.apply(String.class), ClassTag$.MODULE$.apply(Integer.class)); 55 | 56 | ClassTag tagIn = ClassTag$.MODULE$.apply(String.class); 57 | ClassTag tagOut = ClassTag$.MODULE$.apply(Integer.class); 58 | return input.next(implicits.lite2Par(cut, tagIn, tagOut).$plus$greater$greater(lite, tagOut).$times$times$greater(new AbstractFunction3() { 59 | @Override 60 | public Integer apply(Integer v1, Integer v2, Context v3) { 61 | return null; 62 | } 63 | }, tagOut), tagOut); 64 | } 65 | 66 | public static void main(String[] args) { 67 | new task().e_g_().pulse(/* 68 | TODO: 还无法适配！ 69 | new Pulse.Feedback.Lite(0) { 70 | @Override 71 | public void liteOnComplete(long serialNum, Option value) { 72 | } 73 | }*/ null, false, 128, 3, false, null, null); 74 | } 75 | } 76 | -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/reflow/step/Step.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow.step; 18 | 19 | import java.util.Objects; 20 | import java.util.Set; 21 | 22 | /** 23 | * @author Wei Chou(weichou2010@gmail.com) 24 | * @version 1.0, 03/10/2016 25 | */ 26 | public class Step, T> { 27 | public final Step prev; 28 | public final Unit unit; 29 | 30 | private Step(Step prev, Unit unit) { 31 | this.prev = prev; 32 | this.unit = unit; 33 | } 34 | 35 | public final OUTPUT exec(Callback callback) throws Exception { 36 | return exec(this, unit.newTracker(), callback); 37 | } 38 | 39 | public final Step then(Unit unit) { 40 | return next(this, unit); 41 | } 42 | 43 | public static , X> Step begin(Unit unit) { 44 | return next(null, unit); 45 | } 46 | 47 | private static , X> Step next(Step prev, Unit unit) { 48 | return new Step<>(prev, Objects.requireNonNull(unit)); 49 | } 50 | 51 | private static , X> O exec(Step step, T tracker, Callback callback) throws Exception { 52 | final O output; 53 | if (step == null) { 54 | tracker.onTopReached(); 55 | output = null; 56 | } else { 57 | tracker.increaseCount(); 58 | tracker.pushRequireKeys(step.unit.requireKeys()); 59 | tracker.pushOutKeys(step.unit.outKeys()); 60 | final I prev = exec(step.prev, tracker, callback); 61 | tracker.popOutKeys(); 62 | tracker.popRequireKeys(); 63 | if (callback != null) { 64 | callback.onProgress(tracker.progress(), tracker.count()); 65 | } 66 | step.unit.setTracker(tracker); 67 | final O curr = step.unit.exec$(prev == null ? null : step.prev/*maybe null*/.unit.clone(prev)); 68 | tracker.increaseProgress(); 69 | if (callback != null && tracker.progress() == tracker.count()) { 70 | callback.onProgress(tracker.progress(), tracker.count()); 71 | } 72 | final Set requireKeys = tracker.requireKeys(); 73 | output = prev == null 74 | ? curr == null ? null : step.unit.trim(curr, requireKeys) 75 | : curr == null ? step.unit.trim(step.unit.join(prev), requireKeys) 76 | : step.unit.trim(step.unit.join(prev, curr), requireKeys); 77 | assert step.unit.outKeys(output).containsAll(requireKeys); 78 | } 79 | return output; 80 | } 81 | 82 | public interface Callback { 83 | void onProgress(int progress, int count); 84 | } 85 | } 86 | -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/reflow/step/Tracker.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow.step; 18 | 19 | import java.util.HashSet; 20 | import java.util.Set; 21 | import java.util.Stack; 22 | import java.util.concurrent.atomic.AtomicBoolean; 23 | import java.util.concurrent.atomic.AtomicInteger; 24 | 25 | /** 26 | * @author Wei Chou(weichou2010@gmail.com) 27 | * @version 1.0, 05/10/2016 28 | */ 29 | public class Tracker { 30 | private final AtomicInteger count = new AtomicInteger(0); 31 | private final AtomicInteger progress = new AtomicInteger(0); 32 | private final AtomicBoolean top = new AtomicBoolean(false); 33 | private Stack> stackRequires = new Stack<>(); 34 | private Stack> stackOuts = new Stack<>(); 35 | 36 | final void increaseCount() { 37 | count.incrementAndGet(); 38 | } 39 | 40 | /** 41 | * {@link Unit#exec$(Object)}被调用时, {@link #topReached()}必定为true. 42 | * 43 | * @return 44 | */ 45 | public final int count() { 46 | return count.get(); 47 | } 48 | 49 | final void onTopReached() { 50 | top.set(true); 51 | } 52 | 53 | final boolean topReached() { 54 | return top.get(); 55 | } 56 | 57 | final void increaseProgress() { 58 | progress.incrementAndGet(); 59 | } 60 | 61 | public final int progress() { 62 | return progress.get(); 63 | } 64 | 65 | public final int keyStep() { 66 | return topReached() ? count() - progress() : count(); 67 | } 68 | 69 | final void pushRequireKeys(Set set) { 70 | stackRequires.push(set); 71 | } 72 | 73 | final void pushOutKeys(Set set) { 74 | stackOuts.push(set); 75 | } 76 | 77 | final void popRequireKeys() { 78 | stackRequires.pop(); 79 | } 80 | 81 | final void popOutKeys() { 82 | stackOuts.pop(); 83 | } 84 | 85 | final Set requireKeys() { 86 | final Set set = new HashSet<>(); 87 | for (int i = 0, len = stackRequires.size(); i < len; i++) { 88 | set.removeAll(stackOuts.get(i)); 89 | set.addAll(stackRequires.get(i)); 90 | } 91 | return set; 92 | } 93 | } 94 | -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/reflow/step/Unit.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow.step; 18 | 19 | import java.util.Collections; 20 | import java.util.Set; 21 | 22 | /** 23 | * @author Wei Chou(weichou2010@gmail.com) 24 | * @version 1.0, 03/10/2016 25 | */ 26 | public abstract class Unit, T> { 27 | private TRACKER tracker; 28 | 29 | final OUTPUT exec$(INPUT input) throws Exception { 30 | return input == null ? def() : exec(input); 31 | } 32 | 33 | protected abstract OUTPUT exec(INPUT input) throws Exception; 34 | 35 | protected abstract OUTPUT def() throws Exception; 36 | 37 | public OUTPUT clone(OUTPUT output) { 38 | return output; 39 | } 40 | 41 | final void setTracker(TRACKER tracker) { 42 | this.tracker = tracker; 43 | } 44 | 45 | protected final TRACKER tracker() { 46 | return tracker; 47 | } 48 | 49 | protected abstract TRACKER newTracker(); 50 | 51 | protected Set requireKeys() { 52 | return Collections.emptySet(); 53 | } 54 | 55 | protected Set outKeys() { 56 | return Collections.emptySet(); 57 | } 58 | 59 | protected Set outKeys(OUTPUT output) { 60 | return Collections.emptySet(); 61 | } 62 | 63 | protected OUTPUT join(INPUT prev) { 64 | return null; 65 | } 66 | 67 | protected OUTPUT join(INPUT prev, OUTPUT output) { 68 | return output; 69 | } 70 | 71 | protected OUTPUT trim(OUTPUT output, Set requireKeys) { 72 | return output; 73 | } 74 | 75 | public final Step then(Unit unit) { 76 | return Step.begin(this).then(unit); 77 | } 78 | } 79 | -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/reflow/step/UnitM.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow.step; 18 | 19 | import java.util.HashMap; 20 | import java.util.HashSet; 21 | import java.util.Map; 22 | import java.util.Set; 23 | 24 | /** 25 | * @author Wei Chou(weichou2010@gmail.com) 26 | * @version 1.0, 05/10/2016 27 | */ 28 | public abstract class UnitM extends Unit, Map, Tracker, String> { 29 | @Override 30 | public Map clone(Map map) { 31 | return new HashMap<>(map); 32 | } 33 | 34 | @Override 35 | protected Set outKeys(Map map) { 36 | return map.keySet(); 37 | } 38 | 39 | @Override 40 | protected Map join(Map prev) { 41 | return prev; 42 | } 43 | 44 | @Override 45 | protected Map join(Map prev, Map map) { 46 | if (prev == null || prev.isEmpty()) { 47 | return map; 48 | } else { 49 | prev.putAll(map); 50 | return prev; 51 | } 52 | } 53 | 54 | @Override 55 | protected Map trim(Map map, Set requireKeys) { 56 | for (String k : new HashSet<>(map.keySet())) { 57 | if (!requireKeys.contains(k)) { 58 | map.remove(k); 59 | } 60 | } 61 | return map; 62 | } 63 | 64 | @Override 65 | protected Tracker newTracker() { 66 | return new Tracker<>(); 67 | } 68 | } 69 | -------------------------------------------------------------------------------- /src/main/java/hobby/wei/c/tool/Reflect.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.tool; 18 | 19 | import java.lang.reflect.Array; 20 | import java.lang.reflect.GenericArrayType; 21 | import java.lang.reflect.ParameterizedType; 22 | import java.lang.reflect.Type; 23 | import java.lang.reflect.TypeVariable; 24 | import java.lang.reflect.WildcardType; 25 | import java.util.Locale; 26 | 27 | /** 28 | * 摘自Gson源码。 29 | * 30 | * @author Wei Chou(weichou2010@gmail.com) 31 | * @version 1.0, 02/07/2016 32 | */ 33 | public class Reflect { 34 | public static Type[] getSuperclassTypeParameter(Class subclass, boolean checkTypeVariable) { 35 | final Type superclass = subclass.getGenericSuperclass(); 36 | if (superclass instanceof Class) { 37 | return throwClassIllegal(subclass); 38 | } else { 39 | final Type[] types = ((ParameterizedType) superclass).getActualTypeArguments(); 40 | if (checkTypeVariable) { 41 | for (Type t : types) { 42 | if (t instanceof TypeVariable) { 43 | throwClassIllegal(subclass); 44 | } 45 | } 46 | } 47 | return types; 48 | } 49 | } 50 | 51 | public static Class getRawType(Type type) { 52 | if (type instanceof Class) { 53 | return (Class) type; 54 | } else if (type instanceof ParameterizedType) { 55 | return (Class) ((ParameterizedType) type).getRawType(); 56 | } else if (type instanceof GenericArrayType) { 57 | return Array.newInstance(getRawType(((GenericArrayType) type).getGenericComponentType()), 0).getClass(); 58 | } else if (type instanceof TypeVariable) { 59 | return Object.class; 60 | } else if (type instanceof WildcardType) { 61 | return getRawType(((WildcardType) type).getUpperBounds()[0]); 62 | } else { 63 | return throwTypeIllegal(type); 64 | } 65 | } 66 | 67 | public static Type[] getSubTypes(Type type) { 68 | if (type instanceof ParameterizedType) { 69 | return ((ParameterizedType) type).getActualTypeArguments(); 70 | } else { 71 | return EMPTY_TYPES; 72 | } 73 | } 74 | 75 | private static Type[] throwClassIllegal(Class subclass) { 76 | throw new IllegalArgumentException(String.format( 77 | Locale.CHINA, "请确保类\"%s\"是泛型类的[匿名]子类。", subclass.getName())); 78 | } 79 | 80 | private static Class throwTypeIllegal(Type type) { 81 | throw new IllegalArgumentException(String.format( 82 | // Locale.CHINA, "不支持的类型\"%s\", 仅支持Class, ParameterizedType, or GenericArrayType.", type)); 83 | Locale.CHINA, "不支持的类型\"%s\", 请确认泛型参数是否正确。", type)); 84 | } 85 | 86 | private static final Type[] EMPTY_TYPES = new Type[0]; 87 | } 88 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Assist.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.basis.TAG.LogTag 21 | import hobby.chenai.nakam.lang.J2S._ 22 | import hobby.chenai.nakam.lang.TypeBring.AsIs 23 | import hobby.chenai.nakam.tool.pool.S._2S 24 | import hobby.wei.c.anno.proguard.Burden 25 | import hobby.wei.c.reflow.Dependency.IsPar 26 | import hobby.wei.c.reflow.Reflow.{logger => log, _} 27 | import hobby.wei.c.reflow.Trait.ReflowTrait 28 | import java.util 29 | import java.util.concurrent.ThreadPoolExecutor 30 | import scala.annotation.tailrec 31 | import scala.collection._ 32 | import scala.util.control.Breaks._ 33 | 34 | /** 35 | * @author Wei Chou(weichou2010@gmail.com) 36 | * @version 1.0, 02/07/2016 37 | */ 38 | object Assist { 39 | def between(min: Float, value: Float, max: Float) = min max value min max 40 | 41 | @Burden 42 | def assertx(b: Boolean): Unit = assertf(b, "", force = false) 43 | 44 | @Burden 45 | def assertx(b: Boolean, msg: => String): Unit = assertf(b, msg, force = false) 46 | def assertf(b: Boolean): Unit = assertf(b, "", force = true) 47 | def assertf(b: Boolean, msg: => String): Unit = assertf(b, msg, force = true) 48 | private def assertf(b: Boolean, msg: => String, force: Boolean = true): Unit = if ((force || debugMode) && !b) Throws.assertError(msg) 49 | 50 | def requireNonEmpty(s: String): String = { assertf(s.nonNull && s.nonEmpty); s } 51 | 52 | def requireElemNonNull[C <: Set[_ <: AnyRef]](col: C): C = { 53 | if (debugMode) col.seq.foreach(t => assertf(t.nonNull, "元素不能为null.")) 54 | col 55 | } 56 | 57 | def requireTaskNameDiffAndUpdate(trat: Trait, names: mutable.Set[String]): Unit = { 58 | val name = trat.name$ 59 | if (names.contains(name)) Throws.sameName(name) 60 | names.add(name) 61 | } 62 | 63 | def requirePulseKeyDiffAndUpdate(depth: Int, trat: Trait, parent: Option[ReflowTrait], keySet: mutable.Set[(Int, String, String)]): Unit = { 64 | val key = resolveKey(depth, trat, parent) 65 | if (keySet.contains(key)) Throws.sameKeyPulse(key) 66 | keySet.add(key) 67 | } 68 | 69 | def requirePulseKeyDiff(reflow: Reflow): Reflow = { 70 | val keySet = new mutable.HashSet[(Int, String, String)] 71 | def loop(trat: Trait, parent: Option[ReflowTrait] = None, depth: Int = 0) { 72 | if (trat.is4Reflow) { // `Tracker`中已忽略`is4Reflow`的`Pulse.Interact`交互，但保险起见，还是加上，看看有没可能与其它非`is4Reflow`碰撞。 73 | requirePulseKeyDiffAndUpdate(depth, trat, parent, keySet) 74 | trat.asSub.reflow.basis.traits.foreach(loop(_, Some(trat.asSub), depth + 1)) 75 | } else if (trat.isPar) trat.asPar.traits().foreach(loop(_, parent, depth)) 76 | else requirePulseKeyDiffAndUpdate(depth, trat, parent, keySet) 77 | } 78 | reflow.basis.traits.foreach(loop(_)) 79 | reflow 80 | } 81 | 82 | def resolveKey(depth: Int, trat: Trait, parent: Option[ReflowTrait]): (Int, String, String) = (depth, trat.name$, parent.map(_.name$).orNull) 83 | 84 | /** 为`key`生成可比较大小的序列号。目前仅支持最多 10 层，每层最多 63 步。满足`走得越远的越大`，即：层级 depth 越低，step 越大的就越大。 85 | * @param which 需要满足与`pulse.serialNum`的大小顺序一致。目前仅支持`0, 1, 2, 3`四个数字。 86 | * @return `> 0`，如果找到了`key`所在的 step（即：`key`有效），`-1`，其它情况。 87 | */ 88 | @deprecated("Has limit.") 89 | def genIncrSerialNum(reflow: Reflow, key: (Int, String, String), which: Int)(implicit tag: LogTag): Long = { 90 | val opt = findKeyStep(key, reflow) 91 | //log.i("[genSerialNum]succeed:%s, lis:%s.", b, lis.map { case (depth, step) => s"(depth:$depth, step:$step)" }.mkString$.s) 92 | if (opt.isDefined) { 93 | val lis = opt.get 94 | // 层级序号是固定的，那就把每一层的 step 依次排列放进 1000000000000000000L 中。 95 | // ((depth:0, step:2), (depth:1, step:0), (depth:2, step:0), (depth:3, step:0), (depth:4, step:0), (depth:5, step:0), (depth:6, step:0), (depth:7, step:0)). 96 | // 支持最多 10 层，每层最多 63 步。 97 | val bits = java.lang.Long.numberOfLeadingZeros(64) 98 | lis.sortBy(_._1).map { case (depth, step) => (step + 1).toLong << (bits - 1 - (64 - bits - 1) * depth) }.sum + which 99 | } else -1 100 | } 101 | 102 | /** @return `(depth, step)`的列表。 */ 103 | def findKeyStep(key: (Int, String, String), reflow: Reflow): Option[List[(Int, Int)]] = { 104 | @tailrec 105 | def stepForKey(r: Reflow, step: Int, parent: Option[ReflowTrait], depth: Int): (Boolean, Int) = { 106 | if (step >= r.basis.steps()) (false, -1) 107 | else { 108 | val trat = r.basis.traits(step) 109 | //if (trat.is4Reflow) // 不向下一层寻找 110 | if (trat.isPar) { 111 | var result = (false, -1) 112 | breakable { 113 | trat.asPar.traits().foreach { t => 114 | if (resolveKey(depth, t, parent) == key) { 115 | result = (true, step); break 116 | } 117 | } 118 | } 119 | if (result._1) result else stepForKey(r, step + 1, parent, depth) 120 | } else if (resolveKey(depth, trat, parent) == key) (true, step) 121 | else stepForKey(r, step + 1, parent, depth) 122 | } 123 | } 124 | def findSub(r: Reflow, step: Int, depth: Int): Seq[(ReflowTrait, Int)] = { 125 | val trat = r.basis.traits(step) 126 | if (trat.isPar) trat.asPar.traits().flatMap { sub => if (sub.is4Reflow) Some((sub.asSub, depth + 1)) else None } 127 | else if (trat.is4Reflow) (trat.asSub, depth + 1) :: Nil 128 | else Nil 129 | } 130 | def findStep(r: Reflow, step: Int = 0, parent: Option[ReflowTrait] = None, depth: Int = 0): (Boolean, List[(Int, Int)]) = { 131 | if (depth == key._1) { 132 | val (b, s) = stepForKey(r, 0, parent, depth) 133 | (b, (depth, s) :: Nil) 134 | } else if (depth < key._1) { 135 | var result = (false, Nil.as[List[(Int, Int)]]); var step1 = step 136 | breakable { 137 | while (step1 < r.basis.steps()) { 138 | findSub(r, step1, depth).foreach { case (s, d) => 139 | result = findStep(s.reflow, 0, Some(s), d) 140 | if (result._1) break 141 | }; step1 += 1 142 | } 143 | } 144 | (result._1, (depth, step1) :: result._2) 145 | } else (false, Nil) 146 | } 147 | val (b, lis) = findStep(reflow) 148 | if (b) Some(lis) else None 149 | } 150 | 151 | /** 152 | * 由于{@link Key$#equals(Object)}是比较了所有参数，所以这里还得重新检查。 153 | */ 154 | def requireKkDiff[C <: Iterable[KvTpe[_ <: AnyRef]]](keys: C): C = { 155 | if (/*debugMode &&*/ keys.nonEmpty) { 156 | val ks = new util.HashSet[String] 157 | for (k <- keys.seq) { 158 | if (ks.contains(k.key)) Throws.sameKey$k(k) 159 | ks.add(k.key) 160 | } 161 | } 162 | keys 163 | } 164 | 165 | /** 166 | * 要求相同的输入key的type也相同，且不能有相同的输出k.key。 167 | */ 168 | def requireTransInTpeSame$OutKDiff[C <: Set[Transformer[_ <: AnyRef, _ <: AnyRef]]](tranSet: C): C = { 169 | if (/*debugMode &&*/ tranSet.nonEmpty) { 170 | val map = new mutable.AnyRefMap[String, Transformer[_ <: AnyRef, _ <: AnyRef]]() 171 | for (t <- tranSet) { 172 | if (map.contains(t.in.key)) { 173 | val trans = map(t.in.key) 174 | if (t.in != trans.in) Throws.tranSameKeyButDiffType(t.in, trans.in) 175 | } else { 176 | map.put(t.in.key, t) 177 | } 178 | } 179 | requireKkDiff(tranSet.map(_.out)) 180 | } 181 | tranSet 182 | } 183 | 184 | def eatExceptions(work: => Unit)(implicit tag: LogTag) { 185 | try { work } 186 | catch { case e: Exception => log.w(e, "eatExceptions.") } 187 | } 188 | 189 | private[reflow] object Throws { 190 | def sameName(name: String) = throw new IllegalArgumentException(s"任务队列中不可以有相同的任务名称。名称为`$name`的 Task 已存在, 请检查。建议把 lite.Task 定义的 val 改为 def（如果用 lite 库的话），或尝试重写其 Trait.name() 方法。") 191 | def sameKeyPulse(key: (Int, String, String)) = throw new IllegalArgumentException(s"Pulse 中不可以有`层级、任务名称、父任务名称`三者都相同的组。组名称为`$key`的 Task 已存在, 请检查。建议把 lite.Task 定义的 val 改为 def（如果用 lite 库的话），或尝试重写其 Trait.name() 方法。") 192 | def sameOutKeyParallel(kvt: KvTpe[_ <: AnyRef], trat: Trait) = throw new IllegalArgumentException(s"并行的任务不可以有相同的输出。key: `${kvt.key}`, Task: `${trat.name$}`。") 193 | def sameCacheKey(kvt: KvTpe[_ <: AnyRef]) = throw new IllegalArgumentException(s"Task.cache(key, value) 不可以和与该 Task 相关的 Trait.requires() 有相同的 key: `${kvt.key}`。") 194 | def sameKey$k(kvt: KvTpe[_ <: AnyRef]) = throw new IllegalArgumentException(s"集合中的 KvTpe.key 不可以重复: `$kvt`。") 195 | def lackIOKey(kvt: KvTpe[_ <: AnyRef], in$out: Boolean) = throw new IllegalStateException(s"缺少${if (in$out) "输入" else "输出"}参数: `$kvt`。") 196 | def lackOutKeys() = throw new IllegalStateException("所有任务的输出都没有提供最终输出, 请检查。") 197 | def typeNotMatch(kvt: KvTpe[_ <: AnyRef], clazz: Class[_]) = throw new IllegalArgumentException(s"key 为`${kvt.key}`的参数值类型与定义不一致: 应为`${kvt.tpe}`, 实际为`$clazz`。") 198 | def typeNotMatch4Trans(from: KvTpe[_ <: AnyRef], to: KvTpe[_ <: AnyRef]) = typeNotMatch(to, from, "转换。") 199 | def typeNotMatch4Consume(from: KvTpe[_ <: AnyRef], to: KvTpe[_ <: AnyRef]) = typeNotMatch(to, from, "消化需求。") 200 | def typeNotMatch4Required(from: KvTpe[_ <: AnyRef], to: KvTpe[_ <: AnyRef]) = typeNotMatch(to, from, "新增初始输入。") 201 | def typeNotMatch4RealIn(from: KvTpe[_ <: AnyRef], to: KvTpe[_ <: AnyRef]) = typeNotMatch(to, from, "实际输入。") 202 | private def typeNotMatch(from: KvTpe[_ <: AnyRef], to: KvTpe[_ <: AnyRef], opt: String) = throw new IllegalArgumentException(s"赋值类型不匹配: `${to.tpe}` but `${from.tpe}`. 操作: `$opt`。") 203 | def tranSameKeyButDiffType(one: KvTpe[_ <: AnyRef], another: KvTpe[_ <: AnyRef]) = throw new IllegalArgumentException(s"多个转换使用同一输入key但类型不一致: key: `${one.key}`, types: `${one.tpe}`、`${another.tpe}`。") 204 | def assertError(msg: String) = throw new AssertionError(msg) 205 | } 206 | 207 | private[reflow] object Monitor extends TAG.ClassName { 208 | private def tag(name: String): LogTag = new LogTag(className + "/" + name) 209 | 210 | @Burden 211 | def duration(name: String, begin: Long, end: Long, period: Reflow.Period.Tpe): Unit = if (debugMode) { 212 | val duration = end - begin 213 | val avg = period.average(duration) 214 | if (avg == 0 || duration <= avg) { 215 | log.i("task:%s, period:%s, duration:%fs, average:%fs.", name.s, period, duration / 1000f, avg / 1000f)(tag("duration")) 216 | } else { 217 | log.w("task:%s, period:%s, duration:%fs, average:%fs.", name.s, period, duration / 1000f, avg / 1000f)(tag("duration")) 218 | } 219 | } 220 | 221 | @Burden 222 | def duration(reflow: TAG.ClassName, begin: Long, end: Long, state: State.Tpe, state$ : State.Tpe, subReflow: Boolean): Unit = if (!subReflow) 223 | if (debugMode) log.w("[Reflow Time Duration]>>>>>>>>>> duration:%fs, state:%s/%s <<<<<<<<<<.", (end - begin) / 1000f, state, state$)(reflow.className) 224 | 225 | @Burden 226 | def abortion(trigger: String, task: String, forError: Boolean): Unit = if (debugMode) log.i("trigger:%1$s, task:%2$s, forError:%3$s.", trigger.s, task.s, forError)(tag("abortion")) 227 | 228 | @Burden 229 | def assertStateOverride(prev: State.Tpe, state: State.Tpe, success: Boolean) { 230 | if (!success) { 231 | log.w("`state override` maybe illegal, IGNORED! prev:%s, state:%s.", prev, state)(tag("abortion")) 232 | // 允许`success = false`, 不用中止。 233 | //assertx(success) 234 | } 235 | } 236 | 237 | @Burden 238 | def complete(step: => Int, out: Out, flow: Out, trimmed: Out): Unit = if (debugMode) log.i("step:%d, out:%s, flow:%s, trimmed:%s.", step, out, flow, trimmed)(tag("complete")) 239 | 240 | @Burden 241 | def threadPool(pool: ThreadPoolExecutor, addThread: Boolean, reject: Boolean): Unit = if (debugMode) log.i( 242 | "{ThreadPool}%s, active/core/max:(%d/%d/%d), queueSize:%d, taskCount:%d, largestPool:%d.", 243 | if (reject) "reject runner".s else if (addThread) "add thread".s else "offer queue".s, 244 | pool.getActiveCount, 245 | pool.getPoolSize, 246 | pool.getMaximumPoolSize, 247 | pool.getQueue.size(), 248 | pool.getTaskCount, 249 | pool.getLargestPoolSize 250 | )(tag("threadPool")) 251 | 252 | def threadPoolError(t: Throwable): Unit = log.e(t)(tag("threadPoolError")) 253 | } 254 | 255 | class FailedException(t: Throwable) extends Exception(t: Throwable) 256 | class AbortException(t: Throwable) extends Exception(t: Throwable) 257 | class FailedError(t: Throwable) extends Error(t: Throwable) 258 | class AbortError(t: Throwable = null) extends Error(t: Throwable) 259 | class InnerError(t: Throwable) extends Error(t: Throwable) 260 | } 261 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/CodeException.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | /** 20 | * @author Wei Chou(weichou2010@gmail.com) 21 | * @version 1.0, 25/07/2016 22 | */ 23 | class CodeException(t: Throwable) extends Exception(t: Throwable) 24 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Config.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | /** 20 | * @author Wei Chou(weichou2010@gmail.com) 21 | * @version 1.0, 11/04/2016 22 | */ 23 | class Config protected () { 24 | require(maxPoolSize >= corePoolSize && corePoolSize > 0) 25 | 26 | lazy val corePoolSize = Config.CPU_COUNT + 1 27 | lazy val maxPoolSize = Config.CPU_COUNT * 5 + 1 28 | /** 29 | * 空闲线程保留时间。单位: 秒。 30 | */ 31 | lazy val keepAliveTime = 5 32 | } 33 | 34 | object Config { 35 | lazy val CPU_COUNT = Runtime.getRuntime.availableProcessors 36 | lazy val DEF = new Config 37 | 38 | lazy val SINGLE_THREAD = Config(1, 1) 39 | 40 | def apply(coreSize: => Int, maxSize: => Int, aliveTime: => Int = DEF.keepAliveTime): Config = new Config() { 41 | require(coreSize <= maxSize) 42 | require(aliveTime > 0) 43 | override lazy val corePoolSize = coreSize 44 | override lazy val maxPoolSize = maxSize 45 | override lazy val keepAliveTime = aliveTime 46 | } 47 | } 48 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Env.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2017-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.lang.J2S.NonNull 21 | import hobby.chenai.nakam.lang.TypeBring.AsIs 22 | import hobby.wei.c.reflow.Reflow.{debugMode, logger => log} 23 | import hobby.wei.c.reflow.Trait.ReflowTrait 24 | 25 | /** 26 | * @author Wei Chou(weichou2010@gmail.com) 27 | * @version 1.0, 31/01/2018 28 | */ 29 | private[reflow] trait Env extends TAG.ClassName { 30 | private[reflow] val trat: Trait 31 | private[reflow] val tracker: Tracker 32 | 33 | private[reflow] final lazy val input: Out = { 34 | val in = new Out(trat.requires$) 35 | in.fillWith(tracker.getPrevOutFlow) 36 | val cached = if (isPulseMode) if (tracker.isInput(trat) || trat.is4Reflow) None else tracker.pulse.getCache(depth, trat, parent) else myCache(create = false) 37 | if (cached.isDefined) in.cache(cached.get) 38 | in 39 | } 40 | private[reflow] final lazy val out: Out = new Out(trat.outs$) 41 | 42 | private final def superCache: ReinforceCache = tracker.getCache 43 | 44 | /** 在 reinforce 阶段，从缓存中取回。 */ 45 | private[reflow] final def obtainCache: Option[ReinforceCache] = { 46 | assert(isReinforcing) 47 | superCache.subs.get(trat.name$) 48 | } 49 | 50 | /** `Task`的当前缓存。 */ 51 | private[reflow] final def myCache(create: Boolean = false): Option[Out] = 52 | if (create) Some(superCache.caches.getOrElseUpdate(trat.name$, new Out(Helper.KvTpes.empty()))) 53 | else superCache.caches.get(trat.name$) 54 | 55 | private[reflow] final def cache[V](key: String, value: V): Unit = myCache(create = true).get.cache(key, value) 56 | 57 | final def depth: Int = tracker.subDepth 58 | final def reflow: Reflow = tracker.reflow 59 | final def reflowTop: Reflow = tracker.reflowTop 60 | final def serialNum: Long = tracker.serialNum 61 | final def globalTrack: Boolean = tracker.globalTrack 62 | 63 | final lazy val weightPar: Int = reflow.basis.weightedPeriod(trat) 64 | 65 | /** 请求强化运行。 66 | * @return （在本任务或者本次调用）之前是否已经请求过, 同`isReinforceRequired()`。 67 | */ 68 | final def requireReinforce(): Boolean = tracker.requireReinforce(trat) 69 | final def isReinforceRequired: Boolean = tracker.isReinforceRequired 70 | final def isReinforcing: Boolean = tracker.isReinforcing 71 | final def isPulseMode: Boolean = tracker.isPulseMode 72 | final def isSubReflow: Boolean = tracker.isSubReflow 73 | final def parent: Option[ReflowTrait] = tracker.parent 74 | } 75 | 76 | private[reflow] object Env { 77 | 78 | def apply(_trat: Trait, _tracker: Tracker): Env = new Env { 79 | override private[reflow] val trat = _trat 80 | override private[reflow] val tracker = _tracker 81 | } 82 | } 83 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Feedback.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import java.util.concurrent.locks.ReentrantLock 20 | import hobby.chenai.nakam.basis.TAG 21 | import hobby.chenai.nakam.lang.J2S.{NonNull, Obiter} 22 | import hobby.chenai.nakam.lang.TypeBring.AsIs 23 | import hobby.wei.c.reflow.Feedback.Progress 24 | import hobby.wei.c.reflow.Feedback.Progress.Strategy.{Depth, Fluent} 25 | import hobby.wei.c.reflow.Reflow.{logger => log, _} 26 | import hobby.wei.c.reflow.Trait.ReflowTrait 27 | import hobby.wei.c.tool.Locker 28 | import scala.collection._ 29 | 30 | /** 31 | * @author Wei Chou(weichou2010@gmail.com) 32 | * @version 1.0, 02/07/2016 33 | */ 34 | trait Feedback extends Equals { 35 | /** 36 | * 任务已提交到线程池，等待被执行。 37 | */ 38 | def onPending(): Unit 39 | 40 | /** 41 | * 第一个任务开始执行。 42 | */ 43 | def onStart(): Unit 44 | 45 | /** 46 | * 进度反馈。 47 | * 48 | * @param progress 进度对象。 49 | * @param out 进度的时刻已经获得的输出。 50 | * @param fromDepth 触发当前进度反馈的`子任务流（SubReflow）`的嵌套深度（顶层为`0`，并按照`SubReflow`的嵌套层级依次递增）。 51 | */ 52 | def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit 53 | 54 | /** 55 | * 任务流执行完成。 56 | * 57 | * @param out 任务的输出结果。 58 | */ 59 | def onComplete(out: Out): Unit 60 | 61 | /** 62 | * 强化运行完毕之后的最终结果。 63 | * 64 | * @see Task#requireReinforce() 65 | */ 66 | def onUpdate(out: Out): Unit 67 | 68 | /** 69 | * 任务流中断。 70 | * 71 | * @param trigger 触发中断的`Trait`，为`None`表示客户代码通过`scheduler`主动触发。 72 | * @param depth 若`< 0`表示客户代码通过`scheduler`主动触发。 73 | */ 74 | def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit 75 | 76 | /** 77 | * 任务失败。 78 | * 79 | * @param trat 触发失败的`Trait`。 80 | * @param e 分为两类: 81 | * 第一类是客户代码自定义的 Exception, 即显式传给`Task#failed(Exception)`方法的参数, 可能为`null`; 82 | * 第二类是由客户代码质量问题导致的 RuntimeException, 如`NullPointerException`等, 83 | * 这些异常被包装在`CodeException`里, 可以通过`CodeException#getCause()`方法取出具体异对象。 84 | */ 85 | def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit 86 | 87 | override def equals(any: Any) = super.equals(any) 88 | override def canEqual(that: Any) = false 89 | } 90 | 91 | object Feedback { 92 | import Progress.Weight 93 | 94 | /** 95 | * 表示任务的进度。由于任务可以嵌套，所以进度也需要嵌套，以便实现更精确的管理。 96 | * 97 | * @param sum 当前进度的总步数。 98 | * @param step 当前进度走到了第几步。 99 | * @param trat 当前`step`对应的 top level `Trait`。分为3种情况： 100 | * 1. 若`subs`为`None`，则`trat`一定也为`None`，表示某`Task.progress(step, sum)`出来的进度； 101 | * 2. 若当前是一个具体的（顺序依赖的）任务触发出的进度，则`trat`表示该任务，同时`subs`中会有唯一一个进度与之对应，该进度其性质同 1； 102 | * 3. 若`trat`是并行的（`_.isPar == true`），则`subs`代表了所有当前正在并行执行的任务。 103 | * 注意：`subs`中的进度同样分为以上 3 种情况。 104 | * @param trigger 表示触发进度反馈的那个进度，以便知道哪个任务是开始还是完成。注意：深层的[[trigger]]总是会被提到顶层（总是有值），不需要再关注递归的[[trigger]]属性（可能为`null`）（13/10/2020 增加）。 105 | * @param subs 子任务。可以是并行的，所以用了`Seq`。 106 | * @param weight 根据`Period`得到的加权参数（29/04/2021 增加）。 107 | */ 108 | final case class Progress(sum: Int, step: Int, weight: Weight, trat: Option[Trait] = None, trigger: Progress = null, private var subs: Option[Seq[Progress]] = None) { 109 | assert(step < sum || (step == sum && subs.isEmpty)) 110 | assert(subs.fold(true)(_.forall(_.nonNull))) 111 | 112 | lazy val main: Float = (weight.serial + sub * weight.rate) / sum 113 | lazy val sub: Float = subs.fold[Float](0) { seq => seq.map(p => p() * p.weight.par).sum / seq.map(_.weight.par).sum }.ensuring { _ => subs = None; true } 114 | 115 | @inline def apply(): Float = main 116 | 117 | override def toString = 118 | s"Progress(sum:$sum, step:$step, weight:$weight, p:$main, sub:$sub, name:${trat.map(_.name$).orNull}, trigger:${if (trigger.isNull) null else { trigger.trat.map(_.name$).orNull + "(" + trigger.main + ")" }})" 119 | } 120 | 121 | object Progress { 122 | final case class Weight(serial: Float, rate: Float, par: Int = 10) 123 | 124 | /** 进度反馈的优化策略。 */ 125 | trait Strategy extends Ordering[Strategy] { 126 | outer => 127 | val priority: Int 128 | def genDelegator(feedback: Feedback): Feedback = feedback 129 | def ->(strategy: Strategy): Strategy = new Multiply(this, strategy) 130 | final def isFluentMode: Boolean = this <= Fluent 131 | def base = this 132 | 133 | /** 生成用于传递到`SubReflow`的`Strategy`。 */ 134 | def toSub = this match { 135 | case Depth(level) => Depth(level - 1) // 每加深一层即递减 136 | case p => p 137 | } 138 | 139 | final def revise(strategy: Strategy): Strategy = 140 | if (strategy.base equiv this.base) { // 如果相等，其中一个必然是`Depth`。 141 | strategy match { 142 | case _: Depth => strategy 143 | case _ => this 144 | } 145 | } else if (strategy.base > this.base) strategy 146 | else this 147 | 148 | // 优先级越高，数值越小。 149 | override def compare(x: Strategy, y: Strategy) = if (x.priority > y.priority) -1 else if (x.priority < y.priority) 1 else 0 150 | } 151 | 152 | class Multiply(val before: Strategy, val after: Strategy) extends Strategy { 153 | override val priority = (before min after).priority 154 | 155 | override def base = before.base 156 | override def toSub = before.toSub -> after.toSub 157 | override def genDelegator(feedback: Feedback) = before.genDelegator(after.genDelegator(feedback)) 158 | } 159 | 160 | object Strategy { 161 | 162 | /** 全量。不错过任何进度细节。 */ 163 | object FullDose extends Strategy { 164 | override val priority = 0 165 | } 166 | 167 | /** 流畅的。即：丢弃拥挤的消息。（注意：仅适用于`Poster`之类有队列的）。 */ 168 | object Fluent extends Strategy { 169 | override val priority = 1 170 | 171 | // 虽然`Tracker`内部已经实现，但仍需增强。 172 | override def genDelegator(feedback: Feedback) = new Delegator(feedback) with TAG.ClassName { 173 | @volatile private var main = -1f 174 | 175 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = { 176 | if (debugMode) log.i("~~~~~~~~~~~~~~~~~~~~~~~~[Fluent]fromDepth:%s, progress:%s.", fromDepth, progress) 177 | if ( 178 | (progress.main > main).obiter { 179 | main = progress.main 180 | } 181 | ) { 182 | super.onProgress(progress, out, fromDepth) 183 | } 184 | } 185 | } 186 | } 187 | 188 | /** 189 | * 基于子进度的深度。 190 | * 191 | * @param level 子进度的深度水平。`0`表示放弃顶层进度；`1`表示放弃子层进度；`2`表示放弃次子层进度。以此类推。 192 | */ 193 | case class Depth(level: Int) extends Strategy { 194 | override val priority = Fluent.priority - (level max 0) 195 | 196 | final def isMind(level: Int) = this.level > level 197 | 198 | override def genDelegator(feedback: Feedback) = 199 | if (isFluentMode) Fluent.genDelegator(feedback) 200 | else new Delegator(feedback) with TAG.ClassName { 201 | @volatile private var step: Int = -1 202 | 203 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = { 204 | if (debugMode) log.i("[Depth(%s)]progress:%s, out:%s, fromDepth:%s.", level, progress, out, fromDepth) 205 | if (isMind(0)) { // 关注当前层 206 | if ( 207 | isMind(1) /*关注子层*/ 208 | || (progress.step > step).obiter { 209 | step = progress.step 210 | } 211 | ) { 212 | super.onProgress(progress, out, fromDepth) 213 | } 214 | } 215 | } 216 | } 217 | } 218 | 219 | /** 220 | * 基于反馈时间间隔（构建于`Fluent`之上）。 221 | * 222 | * @param minGap 最小时间间隔，单位：毫秒。 223 | */ 224 | case class Interval(minGap: Int) extends Strategy { 225 | override val priority = 2 226 | 227 | override def genDelegator(feedback: Feedback) = 228 | if (minGap <= 0) super.genDelegator(feedback) 229 | else new Delegator(feedback) { 230 | @volatile private var time = 0L 231 | 232 | override def onStart(): Unit = { 233 | super.onStart() 234 | time = System.currentTimeMillis 235 | } 236 | 237 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = { 238 | if (minGap > 0) { 239 | val curr = System.currentTimeMillis 240 | if (curr - time >= minGap) { 241 | time = curr 242 | super.onProgress(progress, out, fromDepth) 243 | } 244 | } else super.onProgress(progress, out, fromDepth) 245 | } 246 | } 247 | } 248 | } 249 | } 250 | 251 | implicit class Join(fb: Feedback = null) { 252 | 253 | def join(that: Feedback): Feedback = { 254 | // 把 that 放在最前面 255 | if (fb.nonNull && fb.isInstanceOf[Feedback.Observable]) { 256 | val obs = fb.as[Feedback.Observable] 257 | val old = obs.obs.toList 258 | obs.removeAll() 259 | obs.addObservers(that :: old: _*) 260 | obs 261 | } else { 262 | val obs = new Feedback.Observable 263 | obs.addObservers(that :: (if (fb.nonNull) fb :: Nil else Nil): _*) 264 | obs 265 | } 266 | } 267 | 268 | @inline def reverse(): Feedback = { 269 | if (fb.nonNull && fb.isInstanceOf[Feedback.Observable]) fb.as[Feedback.Observable].reverse() 270 | fb 271 | } 272 | } 273 | 274 | implicit class WithPoster(feedback: Feedback) { 275 | 276 | def wizh(poster: Poster): Feedback = 277 | if (poster.isNull) feedback 278 | else if (feedback.isNull) feedback 279 | else new Delegator(feedback) { 280 | require(poster.nonNull) 281 | 282 | override def onPending(): Unit = poster.post(super.onPending()) 283 | override def onStart(): Unit = poster.post(super.onStart()) 284 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = poster.post(super.onProgress(progress, out, fromDepth)) 285 | override def onComplete(out: Out): Unit = poster.post(super.onComplete(out)) 286 | override def onUpdate(out: Out): Unit = poster.post(super.onUpdate(out)) 287 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = poster.post(super.onAbort(trigger, parent, depth)) 288 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = poster.post(super.onFailed(trat, parent, depth, e)) 289 | } 290 | } 291 | 292 | private[reflow] class Delegator(feedback: Feedback) extends Feedback { 293 | require(feedback.nonNull) 294 | 295 | override def onPending(): Unit = feedback.onPending() 296 | override def onStart(): Unit = feedback.onStart() 297 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = feedback.onProgress(progress, out, fromDepth) 298 | override def onComplete(out: Out): Unit = feedback.onComplete(out) 299 | override def onUpdate(out: Out): Unit = feedback.onUpdate(out) 300 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = feedback.onAbort(trigger, parent, depth) 301 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = feedback.onFailed(trat, parent, depth, e) 302 | } 303 | 304 | class Adapter extends Feedback { 305 | override def onPending(): Unit = {} 306 | override def onStart(): Unit = {} 307 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = {} 308 | override def onComplete(out: Out): Unit = {} 309 | override def onUpdate(out: Out): Unit = {} 310 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = {} 311 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = Log.onFailed(trat, parent, depth, e) 312 | } 313 | 314 | /** 315 | * 仅关注需要的值的`Feedback`，以方便客户代码的编写。 316 | * 用法示例：{{{ 317 | * val buttStr = new Feedback.Butt(new Kce[String]("str") {}, watchProgressDepth = 1) { 318 | * override def onValueGotOnProgress(value: Option[String], progress: Progress): Unit = ??? 319 | * override def onValueGot(value: Option[String]): Unit = ??? 320 | * override def onValueGotOnUpdate(value: Option[String]): Unit = ??? 321 | * override def onFailed(trat: Trait, e: Exception): Unit = ??? 322 | * } 323 | * val buttInt = new Feedback.Butt[Integer](new Kce[Integer]("int") {}) { 324 | * override def onValueGot(value: Option[Integer]): Unit = ??? 325 | * } 326 | * // 可将多个值用`join`连接，将返回的`Feedback`传给`Reflow.start()`。 327 | * val feedback = buttStr.join(buttInt) 328 | * }}} 329 | * 注意：如果需要监听`Feedback`的更多事件，只需要在 join 的第一个`butt`下重写需要的回调即可（第一个`butt`在排序上是放在最后的）。 330 | * 331 | * @param kce 所关注值的`Kce[T]`信息。 332 | * @param watchProgressDepth 如果同时关注进度中的反馈值的话，会涉及到 Reflow 嵌套深度的问题。 333 | * 本参数表示关注第几层的进度（即：是第几层的哪个任务会输出`kce`值，Reflow 要求不同层任务的`kce`可以相同）。 334 | */ 335 | abstract class Butt[T >: Null <: AnyRef](kce: KvTpe[T], watchProgressDepth: Int = 0) extends Adapter { 336 | 337 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = { 338 | super.onProgress(progress, out, fromDepth) 339 | if (fromDepth == watchProgressDepth && out.keysDef().contains(kce)) 340 | onValueGotOnProgress(out.get(kce), progress) 341 | } 342 | 343 | override def onComplete(out: Out): Unit = { 344 | super.onComplete(out) 345 | onValueGotOnComplete(out.get(kce)) 346 | } 347 | 348 | override def onUpdate(out: Out): Unit = { 349 | super.onUpdate(out) 350 | onValueGotOnUpdate(out.get(kce)) 351 | } 352 | 353 | def onValueGotOnProgress(value: Option[T], progress: Progress): Unit = {} 354 | def onValueGotOnComplete(value: Option[T]): Unit 355 | def onValueGotOnUpdate(value: Option[T]): Unit = {} 356 | } 357 | 358 | abstract class Lite[-T <: AnyRef](watchProgressDepth: Int = 0) extends Butt(lite.Task.defKeyVType, watchProgressDepth) { 359 | 360 | @deprecated 361 | override final def onValueGotOnProgress(value: Option[AnyRef], progress: Progress): Unit = 362 | liteValueGotOnProgress(value.as[Option[T]], progress) 363 | 364 | @deprecated 365 | override final def onValueGotOnComplete(value: Option[AnyRef]): Unit = liteOnComplete(value.as[Option[T]]) 366 | 367 | @deprecated 368 | override final def onValueGotOnUpdate(value: Option[AnyRef]): Unit = liteOnUpdate(value.as[Option[T]]) 369 | 370 | def liteValueGotOnProgress(value: Option[T], progress: Progress): Unit = {} 371 | def liteOnComplete(value: Option[T]): Unit 372 | def liteOnUpdate(value: Option[T]): Unit = {} 373 | } 374 | 375 | object Lite { 376 | 377 | implicit final object Log extends Feedback.Lite[AnyRef] with TAG.ClassName { 378 | import Reflow.{logger => log} 379 | 380 | override def onPending(): Unit = log.i("[onPending]") 381 | override def onStart(): Unit = log.i("[onStart]") 382 | override def onProgress(progress: Progress, out: Out, depth: Int): Unit = log.i("[onProgress]depth:%s, progress:%s, value:%s.", depth, progress, out /*.get(lite.Task.defKeyVType)*/ ) 383 | override def onComplete(out: Out): Unit = super.onComplete(out) 384 | override def onUpdate(out: Out): Unit = super.onUpdate(out) 385 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = log.w("[onAbort]depth:%s, trigger:%s, parent:%s.", depth, trigger, parent) 386 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = log.e(e, "[onFailed]depth:%s, trat:%s, parent:%s.", depth, trat, parent) 387 | 388 | override def liteOnComplete(value: Option[AnyRef]): Unit = log.w("[liteOnComplete]value:%s.", value) 389 | override def liteOnUpdate(value: Option[AnyRef]): Unit = log.w("[liteOnUpdate]value:%s.", value) 390 | } 391 | 392 | } 393 | 394 | class Observable extends Adapter with TAG.ClassName { 395 | import Assist.eatExceptions 396 | implicit private lazy val lock: ReentrantLock = Locker.getLockr(this) 397 | 398 | @volatile 399 | private[Feedback] var obs: Seq[Feedback] = Nil //scala.collection.concurrent.TrieMap[Feedback, Unit] //CopyOnWriteArraySet[Feedback] 400 | 401 | private[Feedback] def removeAll(): Unit = Locker.syncr(obs = Nil) 402 | private[Feedback] def reverse(): Unit = Locker.syncr(obs = obs.reverse) 403 | 404 | def addObservers(fbs: Feedback*): Unit = Locker.syncr { 405 | obs = (obs.to[mutable.LinkedHashSet] ++= fbs.ensuring(_.forall(_.nonNull))).toList 406 | } 407 | 408 | def removeObservers(fbs: Feedback*): Unit = Locker.syncr { 409 | obs = (obs.to[mutable.LinkedHashSet] --= fbs.ensuring(_.forall(_.nonNull))).toList 410 | } 411 | 412 | override def onPending(): Unit = obs.foreach { fb => eatExceptions(fb.onPending()) } 413 | override def onStart(): Unit = obs.foreach { fb => eatExceptions(fb.onStart()) } 414 | override def onProgress(progress: Progress, out: Out, fromDepth: Int): Unit = obs.foreach { fb => eatExceptions(fb.onProgress(progress, out, fromDepth)) } 415 | override def onComplete(out: Out): Unit = obs.foreach { fb => eatExceptions(fb.onComplete(out)) } 416 | override def onUpdate(out: Out): Unit = obs.foreach { fb => eatExceptions(fb.onUpdate(out)) } 417 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = obs.foreach { fb => eatExceptions(fb.onAbort(trigger, parent, depth)) } 418 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = obs.foreach { fb => eatExceptions(fb.onFailed(trat, parent, depth, e)) } 419 | } 420 | 421 | implicit final object Log extends Feedback with TAG.ClassName { 422 | import Reflow.{logger => log} 423 | 424 | override def onPending(): Unit = log.i("[onPending]") 425 | override def onStart(): Unit = log.i("[onStart]") 426 | override def onProgress(progress: Progress, out: Out, depth: Int): Unit = log.i("[onProgress]depth:%s, progress:%s, out:%s.", depth, progress, out) 427 | override def onComplete(out: Out): Unit = log.w("[onComplete]out:%s.", out) 428 | override def onUpdate(out: Out): Unit = log.w("[onUpdate]out:%s.", out) 429 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = log.w("[onAbort]depth:%s, trigger:%s, parent:%s.", depth, trigger, parent) 430 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = log.e(e, "[onFailed]depth:%s, trat:%s, parent:%s.", depth, trat, parent) 431 | } 432 | } 433 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/GlobalTrack.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.wei.c.reflow.Feedback.Progress 21 | import hobby.wei.c.reflow.Feedback.Progress.Weight 22 | import hobby.wei.c.reflow.Trait.ReflowTrait 23 | 24 | /** 25 | * 全局跟踪器。跟踪当前正在运行的任务流。可用于构建全局`任务管理器`。 26 | * 27 | * @author Wei Chou(weichou2010@gmail.com) 28 | * @version 1.0, 03/04/2018 29 | */ 30 | final case class GlobalTrack(reflow: Reflow, scheduler: Scheduler, parent: Option[ReflowTrait]) extends Equals with TAG.ClassName { 31 | @volatile private var _progress: Progress = { 32 | val pgr = Progress(reflow.basis.traits.size, 0, Weight(0, 1), reflow.basis.traits.headOption) 33 | pgr.copy(trigger = pgr) 34 | } 35 | 36 | private[reflow] def progress(progress: Progress): GlobalTrack = { 37 | _progress = progress 38 | this 39 | } 40 | 41 | def isSubReflow: Boolean = parent.isDefined 42 | def progress = _progress 43 | 44 | /** 取得[正在]和[将要]执行的任务列表。 */ 45 | def remaining = reflow.basis.traits.drop(_progress.step).ensuring(r => _progress.trat.fold(true)(_ == r.head)) 46 | 47 | override def equals(any: Any) = any match { 48 | case that: GlobalTrack if that.canEqual(this) => 49 | (this.reflow eq that.reflow) && (this.scheduler eq that.scheduler) 50 | case _ => false 51 | } 52 | 53 | override def canEqual(that: Any) = that.isInstanceOf[GlobalTrack] 54 | 55 | override def toString = s"[$className]reflow:$reflow, scheduler:$scheduler, isSubReflow:$isSubReflow, parent:${ 56 | parent.fold[String](null)(_.name$) 57 | }, progress:$progress." 58 | } 59 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Helper.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S.NonNull 20 | 21 | import scala.collection._ 22 | 23 | /** 24 | * @author Wei Chou(weichou2010@gmail.com) 25 | * @version 1.0, 14/08/2016 26 | */ 27 | object Helper { 28 | object KvTpes { 29 | def empty(): immutable.Set[KvTpe[_ <: AnyRef]] = immutable.Set.empty 30 | 31 | def +(ks: KvTpe[_ <: AnyRef]*): Builder = new Builder + (ks: _*) 32 | 33 | class Builder private[reflow]() { 34 | private val keys = new mutable.HashSet[KvTpe[_ <: AnyRef]] 35 | 36 | def +(ks: KvTpe[_ <: AnyRef]*): this.type = { 37 | keys ++= ks.ensuring(_.forall(_.nonNull)) 38 | this 39 | } 40 | 41 | def ok(): immutable.Set[KvTpe[_ <: AnyRef]] = keys.toSet 42 | } 43 | } 44 | 45 | object Transformers { 46 | /** 47 | * 将任务的某个输出在转换之后仍然保留。通过增加一个[输出即输入]转换。 48 | */ 49 | def retain[O <: AnyRef](kce: KvTpe[O]): Transformer[O, O] = new Transformer[O, O](kce, kce) { 50 | override def transform(in: Option[O]) = in 51 | } 52 | 53 | def +(trans: Transformer[_ <: AnyRef, _ <: AnyRef]*): Builder = new Builder + (trans: _*) 54 | 55 | class Builder private[reflow]() { 56 | private val trans = new mutable.HashSet[Transformer[_ <: AnyRef, _ <: AnyRef]] 57 | 58 | def +(ts: Transformer[_ <: AnyRef, _ <: AnyRef]*): this.type = { 59 | trans ++= ts.ensuring(_.forall(_.nonNull)) 60 | this 61 | } 62 | 63 | def retain[O <: AnyRef](kce: KvTpe[O]): this.type = this + Transformers.retain[O](kce) 64 | 65 | def ok(): immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]] = trans.toSet 66 | } 67 | } 68 | } 69 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/In.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S 20 | import hobby.wei.c.reflow.Assist._ 21 | import hobby.wei.c.reflow.Dependency.MapTo 22 | import hobby.wei.c.tool.Locker 23 | 24 | import scala.collection._ 25 | import scala.ref.WeakReference 26 | 27 | /** 28 | * @author Wei Chou(weichou2010@gmail.com) 29 | * @version 1.0, 14/08/2016; 30 | * 2.0, 23/04/2019, 修复 Scala 枚举在`In`中的 Bug。 31 | */ 32 | abstract class In protected(_keys: Set[KvTpe[_ <: AnyRef]], _trans: Transformer[_ <: AnyRef, _ <: AnyRef]*) { 33 | private[reflow] val keys: immutable.Set[KvTpe[_ <: AnyRef]] = requireKkDiff(requireElemNonNull(_keys.toSet)) 34 | private[reflow] val trans: immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]] = requireTransInTpeSame$OutKDiff(requireElemNonNull(_trans.toSet)) 35 | 36 | private[reflow] def fillValues(out: Out): Unit = (out.keysDef & keys).foreach { key => out.put(key.key, loadValue(key.key).orNull) } 37 | 38 | protected def loadValue(key: String): Option[Any] 39 | } 40 | 41 | object In { 42 | def map(map: Map[String, Any], mapKes: Map[KvTpe[_ <: AnyRef], AnyRef], trans: Transformer[_ <: AnyRef, _ <: AnyRef]*): In = new M( 43 | generate(map) ++ mapKes.keySet, 44 | (map.mutable /: mapKes) { (m, kv) => m += (kv._1.key, kv._2) }, trans: _*) 45 | 46 | def from(input: Out): In = new M(input._keys.values.toSet, input._map) 47 | 48 | def +[T <: AnyRef](kv: (KvTpe[T], T)): Builder = new Builder + kv 49 | 50 | def +(key: String, value: Any): Builder = new Builder + (key, value) 51 | 52 | class Builder private[reflow]() { 53 | private lazy val mapKes = new mutable.AnyRefMap[KvTpe[_ <: AnyRef], AnyRef] 54 | private lazy val map = new mutable.AnyRefMap[String, Any] 55 | private lazy val tb: Helper.Transformers.Builder = new Helper.Transformers.Builder 56 | 57 | def +[T <: AnyRef](kv: (KvTpe[T], T)): this.type = { 58 | require(!map.contains(kv._1.key)) 59 | mapKes += kv 60 | this 61 | } 62 | 63 | def +(key: String, value: Any): this.type = { 64 | require(!mapKes.exists(_._1.key == key)) 65 | map += (key, value) 66 | this 67 | } 68 | 69 | def +(ts: Transformer[_ <: AnyRef, _ <: AnyRef]*): this.type = { 70 | tb + (ts: _*) 71 | this 72 | } 73 | 74 | def ok(): In = In.map(map, mapKes, tb.ok().toSeq: _*) 75 | } 76 | 77 | private def generate(map: Map[String, Any]): Set[KvTpe[_ <: AnyRef]] = if (map.isEmpty) Set.empty else 78 | (new mutable.HashSet[KvTpe[_ <: AnyRef]] /: map) { (set, kv) => set += new KvTpe(kv._1, kv._2.getClass, 0, true) {} } 79 | 80 | private class M private[reflow](keys: Set[KvTpe[_ <: AnyRef]], map: Map[String, Any], 81 | trans: Transformer[_ <: AnyRef, _ <: AnyRef]*) extends In(keys, trans: _*) { 82 | override protected def loadValue(key: String) = map.get(key) 83 | } 84 | 85 | def empty(): In = Locker.lazyGetr(J2S.getRef(emptyRef).orNull) { 86 | val in = new In(Helper.KvTpes.empty()) { 87 | override private[reflow] def fillValues(out: Out): Unit = {} 88 | 89 | override protected def loadValue(key: String) = None 90 | } 91 | emptyRef = new WeakReference(in) 92 | in 93 | }(Locker.getLockr(this)) 94 | 95 | private var emptyRef: WeakReference[In] = _ 96 | } 97 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/KvTpe.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import java.lang.reflect.Type 20 | import hobby.chenai.nakam.lang.J2S.NonNull 21 | import hobby.chenai.nakam.lang.TypeBring.AsIs 22 | import hobby.wei.c.tool.Reflect 23 | 24 | import scala.collection._ 25 | 26 | /** 27 | * 定义key及value类型。value类型由泛型指定。 28 | * 注意: 本类的子类必须在运行时创建, 即匿名子类, 否则泛型信息可能在Proguard时被删除, 从而导致解析失败。 29 | * 30 | * @author Wei Chou(weichou2010@gmail.com) 31 | * @version 1.0, 21/07/2016 32 | */ 33 | abstract class KvTpe[T <: AnyRef] private[reflow](_key: String, _clazz: Class[_], _index: Int = 0, _raw: Boolean = false) extends Equals { 34 | protected def this(_key: String) = this(_key, null) 35 | 36 | final val key: String = _key.ensuring(_.nonEmpty) 37 | /** 38 | * 泛型参数的类型, 类似于这种结构: java.util.List>。 39 | */ 40 | final val tpe: Type = (if (_raw) _clazz else Reflect.getSuperclassTypeParameter(if (_clazz.isNull) this.getClass else _clazz, true)(_index)).ensuring(_.nonNull) 41 | /** 42 | * 第一级泛型参数的Class表示。 43 | */ 44 | private lazy val rawType: Class[_ >: T] = Reflect.getRawType(tpe).as[Class[_ >: T]] 45 | /** 46 | * 第一级泛型参数的子泛型参数, 可能不存在。作用或结构与tpe类似。 47 | */ 48 | private lazy val subTypes: Array[Type] = Reflect.getSubTypes(tpe) 49 | 50 | /** 51 | * 走这个方法作类型转换, 确保value类型与定义的一致性。 52 | * @return 返回Task在执行时当前key对应值的目标类型。 53 | */ 54 | def asType(value: Any): T = value.as[T] 55 | 56 | def putValue(map: mutable.Map[String, Any], value: Any): Boolean = putValue(map, value, ignoreTpeDiff = false) 57 | 58 | /** 59 | * 将输出值按指定类型(作类型检查)插入Map。 60 | * 61 | * @param map 输出到的Map。 62 | * @param value 要输出的值。 63 | * @param ignoreTpeDiff 如果value参数类型不匹配，是否忽略。 64 | * @return true成功，else失败。 65 | */ 66 | def putValue(map: mutable.Map[String, Any], value: Any, ignoreTpeDiff: Boolean): Boolean = { 67 | val v = requireSameType(value, ignoreTpeDiff) 68 | if (v.nonNull) { 69 | map.put(key, v) 70 | true 71 | } else false 72 | } 73 | 74 | /** 75 | * 走这个方法取值, 确保value类型与定义的一致性。 76 | * 77 | * @param map Task的输入参数。 78 | * @return 返回Task在执行时当前key对应值的目标类型。 79 | */ 80 | def takeValue(map: Map[String, Any]): Option[T] = { 81 | // 强制类型转换比较宽松, 只会检查对象类型, 而不会检查泛型。 82 | // 但是由于value值对象无法获得泛型类型, 因此这里不再作泛型检查。也避免了性能问题。 83 | map.get(key).as[Option[T]] 84 | } 85 | 86 | private def requireSameType(value: Any, ignoreDiffType: Boolean): Any = { 87 | if (value.nonNull) { 88 | val clazz = value.getClass 89 | if (!rawType.isAssignableFrom(clazz)) { 90 | if (ignoreDiffType) null 91 | else Assist.Throws.typeNotMatch(this, clazz) 92 | } else value 93 | // 数值对象通常已经失去了泛型参数, 因此不作检查 94 | } else value 95 | } 96 | 97 | /** 98 | * 参数的value类型是否与本value类型相同或者子类型。 99 | * 同{Class#isAssignableFrom(Class)} 100 | * 101 | * @param key 102 | * @return 103 | */ 104 | def isAssignableFrom(key: KvTpe[_ <: AnyRef]): Boolean = { 105 | if (!rawType.isAssignableFrom(key.rawType)) false 106 | else if (subTypes.length != key.subTypes.length) false 107 | else subTypes.indices.forall(i => subTypes(i) == key.subTypes(i)) 108 | } 109 | 110 | override def equals(any: scala.Any) = any match { 111 | case that: KvTpe[_] if that.canEqual(this) => that.key == this.key && that.tpe == this.tpe 112 | case _ => false 113 | } 114 | 115 | override def canEqual(that: Any) = that.isInstanceOf[KvTpe[T]] 116 | 117 | override def hashCode = key.hashCode * 41 + tpe.hashCode 118 | 119 | override def toString = s"${classOf[KvTpe[_]].getSimpleName}[$key -> $tpe]" 120 | } 121 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Out.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S.NonNull 20 | import hobby.chenai.nakam.lang.TypeBring.AsIs 21 | import hobby.wei.c.reflow.Assist._ 22 | import hobby.wei.c.reflow.Reflow._ 23 | 24 | import scala.collection._ 25 | 26 | /** 27 | * @author Wei Chou(weichou2010@gmail.com) 28 | * @version 1.0, 26/06/2016 29 | */ 30 | class Out private[reflow] (map: Map[String, KvTpe[_ <: AnyRef]]) { 31 | private[reflow] def this() = this(Map.empty[String, KvTpe[_ <: AnyRef]]) 32 | 33 | def this(keys: Set[KvTpe[_ <: AnyRef]]) = this((new mutable.AnyRefMap[String, KvTpe[_ <: AnyRef]] /: keys) { (m, k) => m += (k.key, k) }) 34 | 35 | // 仅读取 36 | private[reflow] val _keys = map.toMap[String, KvTpe[_ <: AnyRef]] 37 | // 由于并行的任务，不可能有相同的key, 没有必要让本类的整个方法调用都进行sync, 因此用并行库是最佳方案。 38 | private[reflow] val _map = new concurrent.TrieMap[String, Any] 39 | private[reflow] val _nullValueKeys = new concurrent.TrieMap[String, KvTpe[_ <: AnyRef]] 40 | 41 | private[reflow] def fillWith(out: Out, fullVerify: Boolean = true): Unit = putWith(out._map, out._nullValueKeys, ignoreTpeDiff = true, fullVerify) 42 | 43 | private[reflow] def verify(): Unit = putWith(immutable.Map.empty, immutable.Map.empty, ignoreTpeDiff = false, fullVerify = true) 44 | 45 | /** 46 | * 若调用本方法, 则必须一次填满, 否则报异常。 47 | * 48 | * @param map 要填充到`_map`的映射集合。 49 | * @param nulls 因为`value`为`null`的无法插入到`_map`集合。 50 | * @param ignoreDiffType 是否忽略不同值类型(`Key$`)。仅有一个场景用到：使用上一个任务的输出填充当前输出（在当前对象创建的时候。 51 | * 通常情况下，前面任务的输出可能继续向后流动），有时候后面可能会出现相同的`k.key`但类型不同，这是允许的（即：相同的key可以被覆盖）。 52 | * 但在使用上一个任务的输出填充当前对象的时候，如果`k.key`相同但类型不匹配，会抛出异常。为了简化起见，设置了本参数。 53 | * @param fullVerify 检查`_keys`是否全部输出。 54 | */ 55 | private[reflow] def putWith(map: Map[String, Any], nulls: Map[String, KvTpe[_ <: AnyRef]], ignoreTpeDiff: Boolean = false, fullVerify: Boolean = false): Unit = { 56 | _keys.values.foreach { k => 57 | if (map.contains(k.key)) { 58 | if (k.putValue(_map, map(k.key), ignoreTpeDiff)) { 59 | _nullValueKeys.remove(k.key) 60 | } 61 | } else if (!_map.contains(k.key)) { 62 | if (nulls.contains(k.key)) { 63 | _nullValueKeys.put(k.key, k) 64 | } else if (fullVerify) { 65 | if (!_nullValueKeys.contains(k.key)) { 66 | Throws.lackIOKey(k, in$out = false) 67 | } 68 | } 69 | } 70 | } 71 | } 72 | 73 | private[reflow] def put[T](key: String, value: T): Boolean = { 74 | if (_keys.contains(key)) { 75 | val k = _keys(key) 76 | if (value.isNull && !_map.contains(key)) { 77 | _nullValueKeys.put(k.key, k) 78 | } else { 79 | k.putValue(_map, value) 80 | _nullValueKeys.remove(k.key) 81 | } 82 | true 83 | } else false 84 | } 85 | 86 | private[reflow] def cache(out: Out): Unit = { 87 | out._map.toMap.foreach { kv: (String, Any) => 88 | cache(kv._1, kv._2) 89 | } 90 | } 91 | 92 | /** 有reinforce需求的任务, 可以将中间结果缓存在这里。 93 | * 注意: 如果在输入中({#keys Out(Set)}构造器参数)含有本key, 则无法将其缓存。 94 | * 95 | * @return true 成功; false 失败, 说明key重复, 应该换用其它的key。 96 | */ 97 | private[reflow] def cache[T](key: String, value: T): Unit = { 98 | if (_keys.contains(key)) Throws.sameCacheKey(_keys(key)) 99 | else if (value.nonNull) _map.put(key, value) 100 | } 101 | 102 | private[reflow] def remove(key: String): Unit = { 103 | _map -= key 104 | _nullValueKeys -= key 105 | } 106 | 107 | def apply[T >: Null](key: String): T = get[T](key).orNull 108 | 109 | def apply[T >: Null <: AnyRef](kce: KvTpe[T]): T = get[T](kce).orNull 110 | 111 | /** 取得key对应的value。 */ 112 | def get[T](key: String): Option[T] = _map.get(key).as[Option[T]] // 由于`cache`的也是放在一起，其`key`在`_keys`范围之外。所以只能用这种方式读取。 113 | //_keys.get(key).fold[Option[T]](None) { k => get(k.as[Kce[T]]) } 114 | 115 | /** 取得key对应的value。 */ 116 | def get[T <: AnyRef](key: KvTpe[T]): Option[T] = key.takeValue(_map) 117 | 118 | /** 取得预定义的keys及类型。即: 希望输出的keys。 */ 119 | def keysDef(): immutable.Set[KvTpe[_ <: AnyRef]] = _keys.values.toSet 120 | 121 | /** 取得实际输出的keys。 */ 122 | def keys(): immutable.Set[KvTpe[_ <: AnyRef]] = _keys.values.filter { k => _map.contains(k.key) || _nullValueKeys.contains(k.key) }.toSet 123 | 124 | override def toString = "keys:" + _keys.values + ", values:" + _map + (if (_nullValueKeys.isEmpty) "" else ", null:" + _nullValueKeys.values) 125 | } 126 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Poster.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | /** 20 | * 用于将`Feedback`传送到目标线程（如：UI 线程）去运行的时光机。 21 | * 22 | * 类似于[[scala.concurrent.ExecutionContext]]。 23 | * 24 | * @author Wei Chou(weichou2010@gmail.com) 25 | * @version 1.0, 23/07/2016 26 | */ 27 | trait Poster { 28 | def post(f: => Unit): Unit = post(() => f) 29 | 30 | def post(runner: Runnable): Unit // 可这样作默认实现 = runner.run() 31 | } 32 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Reflow.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.lang.J2S.NonNull 21 | import hobby.chenai.nakam.lang.TypeBring.AsIs 22 | import hobby.wei.c.anno.proguard.{KeepMp$, KeepVp$} 23 | import hobby.wei.c.log.Logger 24 | import hobby.wei.c.reflow 25 | import hobby.wei.c.reflow.Assist.{eatExceptions, requirePulseKeyDiff} 26 | import hobby.wei.c.reflow.Dependency._ 27 | import hobby.wei.c.reflow.Feedback.Progress.Strategy 28 | import hobby.wei.c.reflow.Reflow.GlobalTrack.Feedback4GlobalTrack 29 | import hobby.wei.c.reflow.Reflow.Period.TRANSIENT 30 | import hobby.wei.c.reflow.Trait.ReflowTrait 31 | import hobby.wei.c.tool.Locker 32 | import java.util.concurrent._ 33 | import java.util.concurrent.locks.ReentrantLock 34 | import scala.collection._ 35 | import scala.collection.concurrent.TrieMap 36 | 37 | /** 38 | * 任务[串/并联]组合调度框架。 39 | *

40 | * 概述 41 | *

42 | * 本框架为`简化复杂业务逻辑中`多任务之间的数据流转和事件处理的`编码复杂度`而生。通过`要求显式定义`任务的I/O、 43 | * 基于`关键字`和`值类型`分析的智能化`依赖管理`、一致的`运行调度`、`事件反馈`及`错误处理`接口等设计，实现了既定目标：`任务 44 | * 串/并联组合调度`。 数据即`电流`， 任务即`元件`。在简化编码复杂度的同时，确定的框架可以将原本杂乱无章、错综复杂的写法 45 | * 规范化，编码失误也极易被检测，这将大大增强程序的`易读性`、`健壮性`和`可扩展性`。 46 | *

47 | * 此外还有优化的`可配置`线程池、基于`优先级`和`预估时长`的`按需的`任务装载机制、便捷的`同/异（默认）步`切换调度、巧妙的`中断 48 | * 策略`、任务的`无限`嵌套组装、`浏览/强化`运行模式、无依赖输出`丢弃`、事件反馈可`指定到线程`和对异常事件的`确定性分类`等设计， 49 | * 实现了线程的`无`阻塞高效利用、全局`精准`的任务管理、内存的`有效利用（垃圾丢弃）`、以及数据的`快速加载（浏览模式）`和进度的`策略化 50 | * 反馈`，极大地满足了大型项目的需求。 51 | *

52 | * 相关 53 | * 54 | *

55 | * 本框架的主要功能类似Facebook 56 | * Bolts和RxJava，可以视为对它们[任务组合能力]的细粒度扩展， 57 | * 但更加严谨、高规格和`贴近实际项目需求`。 58 | *

59 | * 本框架基于{@link java.util.concurrent.ThreadPoolExecutor 60 | * 线程池}实现而非{@link java.util.concurrent.ForkJoinPool 61 | * Fork/Join框架（JDK 1.7）}，并对前者作了改进以符合[先增加线程数到{@link Config#maxPoolSize 62 | * 最大}，再入队列，空闲释放线程]这个基本逻辑； 63 | * 后者适用于计算密集型任务，但不适用于本框架的设计目标，也不适用于资源受限的设备(如：手机等)。 64 | * 65 | * @author Wei Chou(weichou2010@gmail.com) 66 | * @version 1.0, 12/04/2015 67 | */ 68 | object Reflow { 69 | /** 70 | * 高优先级的任务将被优先执行(注意: 不是线程优先级)。在{@link Period}相同的情况下。 71 | * 72 | * @see #P_NORMAL 73 | */ 74 | val P_HIGH = -10 75 | /** 76 | * 相同优先级的任务将根据提交顺序依次执行(注意: 不是线程优先级)。在{@link Period}相同的情况下。 77 | *

78 | * 注意：并不是优先级高就必然能获得优先调度权，还取决于{@link Period}以及系统资源占用情况。 79 | * 80 | * @see Period#INFINITE 81 | * @see Config#maxPoolSize() 82 | */ 83 | val P_NORMAL = 0 84 | /** 85 | * 低优先级的任务将被延后执行(注意: 不是线程优先级)。在{@link Period}相同的情况下。 86 | * 87 | * @see #P_NORMAL 88 | */ 89 | val P_LOW = 10 90 | 91 | /** 92 | * 任务大概时长。 93 | */ 94 | @ KeepVp$ 95 | @ KeepMp$ 96 | object Period extends Enumeration { 97 | type Tpe = Period$ 98 | 99 | /** 100 | * @param weight 辅助任务{Trait#priority() 优先级}的调度策略参考。 101 | */ 102 | private[reflow] case class Period$(weight: Int) extends Val with Ordering[Period$] { 103 | private implicit lazy val lock: ReentrantLock = Locker.getLockr(this) 104 | private var average = 0L 105 | private var count = 0L 106 | 107 | override def compare(x: Period$, y: Period$) = if (x.weight < y.weight) -1 else if (x.weight == y.weight) 0 else 1 108 | 109 | def average(duration: Long): Long = Locker.syncr { 110 | if (duration > 0) { 111 | val prevAvg = average 112 | val prevCnt = count 113 | average = (prevAvg * prevCnt + duration) / { 114 | count += 1 115 | count 116 | } 117 | } 118 | average 119 | } 120 | } 121 | /** 122 | * 任务执行时间：瞬间。 123 | */ 124 | val TRANSIENT = Period$(1) 125 | /** 126 | * 任务执行时间：很短。 127 | */ 128 | val SHORT = Period$(2) 129 | /** 130 | * 任务执行时间：很长。 131 | */ 132 | val LONG = Period$(5) 133 | /** 134 | * 任务执行时间：无限长。 135 | *

136 | * 注意：只有在当前时长任务的优先级{@link #P_HIGH 最高}而{@link 137 | * #TRANSIENT}任务的优先级{@link #P_LOW 最低}时，才能真正实现优先于{@link 138 | * #TRANSIENT}执行。 139 | * 140 | * @see #weight 141 | */ 142 | val INFINITE = Period$(20) 143 | } 144 | 145 | private var _debugMode = true 146 | private var _config: Config = Config.DEF 147 | private var _logger: Logger = new Logger() 148 | 149 | /** 150 | * 设置自定义的{@link Config}. 注意: 只能设置一次。 151 | */ 152 | def setConfig(conf: Config): Unit = { 153 | if (_config == Config.DEF && conf != Config.DEF) { 154 | _config = conf 155 | Worker.updateConfig(_config) 156 | } 157 | } 158 | 159 | def config = _config 160 | def setThreadResetor(resetor: ThreadResetor): Unit = Worker.setThreadResetor(resetor) 161 | def setLogger(logr: Logger): Unit = _logger = logr 162 | def logger = _logger 163 | def setDebugMode(b: Boolean): Unit = _debugMode = b 164 | def debugMode = _debugMode 165 | 166 | @deprecated(message = "仅用于结束应用：关闭之后不可以再重启。", since = "0.0.1") 167 | def shutdown(): Unit = Worker.sThreadPoolExecutor.shutdown() 168 | 169 | /** 170 | * 创建以参数开始的新任务流。 171 | * 172 | * @param trat 打头的`Trait`。 173 | * @param trans 转换器列表。 174 | * @return 新的任务流。 175 | */ 176 | def create(trat: Trait, trans: Transformer[_ <: AnyRef, _ <: AnyRef]*): Dependency = builder.next(trat) 177 | 178 | /** 复制参数到新的任务流。 */ 179 | def create(dependency: Dependency): Dependency = builder.next(dependency) 180 | 181 | /** 182 | * 为简单代码段或`Runnable`提供运行入口，以便将其纳入框架的调度管理。 183 | * 184 | * @tparam V 执行的结果类型。 185 | * @param _runner 包装要运行代码。如果是`Runnable`，可以写`runnable.run()`。 186 | * @param _period 同`Trait#period()`。 187 | * @param _priority 同`Trait#priority()`。 188 | * @param _desc 同`Trait#description()`。 189 | * @param _name 同`Trait#name()`。 190 | */ 191 | def submit[V >: Null](_runner: => V)(_period: Period.Tpe, _priority: Int = P_NORMAL, _desc: String = null, _name: String = null): Future[V] = { 192 | import implicits._ 193 | @volatile var callableOut: V = null 194 | val future = new FutureTask[V](() => callableOut) 195 | val trat = new Trait.Adapter() { 196 | override protected def name() = if (_name.isNull || _name.isEmpty) super.name() else _name 197 | override protected def priority() = _priority 198 | override protected def period() = _period 199 | override protected def desc() = if (_desc.isNull) name$ else _desc 200 | override def newTask() = new Task { 201 | override protected def doWork(): Unit = { 202 | callableOut = _runner 203 | future.run() 204 | } 205 | } 206 | } 207 | create(trat).submit(none).start(none, new reflow.Feedback.Adapter)(FullDose, null) 208 | future 209 | } 210 | 211 | /** 212 | * 本实现存在两个问题：

213 | * 1. 无法被`GlobalTrack`监控到；

214 | * 2. FutureTask 会吞掉异常（现已修正）。

215 | * 所以简版仅留给内部使用。 216 | */ 217 | private[reflow] def submit$[V >: Null](_runner: => V)(_period: Period.Tpe, _priority: Int = P_NORMAL) { 218 | Worker.scheduleRunner(new Worker.Runner( 219 | new Trait.Adapter() { 220 | override protected def priority() = _priority 221 | override protected def period() = _period 222 | override protected def desc() = name$ 223 | override def newTask() = null 224 | }, 225 | () => _runner 226 | )) 227 | } 228 | 229 | private[reflow] def builder = new Dependency() 230 | 231 | ////////////////////////////////////////////////////////////////////////////////////// 232 | //********************************** Global Track **********************************// 233 | 234 | object GlobalTrack { 235 | private lazy val observersMap = new TrieMap[GlobalTrackObserver, Feedback4Observer] 236 | private[reflow] lazy val globalTrackMap = new TrieMap[Feedback, GlobalTrack] 237 | private[reflow] lazy val obtainer = getAllItems _ 238 | 239 | def getAllItems = globalTrackMap.readOnlySnapshot.values 240 | 241 | def registerObserver(observer: GlobalTrackObserver)(implicit poster: Poster): Unit = { 242 | val feedback = new Feedback4Observer(observer.ensuring(_.nonNull)) 243 | observersMap.put(observer, feedback) 244 | } 245 | 246 | def unregisterObserver(observer: GlobalTrackObserver): Unit = observersMap.remove(observer) 247 | 248 | trait GlobalTrackObserver { 249 | type All = obtainer.type 250 | 251 | /** 252 | * 当任何任务流有更新时，会回调本方法。 253 | * 254 | * @param current 当前变化的`全局跟踪器`。 255 | * @param items 用于获得当前全部的跟踪器。用法示例：{{{items().foreach(println(_))}}}。 256 | */ 257 | def onUpdate(current: GlobalTrack, items: All): Unit 258 | } 259 | 260 | private class Feedback4Observer(observer: GlobalTrackObserver)(implicit poster: Poster) extends TAG.ClassName { 261 | private def reportOnUpdate(gt: GlobalTrack): Unit = eatExceptions(observer.onUpdate(gt, obtainer)) 262 | 263 | def onPending(gt: GlobalTrack): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 264 | def onStart(gt: GlobalTrack): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 265 | 266 | def onProgress(gt: GlobalTrack, progress: Feedback.Progress, out: Out, fromDepth: Int): Unit = if (poster.isNull) reportOnUpdate(gt.progress(progress)) else poster.post(reportOnUpdate(gt.progress(progress))) 267 | def onComplete(gt: GlobalTrack, out: Out): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 268 | def onUpdate(gt: GlobalTrack, out: Out): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 269 | def onAbort(gt: GlobalTrack, trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 270 | def onFailed(gt: GlobalTrack, trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = if (poster.isNull) reportOnUpdate(gt) else poster.post(reportOnUpdate(gt)) 271 | } 272 | 273 | private[reflow] class Feedback4GlobalTrack extends Feedback with TAG.ClassName { 274 | private lazy val tracker: GlobalTrack = globalTrackMap(this) 275 | 276 | override def onPending(): Unit = observersMap.snapshot.values.foreach(_.onPending(tracker)) 277 | override def onStart(): Unit = observersMap.snapshot.values.foreach(_.onStart(tracker)) 278 | 279 | override def onProgress(progress: Feedback.Progress, out: Out, fromDepth: Int): Unit = 280 | observersMap.snapshot.values.foreach(_.onProgress(tracker, progress, out, fromDepth)) 281 | 282 | override def onComplete(out: Out): Unit = { 283 | observersMap.snapshot.values.foreach(_.onComplete(tracker, out)) 284 | if (tracker.scheduler.isDone) globalTrackMap.remove(this) 285 | } 286 | 287 | override def onUpdate(out: Out): Unit = { 288 | observersMap.snapshot.values.foreach(_.onUpdate(tracker, out)) 289 | globalTrackMap.remove(this) 290 | } 291 | 292 | override def onAbort(trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = { 293 | observersMap.snapshot.values.foreach(_.onAbort(tracker, trigger, parent, depth)) 294 | globalTrackMap.remove(this) 295 | } 296 | 297 | override def onFailed(trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = { 298 | observersMap.snapshot.values.foreach(_.onFailed(tracker, trat, parent, depth, e)) 299 | globalTrackMap.remove(this) 300 | } 301 | } 302 | } 303 | 304 | ////////////////////////////////////////////////////////////////////////////////////// 305 | //********************************** Reflow Impl **********************************// 306 | 307 | private[reflow] class Impl private[reflow] (override val basis: Dependency.Basis, inputRequired: immutable.Map[String, KvTpe[_ <: AnyRef]]) extends Reflow(basis: Dependency.Basis) with TAG.ClassName { 308 | 309 | override private[reflow] def start$(inputs: In, feedback: Feedback, strategy: Strategy, poster: Poster, outer: Env = null, pulse: Pulse.Interact = null, serialNum: Long = -1, globalTrack: Boolean = false): Scheduler.Impl = { 310 | require(feedback.nonNull) 311 | require(strategy.nonNull) 312 | val (reqSet, tranSet) = prepare(inputs, outer, pulse) 313 | val traitIn = new Trait.Input(this, inputs, reqSet) 314 | // 全局记录跟踪。SubReflow 还会再次走到这里，所以仅关注两层进度即可。 315 | lazy val feedback4track = new Feedback4GlobalTrack 316 | lazy val trackStrategy = Strategy.Depth(2) -> Strategy.Fluent 317 | val scheduler = new Scheduler.Impl( 318 | this, 319 | traitIn, 320 | tranSet, 321 | if (globalTrack) trackStrategy.genDelegator(feedback4track).join(strategy.genDelegator(feedback.wizh(poster))) else strategy.genDelegator(feedback.wizh(poster)), 322 | strategy /*由于内部实现仅关注 isFluentMode，本处不需要考虑 trackStrategy。*/, 323 | outer, 324 | pulse, 325 | serialNum, 326 | globalTrack 327 | ) 328 | // 放在异步启动的外面，以防止后面调用`sync()`出现问题。 329 | if (globalTrack) GlobalTrack.globalTrackMap.put(feedback4track, new GlobalTrack(this, scheduler, Option(if (outer.isNull) null else outer.trat.as[ReflowTrait]))) 330 | Reflow.submit$ { scheduler.start$() }(TRANSIENT, P_HIGH) 331 | scheduler 332 | } 333 | 334 | private var preparedSets: (immutable.Set[KvTpe[_ <: AnyRef]], immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]]) = _ 335 | 336 | private def prepare(inputs: In, outer: Env, pulse: Pulse.Interact): (immutable.Set[KvTpe[_ <: AnyRef]], immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]]) = { 337 | def isPulseMode = pulse.nonNull 338 | 339 | if (isPulseMode && preparedSets.nonNull) preparedSets 340 | else { 341 | // requireInputsEnough(inputs, inputRequired) // 有下面的方法组合，不再需要这个。 342 | val required = inputRequired.mutable 343 | val tranSet = inputs.trans.mutable 344 | val realIn = putAll(new mutable.AnyRefMap[String, KvTpe[_ <: AnyRef]], inputs.keys) 345 | consumeTranSet(tranSet, required, realIn, check = true, trim = true) 346 | val reqSet = required.values.toSet 347 | requireRealInEnough(reqSet, realIn) 348 | if (debugMode) { 349 | import Logger._ 350 | logger.w("[start]required:%s, inputTrans:%s.", reqSet, tranSet) 351 | logger.i("[start]intents:%s, parent:%s.", basis.traits.map(_.name$).mkString(", ").s, if (outer.isNull) null else outer.trat.name$.s) 352 | } 353 | if (isPulseMode) { 354 | preparedSets = (reqSet, tranSet.toSet) 355 | preparedSets 356 | } else (reqSet, tranSet.toSet) 357 | } 358 | } 359 | 360 | override def toSub(_name: String, _desc: String = null): ReflowTrait = new ReflowTrait(this) { 361 | override protected def name() = if (_name.isNull) super.name() else _name 362 | override protected def requires() = inputRequired.values.toSet 363 | override protected def outs() = reflow.basis.outs 364 | override protected def desc() = if (_desc.isNull) name$ else _desc 365 | } 366 | 367 | override def fork() = new Impl(basis, inputRequired) 368 | } 369 | } 370 | 371 | abstract class Reflow private[reflow] (val basis: Dependency.Basis) { 372 | /** 373 | * 启动任务。可并行启动多个。 374 | * 375 | * @param inputs 输入内容的加载器。 376 | * @param feedback 事件反馈回调接口。 377 | * @param strategy 进度反馈的优化策略。可以叠加使用，如：`Strategy.Depth(3) -> Strategy.Fluent -> Strategy.Interval(600)`。 378 | * @param poster 转移`feedback`的调用线程, 可为`null`。 379 | * @return `true`启动成功，`false`正在运行。 380 | */ 381 | final def start(inputs: In, feedback: Feedback, globalTrack: Boolean = false)(implicit strategy: Strategy, poster: Poster): Scheduler = start$(inputs, feedback, strategy, poster, globalTrack = globalTrack) 382 | 383 | /** 384 | * 启动一个流处理器[[Pulse]]。 385 | * 386 | * @return `Pulse`实例，可进行无数次的`input(In)`操作。 387 | */ 388 | final def pulse(feedback: Pulse.Feedback, abortIfError: Boolean = false, inputCapacity: Int = Config.DEF.maxPoolSize * 3, execCapacity: Int = 3, globalTrack: Boolean = false)(implicit strategy: Strategy, poster: Poster): Pulse = 389 | new Pulse(requirePulseKeyDiff(this), feedback, abortIfError, inputCapacity, execCapacity, globalTrack) 390 | 391 | private[reflow] def start$(inputs: In, feedback: Feedback, strategy: Strategy, poster: Poster, outer: Env = null, pulse: Pulse.Interact = null, serialNum: Long = -1, globalTrack: Boolean = false): Scheduler.Impl 392 | 393 | /** 394 | * 转换为一个`Trait`（用`Trait`将本`Reflow`打包）以便嵌套构建任务流。 395 | */ 396 | def toSub(name: String, desc: String = null): ReflowTrait 397 | 398 | def fork(): Reflow 399 | } 400 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Scheduler.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S 20 | import hobby.wei.c.reflow.Feedback.Progress.Strategy 21 | import hobby.wei.c.reflow.State._ 22 | import hobby.wei.c.tool.Locker 23 | import java.util.concurrent.locks.ReentrantLock 24 | import scala.collection._ 25 | 26 | /** 27 | * @author Wei Chou(weichou2010@gmail.com) 28 | * @version 1.0, 02/07/2016 29 | */ 30 | trait Scheduler { 31 | 32 | /** @see #sync(boolean, long) */ 33 | @deprecated(message = "好用但应慎用。会 block 住当前线程，几乎是不需要的。", since = "0.0.1") 34 | def sync(reinforce: Boolean = false): Out 35 | 36 | /** 等待任务运行完毕并输出最终结果。如果没有拿到结果(已经{@link #isDone()}), 则会重新{@link Impl#start()} 启动。但这种情况极少见。 37 | * 38 | * @param reinforce 是否等待`reinforce`阶段结束。 39 | * @param milliseconds 延迟的deadline, 单位：毫秒。 40 | * @return 任务的最终结果，不会为`null`。 41 | * @throws InterruptedException 到达deadline了或者被中断。 42 | */ 43 | @deprecated(message = "好用但应慎用。会block住当前线程，几乎是不需要的。", since = "0.0.1") 44 | @throws[InterruptedException] 45 | def sync(reinforce: Boolean, milliseconds: Long): Option[Out] 46 | 47 | def abort(): Unit 48 | 49 | def getState: Tpe 50 | 51 | /** 判断整个任务流是否运行结束。 52 | * 注意: 此时的{@link #getState()}值可能是{@link State#COMPLETED}、{@link State#FAILED}、 53 | * {@link State#ABORTED}或{@link State#UPDATED}中的某一种。 54 | * 55 | * @return true 已结束。 56 | */ 57 | def isDone: Boolean 58 | } 59 | 60 | object Scheduler { 61 | 62 | /** 63 | * @author Wei Chou(weichou2010@gmail.com) 64 | * @version 1.0, 07/08/2016 65 | */ 66 | private[reflow] class Impl( 67 | reflow: Reflow, 68 | traitIn: Trait, 69 | inputTrans: immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]], 70 | feedback: Feedback, 71 | strategy: Strategy, 72 | outer: Env, 73 | pulse: Pulse.Interact, 74 | serialNum: Long, 75 | globalTrack: Boolean 76 | ) extends Scheduler { 77 | private lazy val state = new State$() 78 | 79 | @volatile private var delegatorRef: ref.WeakReference[Tracker.Impl] = _ 80 | 81 | private[reflow] def start$(): this.type = { 82 | var permit = false 83 | Locker.syncr { 84 | if (isDone) { 85 | state.reset() 86 | permit = true 87 | } else { 88 | permit = !state.isOverridden 89 | } 90 | }(state.lock) 91 | if (permit && state.forward(PENDING) /*可看作原子锁*/ ) { 92 | val tracker = new Tracker.Impl(reflow, traitIn, inputTrans, state, feedback, strategy, Option(outer), pulse, serialNum, globalTrack) 93 | // tracker启动之后被线程引用, 任务完毕之后被线程释放, 同时被gc。 94 | // 这里增加一层弱引用, 避免在任务完毕之后得不到释放。 95 | delegatorRef = new ref.WeakReference[Tracker.Impl](tracker) 96 | tracker.start$() 97 | this 98 | } else null 99 | } 100 | 101 | private[reflow] def getDelegator: Option[Tracker.Impl] = J2S.getRef(delegatorRef) 102 | 103 | override def sync(reinforce: Boolean = false): Out = { 104 | try { 105 | sync(reinforce, -1).get 106 | } catch { 107 | case e: InterruptedException => throw new IllegalStateException(e) 108 | } 109 | } 110 | 111 | @throws[InterruptedException] 112 | override def sync(reinforce: Boolean, milliseconds: Long): Option[Out] = { 113 | val begin = System.nanoTime 114 | var loop = true 115 | var delegator: Tracker.Impl = null 116 | while (loop) { 117 | getDelegator.fold { 118 | Option(start$()).fold { 119 | // 如果还没拿到, 说明其他线程也在同时start(). 120 | Thread.`yield`() // 那就等一下下再看看 121 | } { _ => } // 重启成功，再走一次循环拿值。 122 | } { d => 123 | delegator = d 124 | loop = false 125 | } 126 | } 127 | delegator.sync(reinforce, if (milliseconds == -1) -1 else milliseconds - ((System.nanoTime - begin) / 1e6).toLong) 128 | } 129 | 130 | override def abort(): Unit = getDelegator.fold()(_.abort()) 131 | 132 | override def getState: State.Tpe = state.get 133 | 134 | override def isDone: Boolean = { 135 | val state = this.state.get 136 | state == COMPLETED && getDelegator.fold(true /*若引用释放,说明任务已不被线程引用,即运行完毕。*/ ) { 137 | !_.isReinforceRequired 138 | } || state == FAILED || state == ABORTED || state == UPDATED 139 | } 140 | } 141 | 142 | class State$ { 143 | implicit lazy val lock: ReentrantLock = Locker.getLockr(this) 144 | 145 | @volatile private var state = State.IDLE 146 | @volatile private var state$ = State.IDLE 147 | @volatile private var overridden = false 148 | 149 | def forward(state: State.Tpe): Boolean = Locker.syncr { 150 | if (this.state.canOverrideWith(state)) { 151 | this.state = state 152 | this.state$ = state 153 | if (!overridden) overridden = true 154 | true 155 | } else false 156 | } 157 | 158 | /** 159 | * 更新中断后的状态。 160 | * 161 | * @return 返回值与forward(State)方法互补的值。 162 | */ 163 | def abort(): Boolean = Locker.syncr { 164 | state$ = State.ABORTED 165 | state match { 166 | case State.REINFORCE_PENDING | State.REINFORCING => 167 | state = State.COMPLETED 168 | true 169 | case State.COMPLETED | State.UPDATED => true 170 | case _ => false 171 | } 172 | } 173 | 174 | def get: Tpe = state 175 | 176 | def get$ : Tpe = state$ 177 | 178 | private[reflow] def reset(): Unit = Locker.syncr { 179 | state = State.IDLE 180 | state$ = State.IDLE 181 | } 182 | 183 | /** 184 | * 可用于标识是否启动过。 185 | */ 186 | private[reflow] def isOverridden: Boolean = overridden 187 | } 188 | } 189 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/State.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.wei.c.anno.proguard.{KeepMp$, KeepVp$} 20 | 21 | /** 22 | * @author Wei Chou(weichou2010@gmail.com) 23 | * @version 1.0, 22/07/2016 24 | */ 25 | @KeepVp$ 26 | @KeepMp$ // 保留枚举名称 27 | object State extends Enumeration { 28 | type Tpe = State 29 | 30 | private[reflow] case class State(group: Int, sn: Int) extends Val { 31 | def canOverrideWith(state: Tpe): Boolean = ( 32 | state.group == group && Math.abs(state.sn - sn) == 1 /*自己不能覆盖自己*/ 33 | || state.group == group + 1 34 | || state.group > group + 1 && state.sn == sn 35 | ) 36 | } 37 | 38 | val IDLE = State(0, 0) 39 | /** 40 | * 队列提交之后或任务之间的等待（调度或其它高优先级任务）间隙。 41 | */ 42 | val PENDING = State(1, 10) 43 | val EXECUTING = State(1, 11) 44 | /** 45 | * 第一阶段完成, 不代表结束。 46 | * 要查询是否结束, 请使用{@link Scheduler#isDone()}. 47 | */ 48 | val COMPLETED = State(2, 100) 49 | val FAILED = State(2, 200) 50 | val ABORTED = State(2, 300) 51 | /** 52 | * [强化]运行任务的等待间隙。同{@link #PENDING} 53 | */ 54 | val REINFORCE_PENDING = State(5, 100 /*只能override COMPLETED*/) 55 | /** 56 | * [强化]运行状态。每个任务流都可以选择先[摘要]再[强化]运行。 57 | *

58 | * 摘要: 第一遍快速给出结果。可类比为阅读的[浏览]过程; 59 | * 强化: 第二遍精细严格的运行。可类比为阅读的精读。 60 | *

61 | * 摘要过程, 会执行完整的任务流; 62 | * 强化过程, 仅从第一个请求强化运行的任务开始直到完成整个任务流。第一个任务不会重新输入参数。 63 | */ 64 | val REINFORCING = State(5, 101) 65 | /** 66 | * 完成了REINFORCING. 67 | */ 68 | val UPDATED = State(6 /*只能override REINFORCING*/ , 1000) 69 | } 70 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Task.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S.NonNull 20 | import hobby.wei.c.anno.proguard.Keep$$ 21 | import hobby.wei.c.reflow.Assist._ 22 | import hobby.wei.c.reflow.Feedback.Progress 23 | import hobby.wei.c.reflow.Feedback.Progress.Weight 24 | import hobby.wei.c.reflow.Tracker.Runner 25 | import hobby.wei.c.tool.Locker 26 | import java.util.concurrent.locks.ReentrantLock 27 | import scala.collection._ 28 | 29 | /** 30 | * 这里用于编写客户任务代码（重写`doWork()`方法）。注意不可以写异步任务和线程挂起等操作。 31 | * 32 | * @author Wei Chou(weichou2010@gmail.com) 33 | * @version 1.0, 26/06/2016; 34 | * 2.3, 04/01/2021, 增加`autoProgress`控制。 35 | */ 36 | abstract class Task protected () { 37 | 38 | @Keep$$ 39 | private implicit lazy val lock: ReentrantLock = Locker.getLockr(this) 40 | 41 | @volatile private var env$ : Env = _ 42 | @volatile private var thread: Thread = _ 43 | @volatile private var aborted: Boolean = _ 44 | @volatile private var working: Boolean = _ 45 | 46 | /** @return 与本任务相关的执行环境对象。 */ 47 | final def env = env$ 48 | 49 | /** @return 与本任务相关的接口及调度参数信息对象。 */ 50 | final def trat: Trait = env$.trat 51 | 52 | /** 取得输入的 value。 53 | * 54 | * @param key value 对应的 key。 55 | * @tparam T value 的类型参数。 56 | * @return `Option[T]`. 57 | */ 58 | final def input[T >: Null](key: String): Option[T] = env$.input.get(key) 59 | 60 | final def input[T >: Null <: AnyRef](kce: KvTpe[T]): Option[T] = input(kce.key) 61 | 62 | /** 输出结果。 */ 63 | final def output[T](key: String, value: T): Unit = env$.out.put(key, value) 64 | 65 | final def output[T <: AnyRef](kce: KvTpe[T], value: T): Unit = output(kce.key, value) 66 | 67 | final def output(map: Map[String, Any]): Unit = map.foreach { m: (String, Any) => output(m._1, m._2) } 68 | 69 | /** 如果[[isReinforceRequired]]为`true`或在`Pulse`中，则缓存一些参数用于再次执行时使用。 70 | * 问: 为何不直接用全局变量来保存? 71 | * 答: 下次并不重用本对象。 72 | */ 73 | final def cache[T](key: String, value: T): Unit = { 74 | require(env$.isPulseMode || env$.isReinforceRequired, "`cache()`操作必须在`requireReinforce()`之后。") 75 | env$.input.cache(key, null) // 仅用来测试 key 是否重复，null 值不会被放进去。 76 | env$.cache(key, value) 77 | } 78 | 79 | final def cache[T <: AnyRef](kce: KvTpe[T], value: T): Unit = cache(kce.key, value) 80 | 81 | /** 发送一个进度。 82 | * @param _progress 进度百分比，取值区间[0, 1]，必须是递增的。 83 | */ 84 | final def progress(_progress: Float, publish: Boolean): Unit = { 85 | val s = String.valueOf(_progress) 86 | val unt = math.pow(10, s.length - (s.indexOf('.') + 1)).toInt 87 | progress((_progress * unt).round, unt, publish) 88 | } 89 | 90 | /** 发送一个进度。 91 | * @param step 进度的分子。必须是递增的。 92 | * @param sum 进度的分母。必须大于等于`step`且不可变。 93 | */ 94 | final def progress(step: Int, sum: Int, publish: Boolean = true): Unit = env$.tracker.onTaskProgress( 95 | trat, 96 | Progress(sum, step.ensuring(_ <= /*这里必须可以等于*/ sum), Weight(step, 1, env$.weightPar)), 97 | env$.out, 98 | env$.depth, 99 | publish 100 | ) 101 | 102 | /** 请求强化运行。 103 | * @return （在本任务或者本次调用）之前是否已经请求过, 同`isReinforceRequired()`。 104 | */ 105 | final def requireReinforce() = env$.requireReinforce() 106 | 107 | /** @return 当前是否已经请求过强化运行。 */ 108 | final def isReinforceRequired: Boolean = env$.isReinforceRequired 109 | 110 | /** @return 当前是否处于强化运行阶段。 */ 111 | final def isReinforcing: Boolean = env$.isReinforcing 112 | 113 | /** @return 当前任务所在的沙盒`Reflow`是不是（被包装在一个`Trait`里面被调度运行的）子`Reflow`。 */ 114 | final def isSubReflow: Boolean = env$.isSubReflow 115 | 116 | final def isAborted: Boolean = aborted 117 | 118 | /** 如果认为任务失败, 则应该主动调用本方法来强制结束任务。 119 | * 不设计成直接声明[[doWork]]方法 throws 异常, 是为了让客户代码尽量自己处理好异常, 以防疏忽。 120 | * 121 | * @param e 自定义异常，可以为`null`。 122 | */ 123 | final def failed(e: Exception = null) = { 124 | // 简单粗暴的抛出去, 由 Runner 统一处理。 125 | // 这里不抛出 Exception 的子类，是为了防止被客户代码错误的给 catch 住。 126 | // 但是 exec() 方法 catch 了本 Error 并转换为正常的 Assist.FailedException。 127 | throw new FailedError(e) 128 | } 129 | 130 | /** 如果子类在[[doWork]]中检测到了中断请求(如: 在循环里判断[[isAborted]]), 131 | * 应该在处理好了当前事宜、准备好中断的时候调用本方法以中断整个任务。 132 | */ 133 | final def abortionDone = throw new AbortError() 134 | 135 | @throws[CodeException] 136 | @throws[AbortException] 137 | @throws[FailedException] 138 | private[reflow] final def exec(_env: Env, _runner: Runner): Boolean = { 139 | env$ = _env 140 | Locker.syncr { 141 | if (aborted) return false 142 | thread = Thread.currentThread() 143 | working = true 144 | } 145 | try { 146 | exec$(_env, _runner) 147 | } catch { 148 | case e: FailedError => 149 | throw new FailedException(e.getCause) 150 | case e: AbortError => 151 | throw new AbortException(e.getCause) 152 | case e: Exception => 153 | // 各种非正常崩溃的 RuntimeException，如 NullPointerException 等。 154 | throw new CodeException(e) 155 | } finally { 156 | // 不能置为 false, 不然异步执行`exec$()`时，obort() 请求传不到 onAbort()。 157 | // working = false // 节省一个`async`变量 158 | thread = null 159 | } 160 | } 161 | 162 | /** 163 | * @return 同步还是异步执行。 164 | */ 165 | private[reflow] def exec$(_env: Env, _runner: Runner): Boolean = { 166 | // 这里反馈进度有两个用途: 1.Feedback subProgress; 2.并行任务进度统计。 167 | progress(0, 10, publish = autoProgress) 168 | doWork() 169 | progress(10, 10, publish = autoProgress) 170 | true 171 | } 172 | 173 | private[reflow] final def abort(): Unit = { 174 | Locker.syncr { 175 | aborted = true 176 | } 177 | if (working) { // || async 178 | try { 179 | onAbort() 180 | } finally { 181 | val t = thread 182 | // 这个中断信号对框架毫无影响： 183 | // 1. 对于外部对`sync()`的调用，只有外部调用`sync()`的线程对象发出的中断信号才对它起作用； 184 | // 2. 框架内部没有监听这个信号，`Tracker.interruptSync()`的实现也没有。 185 | if (t.nonNull) t.interrupt() 186 | } 187 | } 188 | } 189 | 190 | /** 某些任务只在满足条件时运行，其它情况下隐藏。 */ 191 | protected def autoProgress: Boolean = true 192 | 193 | /** 客户代码的扩展位置。
194 | * 重写本方法应该在必要时监听`isAborted`或`Thread.interrupt()`信号从而及时中断。
195 | * 注意：必须仅有同步代码，不可以执行异步操作（如果有异步需求，应该运用本`Reflow`框架的思想去实现并行化）。 196 | */ 197 | protected def doWork(): Unit 198 | 199 | /** 一般不需要重写本方法，只需在`doWork()`中检测`isAborted`标识即可。 200 | * 重写本方法以获得中断通知，并处理中断后的收尾工作。
201 | * 注意：本方法的执行与`doWork()`并不在同一线程，需谨慎处理。
202 | */ 203 | protected def onAbort(): Unit = {} 204 | 205 | /** 当同一个任务被放到多个[[Reflow]]中运行，而某些代码段需要 Class 范围的串行化时，应该使用本方法包裹。 206 | * 注意: 不要使用 synchronized 关键字，因为它在等待获取锁的时候不能被中断，而本方法使用[[ReentrantLock]]锁机制， 207 | * 当[[abort]]请求到达时，如果还处于等待获取锁状态，则可以立即中断。 208 | * 209 | * @param codes 要执行的代码段。 210 | * @param scope 锁的作用范围。通常应该写某 Task 子类的`Xxx.class`或[[classOf]][Xxx]，而不要去[[getClass]]， 211 | * 因为如果该类再有一个子类, 本方法在不同的实例返回不同的 Class 对象。scope 不同，同步目的将失效。 212 | * @return codes 的返回值。 213 | */ 214 | final def sync[T](scope: Class[_ <: Task])(codes: => T): T = sync(() => codes, scope) 215 | 216 | final def sync[T](codes: Locker.CodeZ[T], scope: Class[_ <: Task]): T = { 217 | try Locker.sync(codes, scope.ensuring(_.nonNull)) 218 | catch { 219 | case e: InterruptedException => 220 | assert(isAborted) 221 | throw new AbortError(e) 222 | } 223 | } 224 | } 225 | 226 | object Task { 227 | 228 | private[reflow] def apply(f: Context => Unit): Task = new Context { 229 | override protected def doWork(): Unit = f(this) 230 | } 231 | abstract class Context private[reflow] () extends Task 232 | } 233 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/ThreadResetor.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | /** 20 | * For Android: 21 | * 22 | * android.os.Process.setThreadPriority(thread.getId(), android.os.Process.THREAD_PRIORITY_BACKGROUND); 23 | * 24 | * 25 | * @author Wei Chou(weichou2010@gmail.com) 26 | * @version 1.0, 05/01/2018 27 | */ 28 | trait ThreadResetor { 29 | /** 30 | * @param thread 需要被重置的线程。 31 | * @param beforeOrOfterWork 在运行任务之前还是之后。`true`表示之前。 32 | * @param runOnCurrentThread 都是本调用在`thread`指定的线程内。 33 | */ 34 | def reset(thread: Thread, beforeOrOfterWork: Boolean, runOnCurrentThread: Boolean): Unit = { 35 | if (thread.getPriority != Thread.NORM_PRIORITY) thread.setPriority(Thread.NORM_PRIORITY) 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Trait.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import java.util.concurrent.atomic.AtomicLong 20 | import hobby.chenai.nakam.lang.J2S.NonNull 21 | import hobby.wei.c.reflow.Assist._ 22 | import hobby.wei.c.reflow.Reflow.{Period, _} 23 | import hobby.wei.c.reflow.Tracker.SubReflowTask 24 | import hobby.wei.c.reflow.implicits.none 25 | import scala.collection.{mutable, _} 26 | import scala.collection.mutable.ListBuffer 27 | 28 | /** 用于发布[[Task]]的 I/O 接口及调度策略信息。而[[Task]]本身仅用于定义任务实现。 29 | *

30 | * 注意: 实例不应保留状态。 31 | * 32 | * @author Wei Chou(weichou2010@gmail.com) 33 | * @version 1.0, 12/04/2015 34 | */ 35 | trait Trait extends Equals { 36 | val is4Reflow: Boolean = false 37 | 38 | /** 任务名称。 */ 39 | protected def name(): String 40 | 41 | /** 创建任务。 */ 42 | /*protected*/ 43 | def newTask(): Task 44 | 45 | /** 必须输入的参数 keys 及 value 类型（可由初始参数传入, 或者在本Task前面执行的`Tasks`输出[[outs]]而获得）。 */ 46 | protected def requires(): immutable.Set[KvTpe[_ <: AnyRef]] 47 | 48 | /** 该任务输出的所有 key-value 类型。 */ 49 | protected def outs(): immutable.Set[KvTpe[_ <: AnyRef]] 50 | 51 | /** 优先级。范围 [ [[P_HIGH]] ~ [[P_LOW]] ]。 */ 52 | protected def priority(): Int 53 | 54 | /** 任务大概时长。 */ 55 | protected def period(): Period.Tpe 56 | 57 | /** 任务描述, 将作为进度反馈的部分信息。 */ 58 | protected def desc(): String 59 | 60 | lazy val name$ : String = name().ensuring(_.nonEmpty) 61 | lazy val requires$ : immutable.Set[KvTpe[_ <: AnyRef]] = requireKkDiff(requireElemNonNull(requires())) 62 | lazy val outs$ : immutable.Set[KvTpe[_ <: AnyRef]] = requireKkDiff(requireElemNonNull(outs())) 63 | lazy val priority$ : Int = between(P_HIGH, priority(), P_LOW).toInt 64 | lazy val period$ : Period.Tpe = period().ensuring(_.nonNull) 65 | lazy val desc$ : String = desc().ensuring(_.nonNull /*可以是""*/ ) 66 | 67 | override def equals(any: scala.Any) = super.equals(any) 68 | override def canEqual(that: Any) = super.equals(that) 69 | override def hashCode() = super.hashCode() 70 | 71 | override def toString = "name:%s, requires:%s, out:%s, priority:%s, period:%s, description: %s" format ( 72 | name$, requires$, outs$, priority$, period$, desc$ 73 | ) 74 | } 75 | 76 | object Trait { 77 | 78 | @deprecated 79 | def apply(_name: String, _period: Period.Tpe, _outs: immutable.Set[KvTpe[_ <: AnyRef]] = none, _requires: immutable.Set[KvTpe[_ <: AnyRef]] = none, _priority: Int = Reflow.P_NORMAL, _desc: String = null)( 80 | _dosth: Task.Context => Unit 81 | ): Trait = new Trait { 82 | override protected def name() = _name 83 | override protected def requires() = _requires 84 | override protected def outs() = _outs 85 | override protected def priority() = _priority 86 | override protected def period() = _period 87 | override protected def desc() = if (_desc.isNull) name$ else _desc 88 | override def newTask() = Task(_dosth) 89 | } 90 | 91 | private final val sCount = new AtomicLong(0) 92 | 93 | private[reflow] final class Parallel private[reflow] (trats: Trait*) extends Trait { 94 | private[reflow] def this(t: Trait) = this(Seq(t): _*) 95 | private var _traits: List[Trait] = Nil 96 | _traits :::= trats.toList 97 | 98 | private[reflow] def traits() = _traits 99 | private[reflow] def add(t: Trait): Unit = _traits ::= t.ensuring(!_.isInstanceOf[Parallel]) 100 | private[reflow] def first(): Trait = _traits.last // 没错，没把 list.reverse。 101 | private[reflow] def last(): Trait = _traits.head 102 | 103 | override protected def name() = classOf[Parallel].getName + "#" + sCount.incrementAndGet 104 | override def newTask() = ??? 105 | override protected def requires() = none 106 | override protected def outs() = none 107 | override protected def priority() = Reflow.P_NORMAL 108 | override protected def period() = Period.LONG 109 | override protected def desc() = name$ 110 | } 111 | 112 | trait Adapter extends Trait { 113 | override protected def name() = classOf[Adapter].getName + "#" + sCount.incrementAndGet 114 | override protected def requires() = none 115 | override protected def outs() = none 116 | override protected def priority() = Reflow.P_NORMAL 117 | override protected def desc() = name$ 118 | } 119 | 120 | private[reflow] final class Input(reflow: Reflow, in: In, outsTrimmed: immutable.Set[KvTpe[_ <: AnyRef]]) extends Adapter { 121 | override protected def name() = classOf[Input].getName + "#" + sCount.incrementAndGet 122 | 123 | override def newTask() = new Task { 124 | override protected def doWork(): Unit = in.fillValues(env.out) 125 | } 126 | 127 | override protected def outs() = outsTrimmed 128 | override protected def priority() = reflow.basis.first(child = true).get.priority$ 129 | override protected def period() = Period.TRANSIENT 130 | } 131 | 132 | abstract class ReflowTrait private[reflow] (val reflow: Reflow) extends Trait { 133 | override final val is4Reflow = true 134 | 135 | override protected def name() = classOf[ReflowTrait].getName + "#" + sCount.incrementAndGet 136 | override final def newTask() = new SubReflowTask() 137 | override protected final def priority() = reflow.basis.first(child = true).get.priority$ 138 | // 这只是一个外壳，调度瞬间完成。子任务执行时，这层壳不会阻塞线程（事件回调机制）。 139 | override protected final def period() = Period.TRANSIENT 140 | } 141 | } 142 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Transformer.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.lang.J2S.NonNull 20 | 21 | import scala.collection._ 22 | 23 | /** 24 | * 任务输出转换器。包括`key`和`value`的转换， 25 | * 可定义仅转换`value`、或仅转换`key`、或`key-value`全都转换。 26 | * 27 | * @constructor 对于要将某key-value转换为其它key'-value'的, 应使用本构造方法。 28 | * @author Wei Chou(weichou2010@gmail.com) 29 | * @version 1.0, 31/07/2016 30 | */ 31 | abstract class Transformer[IN <: AnyRef, OUT <: AnyRef] private(_in: KvTpe[IN], _out: KvTpe[OUT], _keyIn: String, _keyOut: String) extends Equals { 32 | protected def this(keyIn: String, keyOut: String) = this(null, null, keyIn, keyOut) 33 | 34 | protected def this(in: KvTpe[IN], out: KvTpe[OUT]) = this(in, out, null, null) 35 | 36 | lazy val in: KvTpe[IN] = if (_in.nonNull) _in else new KvTpe[IN](_keyIn, this.getClass, 0) {} 37 | lazy val out: KvTpe[OUT] = if (_out.nonNull) _out else new KvTpe[OUT](_keyOut, this.getClass, 1) {} 38 | 39 | final def transform(input: Map[String, _]): Option[OUT] = transform(in.takeValue(input)) 40 | 41 | def transform(in: Option[IN]): Option[OUT] 42 | 43 | override def equals(any: Any): Boolean = any match { 44 | case that: Transformer[_, _] if that.canEqual(this) => 45 | that.in == this.in && that.out == this.out 46 | case _ => false 47 | } 48 | 49 | override def canEqual(that: Any) = that.isInstanceOf[Transformer[_ <: AnyRef, _ <: AnyRef]] 50 | 51 | override def hashCode = in.hashCode * 41 + out.hashCode 52 | 53 | override def toString = s"${classOf[Transformer[_, _]].getSimpleName}[$in -> $out]" 54 | } 55 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/Worker.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2017-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.lang.J2S.{NonFlat$, NonNull} 21 | import hobby.chenai.nakam.lang.TypeBring.AsIs 22 | import hobby.wei.c.log.Logger._ 23 | import hobby.wei.c.reflow.Reflow.{logger => log, _} 24 | import hobby.wei.c.tool.Snatcher 25 | import java.util.concurrent._ 26 | import java.util.concurrent.atomic.AtomicInteger 27 | import scala.collection.JavaConversions.collectionAsScalaIterable 28 | 29 | /** 30 | * 优化的线程池实现。 31 | * 32 | * @author Wei Chou(weichou2010@gmail.com) 33 | * @version 1.0, 11/02/2017; 34 | * 2.0, 05/01/2022, 重构。 35 | */ 36 | object Worker extends TAG.ClassName { 37 | 38 | private final lazy val sThreadFactory = new ThreadFactory() { 39 | private val mIndex = new AtomicInteger(0) 40 | 41 | def newThread(runnable: Runnable): Thread = { 42 | val thread = new Thread(runnable, "pool-thread-" + Worker.getClass.getName + "#" + mIndex.getAndIncrement) 43 | resetThread(thread, beforeOrOfterWork = true, runOnCurrentThread = false) 44 | thread 45 | } 46 | } 47 | 48 | @volatile 49 | private var sThreadResetor = new ThreadResetor() {} 50 | 51 | private def resetThread(thread: Thread, beforeOrOfterWork: Boolean, runOnCurrentThread: Boolean) { 52 | sThreadResetor.reset(thread, beforeOrOfterWork, runOnCurrentThread) 53 | Thread.interrupted() 54 | if (thread.isDaemon) thread.setDaemon(false) 55 | } 56 | 57 | private final lazy val sPoolWorkQueue: BlockingQueue[Runnable] = new LinkedTransferQueue[Runnable]() { 58 | 59 | override def offer(r: Runnable) = { 60 | /* 如果不放入队列并返回 false，会迫使增加线程。但是这样又会导致总是增加线程，而空闲线程得不到重用。 61 | 因此在有空闲线程的情况下就直接放入队列。若大量长任务致使线程数增加到上限， 62 | 则 threadPool 启动 reject 流程（见 ThreadPoolExecutor 构造器的最后一个参数），此时再插入到本队列。 63 | 这样即完美实现[先增加线程数到最大，再入队列，空闲释放线程]这个基本逻辑。*/ 64 | val b = sThreadPoolExecutor.getActiveCount < sThreadPoolExecutor.getPoolSize && super.offer(r) 65 | Assist.Monitor.threadPool(sThreadPoolExecutor, b, reject = false) 66 | b 67 | } 68 | } 69 | 70 | lazy val sThreadPoolExecutor: ThreadPoolExecutor = { 71 | val config = Reflow.config // format: off 72 | new ThreadPoolExecutor(config.corePoolSize, config.maxPoolSize, 73 | config.keepAliveTime, TimeUnit.SECONDS, sPoolWorkQueue, sThreadFactory, (r: Runnable, executor: ThreadPoolExecutor) => { 74 | try { 75 | while (!sPoolWorkQueue.offer(r, 0, TimeUnit.MILLISECONDS)) { 76 | if (debugMode) log.w("[sPoolWorkQueue]########## times loop offer(%s, 0, TimeUnit.MILLISECONDS).", r) 77 | Thread.`yield`() 78 | } 79 | Assist.Monitor.threadPool(sThreadPoolExecutor, addThread = false, reject = true) 80 | } catch { 81 | case ignore: InterruptedException /*不可能出现*/ => 82 | throw ignore 83 | } 84 | }) 85 | } // format: on 86 | 87 | private[reflow] def setThreadResetor(resetor: ThreadResetor) = sThreadResetor = resetor 88 | 89 | private[reflow] def updateConfig(config: Config) { 90 | sThreadPoolExecutor.setCorePoolSize(config.corePoolSize) 91 | sThreadPoolExecutor.setMaximumPoolSize(config.maxPoolSize) 92 | sThreadPoolExecutor.setKeepAliveTime(config.keepAliveTime, TimeUnit.SECONDS) 93 | } 94 | 95 | object sPreparedBuckets { 96 | val sTransient = new PriorityBlockingQueue[Runner] 97 | val sShort = new PriorityBlockingQueue[Runner] 98 | val sLong = new PriorityBlockingQueue[Runner] 99 | val sInfinite = new PriorityBlockingQueue[Runner] 100 | val sQueues = Array[BlockingQueue[Runner]](sTransient, sShort, sLong, sInfinite) 101 | 102 | def queue4(period: Period.Tpe): BlockingQueue[Runner] = { 103 | import Period._ 104 | period match { 105 | case TRANSIENT => sTransient 106 | case SHORT => sShort 107 | case LONG => sLong 108 | case INFINITE => sInfinite 109 | case _ => sLong 110 | } 111 | } 112 | } 113 | 114 | private object sExecuting { 115 | val sTransient = new AtomicInteger(0) 116 | val sShort = new AtomicInteger(0) 117 | val sLong = new AtomicInteger(0) 118 | val sInfinite = new AtomicInteger(0) 119 | val sCounters = Array[AtomicInteger](sTransient, sShort, sLong, sInfinite) 120 | } 121 | 122 | private val sSnatcher = new Snatcher 123 | 124 | def scheduleRunner(runner: Runner, bucket: Boolean = true): Unit = { 125 | if (debugMode) log.i("[scheduleBuckets]>>>>>>>>>> runner:%s.", runner) 126 | var i = 0 127 | while (!sPreparedBuckets.queue4(runner.trat.period$).offer(runner)) { 128 | if (i > 3) log.w("[scheduleBuckets]########## %s times loop offer(%s).", i, runner) 129 | Thread.`yield`() 130 | i += 1 131 | } 132 | if (bucket) scheduleBuckets() 133 | } 134 | 135 | def scheduleBuckets(): Unit = sSnatcher.tryOn { 136 | if (debugMode) log.i("[scheduleBuckets]>>>>>>>>>> bucket queues sizes:%s.", sPreparedBuckets.sQueues.map(_.size).mkString$.s) 137 | val executor = sThreadPoolExecutor 138 | 139 | var runner: Runner = null 140 | var index = -1 141 | def next() { 142 | val allowRunLevel = { 143 | val maxPoolSize = executor.getMaximumPoolSize 144 | // sTransient 和 sShort 至少会有一个线程，策略就是拼优先级了。不过如果线程已经满载， 145 | // 此时即使有更高优先级的任务到来，那也得等着，谁叫你来的晚呢！ 146 | /*if (sExecuting.sShort.get() + sExecuting.sLong.get() 147 | + sExecuting.sInfinite.get() >= maxPoolSize) { 148 | allowRunLevel = 1; 149 | } else*/ 150 | // 给短任务至少留一个线程，因为后面可能还会有高优先级的短任务。 151 | // 但假如只有 3 个以内的线程，其中 2 个被 sInfinite 占用，怎么办呢？ 152 | // 1. 有可能某系统根本就没有 sLong 任务，那么剩下的一个刚好留给短任务； 153 | // 2. 增加一个最大线程数通常不会对系统造成灾难性的影响，那么应该修改配置 Config。 154 | if (sExecuting.sLong.get() + sExecuting.sInfinite.get() >= maxPoolSize - 1) { 155 | 1 156 | } 157 | // 除了长连接等少数长任务外，其它几乎都可以拆分成短任务，因此这里必须限制数量。 158 | else if (sExecuting.sInfinite.get() >= maxPoolSize * 2 / 3) { 159 | 2 160 | } else 3 161 | } 162 | for (i <- 0 to (allowRunLevel min (sPreparedBuckets.sQueues.length - 1))) { 163 | val r = sPreparedBuckets.sQueues(i).peek() 164 | if ( 165 | r.nonNull && (runner.isNull || // 值越小优先级越大 166 | ((r.trat.priority$ + r.trat.period$.weight /*采用混合优先级*/ ) 167 | < runner.trat.priority$ + runner.trat.period$.weight)) 168 | ) { 169 | runner = r 170 | index = i 171 | if (debugMode) log.i("[scheduleBuckets]>>>>>>>>>> preparing exec >: index:%d, runner:%s.", index, runner) 172 | } 173 | } 174 | } 175 | def hasNext: Boolean = { next(); runner.nonNull } 176 | while (hasNext) { 177 | // 队列元素的顺序可能发生改变，不能用 poll(); 而 remove() 是安全的：runner 都是重新 new 出来的，不会出现重复。 178 | while (!sPreparedBuckets.sQueues(index).remove(runner)) { 179 | if (debugMode) log.w("[scheduleBuckets]########## times loop remove(%s).", runner) 180 | Thread.`yield`() 181 | } 182 | sExecuting.sCounters(index).incrementAndGet 183 | 184 | val r = runner; val i = index 185 | runner = null; index = -1 186 | 187 | executor.execute(() => { 188 | resetThread(Thread.currentThread, beforeOrOfterWork = true, runOnCurrentThread = true) 189 | try { r.run() } 190 | catch { 191 | case ignore: Throwable => Assist.Monitor.threadPoolError(ignore) 192 | } finally { 193 | sExecuting.sCounters(i).decrementAndGet 194 | if (debugMode) log.i("[scheduleBuckets]<<<<<<<<<< exec counters:%s.", sExecuting.sCounters.map(_.get).mkString$.s) 195 | resetThread(Thread.currentThread, beforeOrOfterWork = false, runOnCurrentThread = true) 196 | scheduleBuckets() 197 | } 198 | }) 199 | } 200 | if (debugMode) log.w("[scheduleBuckets]<<<<<<<<<< all done.") 201 | } 202 | 203 | class Runner(val trat: Trait, runnable: Runnable) extends Runnable with Comparable[Runner] { 204 | override def compareTo(o: Runner) = Integer.compare(trat.priority$, o.trat.priority$) 205 | 206 | override def run(): Unit = runnable.run() 207 | } 208 | } 209 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/reflow/implicits.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.reflow 18 | 19 | import hobby.wei.c.anno.proguard.{KeepMp$, KeepVp$} 20 | import hobby.wei.c.reflow.Reflow.Period 21 | import hobby.wei.c.reflow.lite.{Lite, Par, Par2, Serial} 22 | import scala.collection.immutable 23 | import scala.language.implicitConversions 24 | import scala.reflect.ClassTag 25 | 26 | /** 27 | * @author Wei Chou(weichou2010@gmail.com) 28 | * @version 1.0, 28/03/2018 29 | */ 30 | @ KeepVp$ 31 | @ KeepMp$ 32 | object implicits { 33 | val P_HIGH = Reflow.P_HIGH 34 | val P_NORMAL = Reflow.P_NORMAL 35 | val P_LOW = Reflow.P_LOW 36 | 37 | val TRANSIENT = Period.TRANSIENT 38 | val SHORT = Period.SHORT 39 | val LONG = Period.LONG 40 | val INFINITE = Period.INFINITE 41 | 42 | lazy val SINGLE_THREAD = Config.SINGLE_THREAD 43 | 44 | lazy val Strategy = Feedback.Progress.Strategy 45 | lazy val FullDose = Strategy.FullDose 46 | lazy val Fluent = Strategy.Fluent 47 | lazy val Depth = Strategy.Depth 48 | lazy val Interval = Strategy.Interval 49 | 50 | type Intent = Trait 51 | val Intent = Trait 52 | 53 | def none[A]: immutable.Set[KvTpe[_ <: AnyRef]] = Helper.KvTpes.empty() 54 | def none: In = In.empty() 55 | 56 | implicit class TransformerRetain(kvt: KvTpe[_ <: AnyRef]) { 57 | @inline def re: Transformer[_ <: AnyRef, _ <: AnyRef] = Helper.Transformers.retain(kvt) 58 | } 59 | implicit def lite2Par[IN >: Null <: AnyRef: ClassTag, OUT >: Null <: AnyRef: ClassTag](lite: Lite[IN, OUT]): Par[IN, OUT] = Par(lite) 60 | implicit def serialInPar[IN >: Null <: AnyRef: ClassTag, OUT >: Null <: AnyRef: ClassTag](serial: Serial[IN, OUT]): Lite[IN, OUT] = serial.inPar() 61 | 62 | implicit class SerialInPar2Par[IN >: Null <: AnyRef: ClassTag, OUT >: Null <: AnyRef: ClassTag](serial: Serial[IN, OUT]) { 63 | def +>>[OUT1 >: Null <: AnyRef: ClassTag](lite: Lite[IN, OUT1]): Par2[IN, OUT, OUT1] = par(lite) 64 | def par[OUT1 >: Null <: AnyRef: ClassTag](lite: Lite[IN, OUT1]): Par2[IN, OUT, OUT1] = lite2Par(serialInPar(serial)) +>> lite 65 | } 66 | 67 | def +|-[IN >: Null <: AnyRef: ClassTag, Next >: Null <: AnyRef: ClassTag](f: IN => Next): Lite[IN, Next] = 68 | lite.Task[IN, Next](TRANSIENT, P_HIGH, visible = false)((in, _) => f(in)) 69 | 70 | // def 方法不能直接起作用，这里转换为函数值。 71 | implicit lazy val f0 = kvTpe2Bdr _ 72 | implicit lazy val f1 = trans2Bdr _ 73 | implicit lazy val f2 = tpeKv2Bdr _ 74 | implicit lazy val f3 = strKv2Bdr _ 75 | 76 | implicit def kvTpe2Bdr(kt: KvTpe[_ <: AnyRef]): Helper.KvTpes.Builder = Helper.KvTpes + kt 77 | implicit def kvTpe2Ok(kt: KvTpe[_ <: AnyRef]): immutable.Set[KvTpe[_ <: AnyRef]] = kt ok () 78 | implicit def kvtBdr2Ok(kb: Helper.KvTpes.Builder): immutable.Set[KvTpe[_ <: AnyRef]] = kb ok () 79 | implicit def trans2Bdr(trans: Transformer[_ <: AnyRef, _ <: AnyRef]): Helper.Transformers.Builder = Helper.Transformers + trans 80 | implicit def trans2Ok(trans: Transformer[_ <: AnyRef, _ <: AnyRef]): immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]] = trans ok () 81 | implicit def transBdr2Ok(tb: Helper.Transformers.Builder): immutable.Set[Transformer[_ <: AnyRef, _ <: AnyRef]] = tb ok () 82 | implicit def tpeKv2Bdr[V <: AnyRef](kv: (KvTpe[V], V)): In.Builder = In + kv 83 | implicit def tpeKv2Ok[V <: AnyRef](kv: (KvTpe[V], V)): In = kv ok () 84 | implicit def strKv2Bdr[V](kv: (String, V)): In.Builder = In + (kv._1, kv._2) 85 | implicit def strKv2Ok[V](kv: (String, V)): In = kv ok () 86 | implicit def inBdr2Ok(ib: In.Builder): In = ib ok () 87 | } 88 | -------------------------------------------------------------------------------- /src/main/scala/hobby/wei/c/tool/Snatcher.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package hobby.wei.c.tool 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.basis.TAG.LogTag 21 | import hobby.chenai.nakam.lang.J2S.NonNull 22 | import hobby.wei.c.log.Logger._ 23 | import hobby.wei.c.reflow.Reflow.{logger => log} 24 | import java.util.concurrent 25 | import java.util.concurrent.atomic.{AtomicBoolean, AtomicReference} 26 | import scala.annotation.tailrec 27 | import scala.util.control.Breaks._ 28 | 29 | /** 30 | * 本组件是一个无锁（lock-free）线程同步工具。用于处理如下场景： 31 | * 用于多个线程竞争去执行某个任务，但这个任务只需要任意一个线程执行即可（只要有人做就 ok，无论是谁），其它线程不必等待或阻塞。 32 | * 但同时也要避免遗漏（信号标志对任意线程要`可见`）：{{{ 33 | * 当执行任务的线程 a 认为做完了准备收工的时候，又来了新任务，但此时 a 还没切换标识，导致其它线程认为有人在做而略过，而 a 接下来又收工了的情况。 34 | * }}} 35 | * 用法示例：{{{ 36 | * val snatcher = new Snatcher() 37 | * if (snatcher.snatch()) { 38 | * breakable { 39 | * while (true) { 40 | * // do something ... 41 | * if (!snatcher.glance()) break 42 | * } 43 | * } 44 | * } 45 | * }}} 46 | * 或：{{{ 47 | * val snatcher = new Snatcher() 48 | * snatcher.tryOn { 49 | * // do something ... 50 | * } 51 | * }}} 52 | * 53 | * @author Wei Chou(weichou2010@gmail.com) 54 | * @version 1.0, 24/01/2018; 55 | * 2.0, 07/07/2018, 增加可重入（`reentrant`）能力； 56 | * 2.1, 08/04/2019, 重构； 57 | * 2.1, 30/12/2021, 重构，并加入`tryOns()`。 58 | */ 59 | class Snatcher extends TAG.ClassName { 60 | private val scheduling = new AtomicBoolean(false) 61 | private val signature = new AtomicBoolean(false) 62 | 63 | private lazy val thread = new AtomicReference[Thread] 64 | 65 | /** 66 | * 线程尝试抢占执行权并执行某任务。 67 | *

68 | * 2.0 版本新增了可重入（`reentrant`）能力，使得本方法可以嵌套使用。但这似乎是个伪命题：
69 | * 多重嵌套代码块（往往是不同且不可预知的），意味着有多个不同的任务需要抢占执行权，这是危险的：抢不到执行权的任务会被忽略不执行。 70 | * 所以仅用于有需求的殊设计的代码块。 71 | *

72 | * 对于 1.0 版本或当前`reentrant = false`时，嵌套的`tryOn()`无法执行。 73 | * 但请注意：本方法仍然仅限用于抢占执行同一个（其它抢占到的线程做的是同样的事情）任务，而不适用于不可预知的嵌套情况。如有 74 | * 多个不同的任务，请换用`ActionQueue.queAc()`让其排队执行。 75 | *

76 | * @param doWork 要执行的任务。本代码块[需要能够被]执行任意多次而不会对逻辑有未预期的影响，而且不应该携带参数，如有需要，可使用[[tryOns]]。 77 | * @param reentrant 是否需要可重入能力。`true`表示需要，`false`拒绝（默认值）。 78 | * 用于执行权已经被抢占而当前可能正处于该线程（当前调用嵌套于正在执行的另一个`doSomething`内）的情况。 79 | * @return `true`抢占成功并执行任务完成，`false`抢占失败，未执行任务。 80 | */ 81 | def tryOn(doWork: => Unit, reentrant: Boolean = false): Boolean = { 82 | if (reentrant && tryReentrant()) { 83 | doWork; true 84 | } else if (snatch()) { 85 | breakable { while (true) { doWork; if (!glance()) break } } 86 | true 87 | } else false 88 | } 89 | 90 | /** 线程尝试抢占执行权。 91 | * @return `true` 抢占成功，`false`抢占失败。 92 | */ 93 | def snatch(): Boolean = { 94 | signature.set(true) // 必须放在前面。标识新的调度请求，防止遗漏。 95 | tryLock() 96 | } 97 | 98 | /** 之前抢占成功（`snatch()`返回`true`）的线程，释放（重置）标识，并再次尝试抢占执行权。 99 | * @return `true` 抢占成功，`false`抢占失败。 100 | */ 101 | def glance(): Boolean = 102 | // 如果返回 false，说明`signature`本来就是 false。加这个判断的目的是减少不必要的原子操作（也是个不便宜的操作，好几个变量）。 103 | if (signature.compareAndSet(true, false)) { 104 | true.ensuring(scheduling.get, "调用顺序不对？请参照示例。") 105 | } else { 106 | thread.set(null) 107 | // 必须放在`signature`前面，确保不会有某个瞬间丢失调度（外部线程拿不到锁，而本线程认为没有任务了）。 108 | scheduling.set(false) 109 | // 再看看是不是又插入了新任务，并重新竞争锁定。 110 | // 如果不要`signature`的判断而简单再来一次是不是就解决了问题呢？ 111 | // 不能。这个再来一次的问题会递归。 112 | signature.get && tryLock() 113 | } 114 | 115 | private def tryLock() = 116 | if (scheduling.compareAndSet(false, true)) { 117 | signature.set(false) // 等竞争到了再置为false. 118 | thread.set(Thread.currentThread) 119 | true // continue 120 | } else false // break 121 | 122 | /** @return 当前是否可重入。 */ 123 | private def tryReentrant(): Boolean = { 124 | val t = thread.get 125 | if (t eq Thread.currentThread) true 126 | else if (t.isNull) false 127 | // else if (debugMode) throw new ReentrantLockError("非当前线程[`tryOn()`的嵌套情况]，不可启用`可重入`功能。") 128 | else false 129 | } 130 | } 131 | 132 | object Snatcher { 133 | 134 | def visWait[T](waitFor: => T)(cond: T => Boolean)(msg: Int => String)(implicit tag: LogTag): T = { 135 | @tailrec def cas(i: Int = 0): T = { 136 | val o = waitFor 137 | if (cond(o)) o 138 | else { 139 | val m = msg(i); if (m.nonEmpty) log.w("[visWait] WAIT !!! | %s | %s.", i, m.s) 140 | Thread.`yield`(); cas(i + 1) 141 | } 142 | } 143 | cas() 144 | } 145 | 146 | /** 147 | * 为避免多线程的阻塞，提高运行效率，可使用本组件将`action`队列化（非`顺序化`）。 148 | *

149 | * 如果没有`顺序化`需求，可略过后面的说明。但如果有，务必请注意：
150 | * 本组件并不能保证入队后的[`action`s]的顺序与入队前想要的一致，这不是本组件的缺陷，而是同步锁固有的性质导致了 151 | * 必然存在这样的问题：`任何同步锁的互斥范围都不可能超过其能够包裹的代码块的范围`。即使本组件的入队操作使用了`公平锁`，也 152 | * 无法保证外层的顺序需求。要实现顺序性，客户代码有两个选择：
153 | * 1. 外层代码必须根据具体的业务逻辑另行实现能够保证顺序的`互斥同步`逻辑，并在同步块内执行`queAc()`操作； 154 | * 2. 在`queAc()`的参数`action`所表示的函数体内实现[让输出具有`顺序性`]逻辑。 155 | *

156 | * 重申：本组件`能且只能`实现：`避免多线程阻塞，提高执行效率`。 157 | * 158 | * @param fluentMode 流畅模式。启用后，在拥挤（队列不空）的情况下，设置了`flag`的`action`将会被丢弃而不执行（除非是最后一个）。默认`不启用`。 159 | */ 160 | class ActionQueue(val fluentMode: Boolean = false) extends Snatcher { 161 | private val queue = new concurrent.LinkedTransferQueue[Action[_]] 162 | 163 | private case class Action[T](necessity: () => T, action: T => Unit, canAbandon: Boolean) { 164 | type A = T 165 | def execN(): A = necessity() 166 | def execA(args: A): Unit = action(args) 167 | } 168 | 169 | /** queueAction()的简写。 */ 170 | def queAc(action: => Unit): Unit = queAc()(action) { _ => } 171 | 172 | /** 173 | * 执行`action`或将其放进队列。 174 | * 175 | * @param canAbandon 是否可以被丢弃（需配合`fluentMode`使用）。默认为`false`。 176 | * @param necessity 必须要执行的，不可以`abandon`的。本函数的返回值将作为`action`的输入。 177 | * @param action 要执行的代码。 178 | */ 179 | def queAc[T](canAbandon: Boolean = false)(necessity: => T)(action: T => Unit): Unit = { 180 | def hasMore = !queue.isEmpty 181 | def newAc = Action(() => necessity, action, canAbandon) 182 | def quelem(): Action[_] = { 183 | val elem = newAc 184 | while (!(queue offer elem)) Thread.`yield`() 185 | elem 186 | } 187 | def execAc(elem: Action[_]): Action[_] = { 188 | val p: elem.A = elem.execN() 189 | if (fluentMode && elem.canAbandon) { // 设置了`abandon`标识 190 | if (hasMore) { // 可以抛弃 191 | } else elem.execA(p) 192 | } else elem.execA(p) 193 | elem 194 | } 195 | 196 | quelem() // format: off 197 | tryOn({ 198 | // 第一次也要检查，虽然前面入队了。因为很可能在当前线程抢占到的时候，自己入队的已经被前一个线程消化掉而退出了。 199 | while (hasMore) { 200 | execAc(queue.remove()) 201 | } 202 | }, false /*必须为`false`，否则调用会嵌套，方法栈会加深。*/) 203 | } 204 | } 205 | } 206 | -------------------------------------------------------------------------------- /src/test/java/StepTest.java: -------------------------------------------------------------------------------- 1 | import java.util.Collections; 2 | import java.util.HashMap; 3 | import java.util.HashSet; 4 | import java.util.Map; 5 | import java.util.Set; 6 | 7 | import hobby.wei.c.reflow.step.Step; 8 | import hobby.wei.c.reflow.step.Tracker; 9 | import hobby.wei.c.reflow.step.Unit; 10 | import hobby.wei.c.reflow.step.UnitM; 11 | 12 | /** 13 | * @author Wei Chou(weichou2010@gmail.com) 14 | * @version 1.0, 06/10/2016 15 | */ 16 | public class StepTest { 17 | public static void main(String[] args) throws Exception { 18 | System.out.println("----->"); 19 | final Step step = new UnitM() { 20 | @Override 21 | protected Map exec(Map map) throws Exception { 22 | final Map output = new HashMap<>(); 23 | output.put("a", 0); 24 | System.out.println("[exec]:" + tracker().progress() + "/" + tracker().count() + ", " + output); 25 | return output; 26 | } 27 | 28 | @Override 29 | protected Map def() throws Exception { 30 | System.out.println("[def]:" + tracker().progress() + "/" + tracker().count()); 31 | return Collections.emptyMap(); 32 | } 33 | }.then(new UnitM() { 34 | @Override 35 | protected Map exec(Map map) throws Exception { 36 | final Map output = new HashMap<>(); 37 | output.put("b", "1"); 38 | output.put("c", 289); 39 | System.out.println("[exec]:" + tracker().progress() + "/" + tracker().count() + ", " + output.toString()); 40 | return output; 41 | } 42 | 43 | @Override 44 | protected Map def() throws Exception { 45 | System.out.println("[def]:" + tracker().progress() + "/" + tracker().count() + ", " + getClass().getName()); 46 | return Collections.emptyMap(); 47 | } 48 | }).then(new Unit, String, Tracker, String>() { 49 | @Override 50 | protected String exec(Map map) throws Exception { 51 | System.out.println("[exec]:" + tracker().progress() + "/" + tracker().count() + ", " + map); 52 | return map.toString(); 53 | } 54 | 55 | @Override 56 | protected String def() throws Exception { 57 | return null; 58 | } 59 | 60 | @Override 61 | protected Set requireKeys() { 62 | final Set set = new HashSet<>(); 63 | set.add("a"); 64 | // set.add("b"); 65 | set.add("c"); 66 | return set; 67 | } 68 | 69 | @Override 70 | protected Tracker newTracker() { 71 | return new Tracker<>(); 72 | } 73 | }).then(new Unit, String>() { 74 | @Override 75 | protected String exec(String s) throws Exception { 76 | System.out.println("[exec]:" + tracker().progress() + "/" + tracker().count() + ", " + getClass().getName()); 77 | return s; 78 | } 79 | 80 | @Override 81 | protected String def() throws Exception { 82 | return null; 83 | } 84 | 85 | @Override 86 | protected Tracker newTracker() { 87 | return new Tracker<>(); 88 | } 89 | }); 90 | System.out.println("---->result:" + step.exec(new Step.Callback() { 91 | @Override 92 | public void onProgress(int progress, int count) { 93 | System.out.println("----->[onProgress]:" + progress + "/" + count); 94 | } 95 | })); 96 | 97 | 98 | if (true) return; 99 | 100 | Map map = new HashMap(); 101 | map.put("a", "abce"); 102 | System.out.println(map.get("a")); 103 | map.put("b", new Object()); 104 | System.out.println(map.get("b")); 105 | map.put("a", new Object()); 106 | System.out.println(map.get("a")); 107 | 108 | Object[] arr = new Object[10]; 109 | arr[0] = 1; 110 | System.out.println(arr[0]); 111 | arr[0] = new Object(); 112 | System.out.println(arr[0]); 113 | } 114 | } 115 | -------------------------------------------------------------------------------- /src/test/java/UnitTest.java: -------------------------------------------------------------------------------- 1 | /* 2 | * This Java source file was generated by the Gradle 'init' task. 3 | */ 4 | import org.junit.Test; 5 | // import static org.junit.Assert.*; 6 | 7 | public class UnitTest { 8 | @Test public void testAppHasAGreeting() { 9 | // App classUnderTest = new App(); 10 | // assertNotNull("app should have a greeting", classUnderTest.getGreeting()); 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /src/test/scala/VolatileTest.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | import java.util.concurrent.{Callable, Executors} 18 | import java.util.concurrent.atomic.AtomicBoolean 19 | 20 | /** 21 | * 一个用于证明`AtomicXxx.get`在多线程中可靠性的验证方法。 22 | * 23 | * @author Wei Chou(weichou2010@gmail.com) 24 | * @version 1.0, 03/02/2018 25 | */ 26 | object VolatileTest extends App { 27 | val executor = Executors.newFixedThreadPool(2) 28 | 29 | var count = 0 30 | 31 | while (true) { 32 | val a = new AtomicBoolean(false) 33 | val b = new AtomicBoolean(false) 34 | 35 | def test(): Boolean = a.get() && b.get() 36 | 37 | val callA = new Callable[Boolean] { 38 | override def call() = { 39 | Thread.sleep(0, (math.random * 10).toInt) 40 | a.set(true) 41 | test() 42 | } 43 | } 44 | val callB = new Callable[Boolean] { 45 | override def call() = { 46 | Thread.sleep(0, (math.random * 5).toInt) 47 | b.set(true) 48 | test() 49 | } 50 | } 51 | val fa = executor.submit(callA) 52 | val fb = executor.submit(callB) 53 | 54 | if (fa.get() || fb.get()) { 55 | // 符合预期。 56 | } else throw new Exception("AtomicXxx.get 不可靠。") 57 | 58 | count += 1 59 | println(s"done. count:$count, fa:${fa.get}, fb:${fb.get}.") 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /src/test/scala/reflow/test/LiteSpec.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2020-present, Chenai Nakam(chenai.nakam@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package reflow.test 18 | 19 | import hobby.chenai.nakam.basis.TAG 20 | import hobby.chenai.nakam.lang.J2S.future2Scala 21 | import hobby.wei.c.reflow.{GlobalTrack, Poster, Reflow, State} 22 | import hobby.wei.c.reflow 23 | import hobby.wei.c.reflow.implicits._ 24 | import hobby.wei.c.reflow.lite._ 25 | import hobby.wei.c.reflow.Reflow.GlobalTrack.GlobalTrackObserver 26 | import hobby.wei.c.reflow.Trait.ReflowTrait 27 | import org.scalatest.{AsyncFeatureSpec, BeforeAndAfter, BeforeAndAfterAll, GivenWhenThen} 28 | import java.util.concurrent.{Callable, FutureTask} 29 | 30 | import scala.annotation.tailrec 31 | import scala.concurrent.duration.DurationInt 32 | import scala.util.control.Breaks 33 | 34 | /** 35 | * @author Chenai Nakam(chenai.nakam@gmail.com) 36 | * @version 1.0, 04/07/2020 37 | */ 38 | class LiteSpec extends AsyncFeatureSpec with GivenWhenThen with BeforeAndAfter with BeforeAndAfterAll { 39 | 40 | override protected def beforeAll(): Unit = { 41 | Reflow.setDebugMode(false) 42 | // Reflow.setConfig(Config(5, 7)) 43 | // Reflow.setConfig(SINGLE_THREAD) 44 | 45 | Reflow.GlobalTrack.registerObserver(new GlobalTrackObserver { 46 | override def onUpdate(current: GlobalTrack, items: All): Unit = { 47 | if (!current.isSubReflow && current.scheduler.getState == State.EXECUTING) { 48 | // println(s"++++++++++++++++++++[[[current.state:${current.scheduler.getState}") 49 | // items().foreach(println) 50 | // println(current) 51 | // println("--------------------]]]") 52 | } 53 | } 54 | })(null) 55 | } 56 | 57 | trait AbsTag extends TAG.ClassName { 58 | // Reflow.logger.i(toString)(implicitly) 59 | override def toString = className.toString 60 | } 61 | class Aaa extends AbsTag 62 | 63 | class Bbb(val aaa: Aaa) extends AbsTag { 64 | override def toString = s"${super.toString}(${aaa.toString})" 65 | } 66 | 67 | class Ccc(val bbb: Bbb) extends AbsTag { 68 | override def toString = s"${super.toString}(${bbb.toString})" 69 | } 70 | 71 | class Ddd(val ccc: Ccc) extends AbsTag { 72 | override def toString = s"${super.toString}(${ccc.toString})" 73 | } 74 | 75 | implicit lazy val a2b = Task[Aaa, Bbb]() { (aaa, ctx) => new Bbb(aaa) } 76 | implicit lazy val b2c = Task[Bbb, Ccc]() { (bbb, ctx) => new Ccc(bbb) } 77 | implicit lazy val c2a = Task[Ccc, Aaa]() { (ccc, ctx) => ccc.bbb.aaa } 78 | implicit lazy val c2b = Task[Ccc, Bbb]() { (ccc, ctx) => ccc.bbb } 79 | implicit lazy val c2d = Task[Ccc, Ddd]() { (ccc, ctx) => new Ddd(ccc) } 80 | implicit lazy val d2b = Task[Ddd, Bbb]() { (ddd, ctx) => ddd.ccc.bbb } 81 | implicit lazy val b2a = Task[Bbb, Aaa]() { (bbb, ctx) => bbb.aaa } 82 | implicit lazy val a2d = Task[Aaa, Ddd]() { (aaa, ctx) => new Ddd(new Ccc(new Bbb(aaa))) } 83 | implicit lazy val c2abc = c2a >>> a2b >>> b2c 84 | 85 | implicit lazy val strategy = FullDose 86 | implicit lazy val poster: Poster = null 87 | 88 | lazy val feedback = new reflow.Feedback.Adapter 89 | 90 | Feature("使用 reflow.lite 库简化 Reflow 编码") { 91 | Scenario("简单`【串】行任务`组装") { 92 | info("以上定义了一些任务") 93 | info("再定义一个输入：") 94 | val input = Task[Aaa] 95 | 96 | Then("组装任务：") 97 | info("1. 利用`类型匹配 + 隐世转换`自动组装；") 98 | 99 | input.next[Bbb].next[Ccc].next[Ddd].run(new Aaa) sync () 100 | 101 | info("2. 直接用任务的`引用`组装；") 102 | input >>> a2b >>> b2c >>> c2d run (new Aaa) sync () 103 | 104 | info("这两种方法是等价的，后面跟`run()`即可运行。") 105 | 106 | When("调用`run(feedback)(strategy，poster)`运行") 107 | info("观察输出") 108 | assert(true) 109 | } 110 | 111 | Scenario("`【串/并】行任务`混合组装") { 112 | val pars = 113 | ( 114 | c2d 115 | +>> 116 | c2abc.inPar("name#c2abc", "c2abc`串行`混入`并行`") 117 | +>> 118 | (c2b >>> b2c >>> c2a >>> a2b).inPar() 119 | +>> 120 | c2a 121 | ) **> { (d, c, b, a, ctx) => 122 | info(a.toString) 123 | info(b.toString) 124 | info(c.toString) 125 | info(d.toString) 126 | d 127 | } 128 | Input[Aaa] >>> a2b >>> b2c >>> pars run (new Aaa) sync () 129 | 130 | assert(true) 131 | } 132 | 133 | Scenario("带有输入类型的 `Pulse` 流处理器") { 134 | val pars = 135 | ( 136 | // c2d 137 | // +>> 138 | (c2abc >>> Task[Ccc, Ddd]() { (ccc, ctx) => 139 | if (ctx.input[String]("flag").isEmpty) { 140 | ctx.cache("flag", "") 141 | throw new IllegalArgumentException 142 | } else { 143 | ctx.cache("flag", "") 144 | new Ddd(ccc) 145 | } 146 | // 一个串行的里面不能有重名的。 147 | } >>> d2b >>> b2a >>> a2d) //.inPar("name#c2abc", "c2abc`串行`混入`并行`") 148 | +>> 149 | (c2b >>> b2c >>> Task[Ccc, Aaa]() { (ccc, ctx) => 150 | val n: Integer = ctx.input[Integer]("int").getOrElse(1) 151 | if (n == 2) { 152 | throw new IllegalStateException 153 | } else { 154 | ctx.cache("int", n + 1) 155 | ccc.bbb.aaa 156 | } 157 | } >>> a2b) 158 | +>> 159 | c2b 160 | +>> 161 | c2a 162 | ) **> { (d, c, b, a, ctx) => 163 | // info(a.toString) 164 | // info(b.toString) 165 | // info(c.toString) 166 | // info(d.toString) 167 | d 168 | } 169 | 170 | @volatile var callableOut: Int = 0 171 | val future = new FutureTask[Int](() => callableOut) 172 | 173 | val repeatCount = 1000 // Int.MaxValue 174 | 175 | val pulse = (Input[Aaa] >>> a2b >>> b2c >>> pars) pulse (new reflow.Pulse.Feedback.Lite[Ddd] { 176 | override def onStart(serialNum: Long): Unit = println( 177 | s"[onStart] |||||||||||||||||||||||||||||||||||||||||||| $serialNum ||||||||||||||||||||||||||||||||||||||||||||" 178 | ) 179 | override def onAbort(serialNum: Long, trigger: Option[Intent], parent: Option[ReflowTrait], depth: Int): Unit = { 180 | callableOut = repeatCount 181 | future.run() 182 | println(s"[onAbort] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! $serialNum, ${trigger.map(_.name$).orNull} !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") 183 | } 184 | override def onFailed(serialNum: Long, trat: Intent, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = { 185 | if (serialNum == repeatCount) { 186 | callableOut = repeatCount 187 | future.run() 188 | } 189 | println(s"[onFailed] ?????????????????????????????????????????? $serialNum, ${trat.name$}, ${e.getMessage} ??????????????????????????????????????????") 190 | } 191 | 192 | override def liteOnComplete(serialNum: Long, value: Option[Ddd]): Unit = { 193 | if (serialNum == repeatCount) { 194 | callableOut = repeatCount 195 | future.run() 196 | } 197 | println(s"[liteOnComplete] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $serialNum, $value ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~") 198 | } 199 | }, abortIfError = false, execCapacity = 7) 200 | 201 | Breaks.breakable { 202 | for (_ <- 0 to repeatCount) { 203 | val in = new Aaa 204 | var i = 0 205 | while (!pulse.input(in)) { 206 | if (pulse.pulse.isDone) Breaks.break 207 | println(s"[pulse.input] @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ wait repeat times: $i @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@") 208 | i += 1 209 | Thread.sleep(250 * i.millis.toMillis) 210 | } 211 | } 212 | } 213 | 214 | future map { result => 215 | require(pulse.pulse.isCurrAllCompleted) 216 | 217 | assertResult(repeatCount)(result) 218 | } 219 | } 220 | 221 | Scenario("`Pulse` 的多层嵌套组装测试") { 222 | val pars = { 223 | // 切记：不能用 val 替换 def，否则不但有并行的同名的任务，而且深层的【同名同层】会导致 Pulse.Interact 接口无法辨识而出现阻塞的情况。 224 | def p = (c2d +>> c2b) **> { (d, _, ctx) => println(":: 0"); ctx.progress(1, 2); d } +|- { (_, d) => d } 225 | @tailrec 226 | def loop(s: () => Serial[Ccc, Ddd], times: Int = 0, limit: Int = 10): Serial[Ccc, Ddd] = 227 | if (times >= limit) s() 228 | else { 229 | def p = 230 | ( 231 | s() +|- { (_, d) => d } 232 | +>> 233 | s() 234 | +>> 235 | s() 236 | ) **> { (d, _, _, ctx) => println(s":: ${times + 1} " + ("::" * (times + 1))); ctx.progress(1, 2); d } 237 | loop(() => p, times + 1, limit) 238 | } 239 | loop(() => p, limit = 3) // 最大 depth = 2 * limit + 1, depth 从 0 开始。 240 | } 241 | 242 | @volatile var callableOut: Int = 0 243 | val future = new FutureTask[Int](() => callableOut) 244 | 245 | val repeatCount = 1000 // Int.MaxValue 246 | 247 | val pulse = (Input[Aaa] >>> a2b >>> b2c >>> pars) pulse (new reflow.Pulse.Feedback.Lite[Ddd] { 248 | override def onStart(serialNum: Long): Unit = {} 249 | override def onAbort(serialNum: Long, trigger: Option[Intent], parent: Option[ReflowTrait], depth: Int): Unit = { 250 | callableOut = repeatCount 251 | future.run() 252 | println(s"[onAbort] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! $serialNum, ${trigger.map(_.name$).orNull} !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") 253 | } 254 | override def onFailed(serialNum: Long, trat: Intent, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = { 255 | if (serialNum == repeatCount) { 256 | callableOut = repeatCount 257 | future.run() 258 | } 259 | future.run() 260 | println(s"[onFailed] ?????????????????????????????????????????? $serialNum, ${trat.name$}, ${e.getMessage} ??????????????????????????????????????????") 261 | } 262 | 263 | override def liteOnComplete(serialNum: Long, value: Option[Ddd]): Unit = { 264 | if (serialNum == repeatCount) { 265 | callableOut = repeatCount 266 | future.run() 267 | } 268 | if (serialNum % 700 == 0) 269 | println(s"[liteOnComplete] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $serialNum, $value ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~") 270 | if (serialNum == repeatCount) 271 | println(s"[liteOnComplete] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ DONE | $serialNum, $value ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~") 272 | } 273 | }, abortIfError = true, execCapacity = 7, globalTrack = false) 274 | 275 | Breaks.breakable { 276 | for (_ <- 0 to repeatCount) { 277 | val in = new Aaa 278 | var i = 0 279 | while (!pulse.input(in)) { 280 | if (pulse.pulse.isDone) Breaks.break 281 | println(s"[pulse.input] @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ wait repeat times: $i @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@") 282 | i += 1 283 | Thread.sleep(250 * i.millis.toMillis) 284 | } 285 | } 286 | } 287 | 288 | future map { result => 289 | require(pulse.pulse.isCurrAllCompleted) 290 | 291 | assertResult(repeatCount)(result) 292 | } 293 | } 294 | } 295 | } 296 | -------------------------------------------------------------------------------- /src/test/scala/reflow/test/PulseSpec.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Chenai Nakam(chenai.nakam@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package reflow.test 18 | 19 | import hobby.chenai.nakam.lang.J2S._ 20 | import hobby.chenai.nakam.tool.pool.S._2S 21 | import hobby.wei.c.reflow._ 22 | import hobby.wei.c.reflow.implicits._ 23 | import hobby.wei.c.reflow.Feedback.Progress.Strategy 24 | import hobby.wei.c.reflow.Trait.ReflowTrait 25 | import org.scalatest.{AsyncFeatureSpec, BeforeAndAfter, BeforeAndAfterAll, GivenWhenThen} 26 | import java.util.concurrent.{Callable, FutureTask} 27 | 28 | /** 29 | * @author Chenai Nakam(chenai.nakam@gmail.com) 30 | * @version 1.0, 07/07/2018; 31 | * 1.5, 04/10/2019, fix 了一个很重要的 bug。 32 | */ 33 | class PulseSpec extends AsyncFeatureSpec with GivenWhenThen with BeforeAndAfter with BeforeAndAfterAll { 34 | 35 | override protected def beforeAll(): Unit = { 36 | Reflow.setDebugMode(false) 37 | // Reflow.setConfig(SINGLE_THREAD) 38 | } 39 | 40 | implicit lazy val strategy: Strategy = Strategy.Depth(3) -> Strategy.Fluent -> Strategy.Interval(600) 41 | implicit lazy val poster: Poster = null 42 | 43 | Feature("`Pulse`脉冲步进流式数据处理") { 44 | Scenario("数据将流经`集成任务集（Reflow）`，并始终保持输入时的先后顺序，多组数据会排队进入同一个任务。") { 45 | Given("创建一个`reflowX`作为嵌套的`SubReflow`") 46 | val reflowX0 = Reflow.create(Trait("pulse-0", SHORT, kvTpes.str) { ctx => 47 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 48 | println(s"再次进入任务${ctx.trat.name$}，缓存参数被累加：${times}。") 49 | if (times == 1) { 50 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 51 | Thread.sleep(200) 52 | } 53 | ctx.cache[Integer](kvTpes.int, times + 1) 54 | ctx.output(kvTpes.str, s"name:${ctx.trat.name$}, 第${times}次。") 55 | }) 56 | .next(Trait("pulse-1", SHORT, kvTpes.str, kvTpes.str) { ctx => 57 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 58 | if (times % 2 == 0) { 59 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 60 | Thread.sleep(500) 61 | } 62 | ctx.cache[Integer](kvTpes.int, times + 1) 63 | ctx.output(kvTpes.str, times + "") 64 | }) 65 | .submit(none) 66 | 67 | val reflowX1 = Reflow.create(Trait("pulse-2", SHORT, kvTpes.str) { ctx => 68 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 69 | println(s"再次进入任务${ctx.trat.name$}，缓存参数被累加：${times}。") 70 | if (times == 1) { 71 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 72 | Thread.sleep(500) 73 | } 74 | ctx.cache[Integer](kvTpes.int, times + 1) 75 | ctx.output(kvTpes.str, s"name:${ctx.trat.name$}, 第${times}次。") 76 | }).and(reflowX0.toSub("pulseX0")) 77 | .next(Trait("pulse-3", SHORT, kvTpes.str, kvTpes.str) { ctx => 78 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 79 | if (times % 2 == 0) { 80 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 81 | Thread.sleep(300) 82 | } 83 | ctx.cache[Integer](kvTpes.int, times + 1) 84 | ctx.output(kvTpes.str, times + "") 85 | }) 86 | .submit(none) 87 | 88 | Given("创建一个顶层`reflow`") 89 | val reflow = Reflow.create(Trait("pulse-4", SHORT, kvTpes.str) { ctx => 90 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 91 | println(s"再次进入任务${ctx.trat.name$}，缓存参数被累加：${times}。") 92 | if (times == 1) { 93 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 94 | Thread.sleep(100) 95 | } 96 | ctx.cache[Integer](kvTpes.int, times + 1) 97 | ctx.output(kvTpes.str, s"name:${ctx.trat.name$}, 第${times}次。") 98 | }).and(reflowX1.toSub("pulseX1")) 99 | .next(Trait("pulse-5", SHORT, kvTpes.str, kvTpes.str) { ctx => 100 | val times: Int = ctx.input(kvTpes.int).getOrElse[Integer](0) 101 | if (times % 2 == 0) { 102 | println(s"------------------->(times:$times, ${ctx.trat.name$})休眠中，后续进入的数据会等待...") 103 | Thread.sleep(200) 104 | } 105 | ctx.cache[Integer](kvTpes.int, times + 1) 106 | ctx.output(kvTpes.str, times + "") 107 | }) 108 | .submit(kvTpes.str) 109 | 110 | @volatile var callableOut: Out = null 111 | val future = new FutureTask[Out](new Callable[Out] { 112 | override def call() = callableOut 113 | }) 114 | 115 | Then("创建 pulse") 116 | lazy val pulse: Pulse = reflow.pulse(feedbackPulse, true) 117 | 118 | lazy val feedbackPulse = new Pulse.Feedback.Adapter { 119 | override def onComplete(serialNum: Long, out: Out): Unit = { 120 | if (serialNum == 20) { 121 | callableOut = out 122 | future.run() 123 | println("abort()...") 124 | pulse.abort() 125 | } 126 | } 127 | 128 | override def onAbort(serialNum: Long, trigger: Option[Trait], parent: Option[ReflowTrait], depth: Int): Unit = { 129 | println("[onAbort]trigger:" + trigger.map(_.name$).orNull) 130 | } 131 | 132 | override def onFailed(serialNum: Long, trat: Trait, parent: Option[ReflowTrait], depth: Int, e: Exception): Unit = { 133 | println("[onFailed]trat:" + trat.name$.s + ", e:" + (e.getClass.getName + ":" + e.getMessage)) 134 | } 135 | } 136 | 137 | // lazy val feedback = new Feedback.Adapter { 138 | // override def onComplete(out: Out): Unit = { 139 | // callableOut = out 140 | // future.run() 141 | // } 142 | // } 143 | 144 | var data = (kvTpes.str, "66666") :: Nil 145 | data ::= data.head 146 | data ::= data.head 147 | data ::= data.head 148 | data ::= data.head 149 | data ::= data.head 150 | data ::= data.head 151 | data ::= data.head 152 | data ::= data.head 153 | data ::= data.head 154 | data ::= data.head 155 | data ::= data.head 156 | data ::= data.head 157 | data ::= data.head 158 | data ::= data.head 159 | data ::= data.head 160 | data ::= data.head 161 | data ::= data.head 162 | data ::= data.head 163 | data ::= data.head 164 | data ::= data.head 165 | Then("创建数据：" + data) 166 | 167 | When("向 pulse 输入数据") 168 | data.foreach(pulse.input(_)) 169 | 170 | Then("等待结果") 171 | future map { out => 172 | require(pulse.isCurrAllCompleted) 173 | 174 | assertResult("20")(out(kvTpes.str)) 175 | } 176 | } 177 | } 178 | 179 | override protected def afterAll(): Unit = { 180 | info("All test done!!!~") 181 | Reflow.shutdown() 182 | } 183 | 184 | before { 185 | info("++++++++++>>>") 186 | } 187 | 188 | after { 189 | info("<<<----------") 190 | } 191 | } 192 | -------------------------------------------------------------------------------- /src/test/scala/reflow/test/SnatcherSpec.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Chenai Nakam(chenai.nakam@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package reflow.test 18 | 19 | import hobby.chenai.nakam.lang.J2S.{future2Scala, Obiter, Run} 20 | import hobby.wei.c.reflow.Reflow 21 | import hobby.wei.c.tool 22 | //import hobby.wei.c.tool.Snatcher.ReentrantLockError 23 | import org.scalatest._ 24 | import java.util 25 | import java.util.concurrent.{LinkedBlockingQueue => _, _} 26 | 27 | /** 28 | * @author Chenai Nakam(chenai.nakam@gmail.com) 29 | * @version 1.0, 13/03/2018 30 | */ 31 | class SnatcherSpec extends AsyncFeatureSpec with GivenWhenThen { 32 | Reflow.setDebugMode(true) 33 | 34 | Feature("Snatcher 测试") { 35 | /*Scenario("并发不同步测试") { // 并不会发生 36 | @volatile var count = -1L 37 | val queue = new ConcurrentLinkedQueue[Option[Any]] 38 | while (true) { 39 | println(s"----------------------count:${count += 1; count}-------------------------------------") 40 | while (!queue.offer(None)) Thread.`yield`() 41 | try { 42 | queue.remove() 43 | } catch { 44 | case t: Throwable => t.printStackTrace() 45 | throw t 46 | } 47 | } 48 | assert(true) 49 | }*/ 50 | 51 | Scenario("Reentrant 抛出异常测试") { 52 | val snatcher = new tool.Snatcher 53 | new Thread(snatcher.tryOn({ 54 | info("第 1 层 tryOn(...)") 55 | snatcher.tryOn({ 56 | info("第 2 层 tryOn(...)") 57 | snatcher.tryOn({ 58 | info("第 3 层 tryOn(...)") 59 | info("do something") 60 | Thread.sleep(3000) 61 | info("第 3 层 tryOn(...), Done.") 62 | }, true) 63 | info("第 2 层 tryOn(...), Done.") 64 | }, true) 65 | info("第 1 层 tryOn(...), Done.") 66 | }, true).run$).start() 67 | 68 | Thread.sleep(10000) 69 | // assertThrows[ReentrantLockError] { 70 | // snatcher.tryOn({ 71 | // info("第 1 层") 72 | // snatcher.tryOn({ 73 | // info("第 2 层") 74 | // info("第 2 层, Done.") 75 | // }, true) 76 | // info("第 1 层, Done.") 77 | // }, true) 78 | // } 79 | assert(true) 80 | } 81 | 82 | Scenario("传名参数") { 83 | val snatcher = new tool.Snatcher.ActionQueue(false) 84 | val future = new FutureTask[Int](new Callable[Int] { 85 | override def call() = 0 86 | }) 87 | sThreadPoolExecutor.execute({ 88 | snatcher.queAc { 89 | println("抢占 | Holding...") 90 | Thread.sleep(5000) 91 | println("抢占 | Done.") 92 | } 93 | }.run$) 94 | Thread.sleep(2000) 95 | snatcher.queAc { 96 | println("在 snatcher 调度器内部执行") 97 | } 98 | snatcher.queAc { 99 | println("在 snatcher 调度器内部执行") 100 | println("在 queueAction Done 之后输出，即为正确。") 101 | future.run() 102 | } 103 | println("queueAction Done.") 104 | 105 | future.map(_ => assert(true)) 106 | } 107 | 108 | Scenario("测试 Snatcher.ActionQueue 并发问题") { 109 | if (false) { 110 | @volatile var count = -1L 111 | @volatile var stop = false 112 | 113 | val snatcher = new tool.Snatcher.ActionQueue(true) 114 | val future = new FutureTask[Long](new Callable[Long] { 115 | override def call() = count 116 | }) 117 | (0 until 3).foreach { i => 118 | sThreadPoolExecutor.submit(new Runnable { 119 | override def run(): Unit = while (!stop) { 120 | println(s"------------ $i ----------------submit snatcher queueAction, active:${sThreadPoolExecutor.getActiveCount}, queue:${sThreadPoolExecutor.getQueue.size}") 121 | sThreadPoolExecutor.submit { 122 | { 123 | try { 124 | snatcher.queAc(canAbandon = (Math.random() >= 0.6).obiter { 125 | println("-----------------------------------------------------------") 126 | }) {} { _ => 127 | count += 1 128 | println(s"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~thread:${Thread.currentThread.getId}~~~~~~~~~~~~~~~~~~~~~~~~~Snatcher第${count}次计算~~~~~") 129 | } 130 | } catch { 131 | case t: Throwable => t.printStackTrace() 132 | stop = true 133 | println(s"Throwable<<<<<>>>>>第${count}次计算") 134 | } 135 | }.run$ 136 | } 137 | } 138 | }) 139 | } 140 | future.get() 141 | } 142 | assert(true) 143 | } 144 | } 145 | 146 | private lazy val sPoolWorkQueue: BlockingQueue[Runnable] = new util.concurrent.LinkedBlockingQueue[Runnable](2048) { 147 | override def offer(r: Runnable) = { 148 | /* 如果不放入队列并返回false，会迫使增加线程。但是这样又会导致总是增加线程，而空闲线程得不到重用。 149 | 因此在有空闲线程的情况下就直接放入队列。若大量长任务致使线程数增加到上限， 150 | 则threadPool启动reject流程(见ThreadPoolExecutor构造器的最后一个参数)，此时再插入到本队列。 151 | 这样即完美实现[先增加线程数到最大，再入队列，空闲释放线程]这个基本逻辑。*/ 152 | val b = sThreadPoolExecutor.getActiveCount < sThreadPoolExecutor.getPoolSize && super.offer(r) 153 | b 154 | } 155 | } 156 | 157 | lazy val sThreadPoolExecutor: ThreadPoolExecutor = new ThreadPoolExecutor(8, 24, 158 | 10, TimeUnit.SECONDS, sPoolWorkQueue, new ThreadFactory { 159 | override def newThread(r: Runnable) = new Thread(r) 160 | }, new RejectedExecutionHandler { 161 | override def rejectedExecution(r: Runnable, executor: ThreadPoolExecutor): Unit = { 162 | } 163 | }) 164 | } 165 | -------------------------------------------------------------------------------- /src/test/scala/reflow/test/tasks.scala: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018-present, Wei Chou(weichou2010@gmail.com) 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package reflow.test 18 | 19 | import hobby.wei.c.reflow.{KvTpe, _} 20 | import hobby.wei.c.reflow.implicits._ 21 | import reflow.test.enum._ 22 | 23 | /** 24 | * @author Wei Chou(weichou2010@gmail.com) 25 | * @version 1.0, 23/03/2018 26 | */ 27 | object kvTpes { 28 | lazy val anyRef = new KvTpe[AnyRef]("anyr") {} 29 | lazy val int = new KvTpe[Integer]("int") {} 30 | lazy val str = new KvTpe[String]("str") {} 31 | lazy val outputstr = new KvTpe[String]("outputstr") {} 32 | lazy val seq = new KvTpe[Seq[_]]("seq") {} 33 | lazy val enum = new KvTpe[EnumTest.Tpe]("enum") {} 34 | } 35 | 36 | object enum { 37 | object EnumTest extends Enumeration { 38 | type Tpe = Value 39 | val A, B = Value 40 | } 41 | } 42 | 43 | object trans { 44 | lazy val int2str = new Transformer[Integer, String]("int", "str") { 45 | override def transform(in: Option[Integer]) = in.map(String.valueOf) 46 | } 47 | lazy val str2int = new Transformer[String, Integer]("str", "int") { 48 | override def transform(in: Option[String]) = in.map(Integer.valueOf) 49 | } 50 | } 51 | 52 | object trats { 53 | import kvTpes._ 54 | 55 | lazy val int2str0 = new Trait.Adapter { 56 | override protected def period() = TRANSIENT 57 | 58 | override protected def requires() = int 59 | 60 | override protected def outs() = str 61 | 62 | override protected def name() = "int2str0" 63 | 64 | override def newTask() = new Task() { 65 | override protected def doWork(): Unit = { 66 | // requireReinforce() 67 | // cache(str, "987654321") 68 | if (isReinforcing) { 69 | Thread.sleep(1000) 70 | output(str, input(str).orNull) 71 | } else output(str.key, String.valueOf(input(int.key).getOrElse(-1))) 72 | } 73 | } 74 | } 75 | lazy val str2int = new Trait.Adapter { 76 | override protected def period() = TRANSIENT 77 | 78 | override protected def requires() = str 79 | 80 | override protected def outs() = int 81 | 82 | override protected def name() = "str2int" 83 | 84 | override def newTask() = new Task() { 85 | override protected def doWork(): Unit = { 86 | requireReinforce() 87 | cache[Integer](int, 987654321) 88 | if (isReinforcing) { 89 | output(int, input(int).orNull) 90 | } else output(int, Integer.valueOf(input(str).getOrElse("-1"))) 91 | } 92 | } 93 | } 94 | 95 | lazy val int2str1 = new Trait.Adapter { 96 | override protected def period() = TRANSIENT 97 | 98 | override protected def requires() = int 99 | 100 | override protected def outs() = str 101 | 102 | override protected def name() = "int2str1" 103 | 104 | override def newTask() = new Task() { 105 | override protected def doWork(): Unit = { 106 | // failed(new Exception("987654321")) 107 | requireReinforce() 108 | cache(str, "987654321") 109 | if (isReinforcing) { 110 | Thread.sleep(1000) 111 | output(str, input(str).orNull) 112 | } else output(str.key, String.valueOf(input(int.key).getOrElse(-1))) 113 | } 114 | } 115 | } 116 | } 117 | 118 | object tasks { 119 | } 120 | -------------------------------------------------------------------------------- /src/test/scala/sample/FunSuitee.scala: -------------------------------------------------------------------------------- 1 | package sample 2 | 3 | import org.scalatest.{BeforeAndAfter, FunSuite} 4 | 5 | class FunSuitee extends FunSuite with BeforeAndAfter { 6 | // http://doc.scalatest.org/1.8/org/scalatest/BeforeAndAfter.html 7 | before { 8 | // ... 9 | } 10 | 11 | test("测试 expect 的用法") { 12 | assertResult(5) { 13 | // 6 14 | 5 15 | } 16 | 17 | /* 18 | === 会在错误时在控制台输出： 19 | 5 did not equal 6 (FunSuitee.scala:15) 20 | 但 == 不会。 21 | */ 22 | // assert(5 === 6) 23 | 24 | // expectXxx 已经不再 work，都改为 assertXxx 了。 25 | // expectResult(5) { 26 | // 6 27 | // } 28 | // 29 | // expect(5 === 6) 30 | } 31 | 32 | after { 33 | // ... 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /src/test/scala/sample/SetSpec.scala: -------------------------------------------------------------------------------- 1 | package sample 2 | 3 | import org.scalatest.Spec 4 | 5 | class SetSpec extends Spec { 6 | object `A Set` { 7 | object `when empty` { 8 | def `should have size 0` { 9 | assert(Set.empty.size === 0) 10 | } 11 | 12 | def `当 '.head' 被调用，应该产生 NoSuchElementException 异常` { 13 | assertThrows[NoSuchElementException] { 14 | Set.empty.head 15 | } 16 | } 17 | } 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /src/test/scala/sample/TVSetActorSpec.scala: -------------------------------------------------------------------------------- 1 | package sample 2 | 3 | import org.scalatest.{AsyncFeatureSpec, GivenWhenThen} 4 | 5 | import scala.concurrent.{ExecutionContext, Future} 6 | 7 | // Defining actor messages 8 | case object IsOn 9 | case object PressPowerButton 10 | 11 | class TVSetActor { // Simulating an actor 12 | private var on: Boolean = false 13 | 14 | def !(msg: PressPowerButton.type): Unit = synchronized { 15 | on = !on 16 | } 17 | 18 | def ?(msg: IsOn.type)(implicit c: ExecutionContext): Future[Boolean] = Future { 19 | synchronized { 20 | on 21 | } 22 | } 23 | } 24 | 25 | class TVSetActorSpec extends AsyncFeatureSpec with GivenWhenThen { 26 | implicit override def executionContext: ExecutionContext = scala.concurrent.ExecutionContext.Implicits.global 27 | 28 | info("As a TV set owner") 29 | info("I want to be able to turn the TV on and off") 30 | info("So I can watch TV when I want") 31 | info("And save energy when I'm not watching TV") 32 | 33 | Feature("TV power button") { 34 | Scenario("User presses power button when TV is off") { 35 | 36 | Given("a TV set that is switched off") 37 | val tvSetActor = new TVSetActor 38 | 39 | When("the power button is pressed") 40 | tvSetActor ! PressPowerButton 41 | 42 | Then("the TV should switch on") 43 | val futureBoolean = tvSetActor ? IsOn 44 | futureBoolean map { isOn => assert(isOn) } 45 | } 46 | 47 | Scenario("User presses power button when TV is on") { 48 | 49 | Given("a TV set that is switched on") 50 | val tvSetActor = new TVSetActor 51 | tvSetActor ! PressPowerButton 52 | 53 | When("the power button is pressed") 54 | tvSetActor ! PressPowerButton 55 | 56 | Then("the TV should switch off") 57 | val futureBoolean = tvSetActor ? IsOn 58 | futureBoolean map { isOn => assert(!isOn) } 59 | } 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /src/test/scala/sample/TVSetTest.scala: -------------------------------------------------------------------------------- 1 | package sample 2 | 3 | import org.scalatest.{FeatureSpec, GivenWhenThen, Matchers} 4 | 5 | class TVSetTest extends FeatureSpec with GivenWhenThen with Matchers { 6 | info("As a TV Set owner") 7 | info("I want to be able to turn the TV on and off") 8 | info("So I can watch TV when I want") 9 | info("And save energy when I'm not watching TV") 10 | 11 | Feature("TV power button") { 12 | Scenario("User press power button when TV is off") { 13 | Given("a TV set that is switched off") 14 | val tv = new TVSet 15 | tv.isOn should be(false) 16 | 17 | When("The power button is pressed") 18 | tv.pressPowerButton 19 | 20 | Then("The TV should switch on") 21 | tv.isOn should be(true) 22 | } 23 | } 24 | } 25 | 26 | class TVSet { 27 | private var _on = false 28 | 29 | def isOn = _on 30 | 31 | def pressPowerButton: Unit = { 32 | _on = true 33 | } 34 | } 35 | -------------------------------------------------------------------------------- /why-snatcher.md: -------------------------------------------------------------------------------- 1 | [Snatcher](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/tool/Snatcher.scala) 是 [Reflow](https://github.com/bdo-cash/reflow) 框架中一个极其重要的 **抗阻塞** 线程同步工具。贯穿于`Reflow`框架的各个角落，是`Reflow`无锁（`lock-free`）的基础。 2 | 3 | 你可能立马要问了：`java`有丰富的锁实现，如最基础的关键字`synchronized`，以及`ReentrantLock`、`ReadWriteLock`、`StampedLock`等。你为何自作聪明要实现一套呢？这里暂重申关键词： 4 | > 无锁（**`lock-free`**） 5 | 6 | #### 一、举例类比 7 | 8 | 为了更形象地介绍`Snatcher`组件，下面举一个假想场景的例子以帮助更好地理解。现假设： 9 | > 有一个理想状态下的工厂。每当工人完成一项工作任务`T`，就需要去一个固定的地点`P`打卡交付任务，并去下一个地点领取下一项任务`T1`的安排及必要的工具和物资，或临时休息（可不领取物资）。`T1`是不可预测的，需要根据上一个领取的任务`T0`决定。报酬会按每项任务的难度和时长合理分配，很公平（不用担心领取任务的“优劣”）。而打卡交付的顺序也是不可预测的，因为每个任务的时长不同。基于上述原因，这个地点`P`有下列问题： 10 | > 1. 基于功能和职责考虑，必须有且只有一个`P`地点； 11 | > 2. 是一个房间，工作内容`T'`是根据屏幕显示的指令`C`处理交付的物品（`C`需要根据实时的资源状况和从传送带输入的物品实时计算得来，不可预知），只需一个工人在此处工作即可； 12 | > 3. 人流量很随机，有时很拥挤，有时没人，有时断断续续有零星几个人前来打卡交付物资； 13 | > 4. 没人愿意在此地工作，因为不能自由安排自己的时间，报酬还不一定多。因为每天工人领到的任务时长也很随机，导致可能某天打卡交付的总体人流量很少，而工作`T'`的报酬也是按劳分配，没有例外。 14 | > - 这也显示了整体工作量不够饱和，浪费人力资源。 15 | 16 | 现在问： 17 | > 如何合理地解决在打卡地`P`的工人安排的问题，而且最好是最优解（使效率最大化，整体工作量饱和）？ 18 | - 重申以下前提： 19 | - 工厂里的每个员工都能够自由地安排自己的时间； 20 | - 工厂里的每个员工都能胜任任何工作（工作`T'`也很简单，而且每处理 _一个人交付的物品_ 的时长也极短，不会大量堆积）； 21 | - 工厂里的每个工作任务都是按劳分配报酬（不会因为任务特殊或没人愿意干而增加报酬）。 22 | 23 | 基于以上条件，现设计出如下规则和流程： 24 | > 1. 每个人`W`在到达地点`P`后，先按下一个信号灯`S0`（打卡和信号合二为一，房间内会亮灯，表示有人来，房间外的大按钮只能点亮）； 25 | > 2. 然后`W`需要看`P`房间里有没有人（门口外面有灯`S1`，房间只有一个门），此时有两种情况： 26 | > - a. 有人（灯`S1`亮）：`W`可以走了（去领取下一个任务，或休息）； 27 | > - b. 没人（`S1`不亮）：`W`需要进入房间并关好门（点亮`S1`），并接手工作`T'`。步骤： 28 | > - ⅰ. 先把灯`S0`熄灭（只能在房间内关闭该灯），以免误判又有人来了（可减少不必要的等待）； 29 | > - ⅱ. 根据屏幕显示的指令操作，直到传送带上 **没有** 物品为止； 30 | > - ⅲ. 看灯`S0`是否点亮，又有两种情况： 31 | > - ①. 亮：重复上述`2.b.ⅰ ~ 2.b.ⅲ`的操作； 32 | > - ②. 灭：打开房门（灯`S1`熄灭），并回头看一眼`S0`： 33 | > - 亮：回到上述`2.b`分支（即：重新关好房门，并重复上述`2.b.ⅰ ~ 2.b.ⅲ`的操作）； 34 | > - 灭：回到上述`2.a`分支（即：离开）。 35 | * 该流程没有要求工人交付任务时必须排队，只要保证把物品放入传送带，并按下`S0`。这样做的目的是使效率最大化：工人可以及时领取下一个任务或休息。 36 | 37 | #### 二、方案分析 38 | 39 | 1. 如果有人`W'`不看灯`S1`，并且直接走了，那么也有几种情况： 40 | - `S1`亮：有人处理，`W'`获得该任务报酬； 41 | - `S1`灭： 42 | - `W'`之后没有其他人（当天）：`W'`的任务得不到真正交付，得不到报酬； 43 | - `W'`在打卡时人很多：别人会看灯`S1`并保证有人处理，`W'`获得该任务报酬； 44 | - `W'`在打卡时只有自己，但过了一会又有人`W''`来打卡： 45 | - `W''`看灯`S1`并保证有人处理，`W'`获得该任务报酬； 46 | - `W''`不看灯，成为`W'`，又回到开头的 case。 47 | 2. 在重复`2.b.ⅰ ~ 2.b.ⅲ`的操作，执行到步骤`2.b.ⅱ`时，发现传送带上并没有物品。其实这是可预见的，也是规则设定的一部分，所以无妨。这说明在上一轮的步骤`2.b.ⅱ`都已经处理完（不考虑工人作弊的情况，有其它机制处理作弊的工人，但与进入房间接手工作`T'`的人`W`无关），如果接下来`S0`也没有被点亮，就意味着`W`此时已经顺利完成工作`T'`，并可以获得相应的报酬。 48 | 3. 如果`W`看到`S0`未被重新点亮，打开房门（灯`S1`熄灭），而 _**不**_ 回头看一眼`S0`，直接离开。会出现什么问题？ 49 | - a. 没问题。回头看一眼`S0`，大多数情况是不亮的（和不看没区别）；即使亮了，外面的人此时也会看到`S1`是熄灭的状态，进而尝试进来（与正要出来的`W`有个沟通决定谁进去）； 50 | - b. 临界情况。外面只有一个人`W'`（后面很久没人来），且看到`S1`是亮的，所以走了。但与此同时，`W`已经看完`S0`（未被重新点亮），准备开门还未打开（刚达到步骤`2.b.ⅲ.②`还未执行，且动作稍慢）。接下来，`S0`亮（因为`W'`总是先点亮`S0`再看`S1`，但动作比较快，几乎同时），`W`开门，直接离开。**现按时间先后顺序梳理一下发生的动作：`W`看`S0`（熄灭），`W'`点亮`S0`，`W'`看`S1`（亮灯），`W'`离开（跑的飞快），`W`开门，`W`离开。** 都走了（重申：后面很久没人来），就漏掉了对`W'`交付的处理。所以这个 **`W`回头看一眼`S0`** 的规则设定很有必要： 51 | - 大多数情况下，走上述`3.a`的 case； 52 | - 再看`3.b`，时间线变成了：`W`看`S0`（熄灭），`W'`点亮`S0`，`W'`看`S1`（亮灯），`W'`离开（跑的飞快），`W`开门，**`W`回头看一眼`S0`，发现灯亮，重新关好房门继续工作。** 53 | 4. 如果不按上述规则和流程走，而是各自进入房间处理自己交付的物品，会有什么问题？显然会堵塞拥挤不堪，原本人流量不密集的，这时也大概率会拥堵。因为工人进进出出，既大幅占用总体时间，也有巨大的沟通摩擦成本。势必效率低下。 54 | 55 | > 到此，是不是完美地解决了在打卡地的工人安排的问题？！ 56 | > 而且房间里处理交付物品的人`W`丝毫不需要多余的等待，干完活就走，也使效率最大化了。至于人流拥挤时，`W`工作时长多一些，但多劳多得（这时可以变通一下，把门打开或把`S1`按灭，如果有人进来，`W`可以商量换人；没有人进来，则说明传送带上已经没有多少物品需要处理了）。 57 | 58 | #### 三、代码解析 59 | 60 | 现在回过头来类比到`Snatcher`组件的源代码。 61 | ```Scala 62 | // 对应上述信号灯`S0` 63 | private val signature = new AtomicBoolean(false) 64 | // 对应上述信号灯`S1` 65 | private val scheduling = new AtomicBoolean(false) 66 | … 67 | def snatch(): Boolean = { 68 | // 外面的人按下信号灯`S0`（true 表示亮灯） 69 | signature.set(true) 70 | // 具体看下面的实现 71 | tryLock() 72 | } 73 | … 74 | private def tryLock() = 75 | // 这是一个合成操作，也是原子操作： 76 | // 保证了“看灯、推门进入、关门（点亮`S1`）”这三个动作 77 | // 一气呵成，不会有第二个人同时进入房间（如果灯`S1`是熄灭状态）， 78 | // 对应于上述步骤`2.a`和`2.b`，不过还未到`2.b.ⅰ`。 79 | // 这里第一个参数`false`，表示“如果灯`S1`是熄灭状态”，第二个参数`true`表示“那就点亮`S1`吧”，合起来就是： 80 | // 如果灯`S1`是熄灭状态，那就点亮吧（代码层面其实没有“门”，只有信号。但有个抢的概念，抢到为上，如果 81 | // 符合本条代码所表示的规则而且执行返回`true`，就意味着抢到了。代码会严格按规则行事，不会像现实中的人那样不遵守规定乱来）。 82 | if (scheduling.compareAndSet(false, true)) { 83 | // 到步骤`2.b.ⅰ`了：熄灭`S0`（false 表示熄灯）。 84 | signature.set(false) 85 | // 暂不用管 86 | thread.set(Thread.currentThread) 87 | // 因为上面已经抢占成功，就还是返回抢占成功的标识。 88 | true 89 | } else { 90 | // 到步骤`2.a`了，与上面的 case 对应，返回表示抢占失败的标识。 91 | false 92 | } 93 | … 94 | 95 | // Snatcher 组件的使用入口（理解了逻辑，也可以根据需要自己改写）。 96 | def tryOn(doWork: => Unit, …): Boolean = { 97 | … 98 | // 这里返回的就是上述的抢占成功或失败 99 | if (snatch()) { // 如果抢占成功 100 | breakable { while (true) { 101 | // 干活，对应上述步骤`2.b.ⅱ`。 102 | doWork 103 | // 干完活，看灯`S0`，对应上述步骤`2.b.ⅲ`的所有 case，以决定是继续重复干活，还是开门离开。 104 | // 具体看下面的实现 105 | if (!glance()) { 106 | break // 表示开门离开 107 | } 108 | // 没走`break` case，那就继续走`while (true)`循环，重复`2.b.ⅰ ~ 2.b.ⅲ`的操作。 109 | } } 110 | true 111 | } else false 112 | } 113 | … 114 | def glance(): Boolean = 115 | // 看灯`S0`，对应步骤`2.b.ⅲ`。 116 | // 也是个合成的原子操作，意思是说：如果灯`S0`被点亮，那就按灭，并继续干活；否则开门… 117 | if (signature.compareAndSet(true, false)) { 118 | // 指示继续干活的标识，对应步骤`2.b.ⅲ.①`。 119 | true.ensuring(scheduling.get, "xxx") 120 | } else { 121 | // 暂不管它 122 | thread.set(null) 123 | // 对应步骤`2.b.ⅲ.②`：打开房门（灯`S1`熄灭）。 124 | scheduling.set(false) 125 | // 对应步骤`2.b.ⅲ.②`：回头看一眼`S0`。如果灯`S0`灭，则直接返回指示“离开”的标识（即：false，也正是灯灭的信号）。 126 | signature.get 127 | // 对应步骤`2.b.ⅲ.②`下面的灯`S0`亮分支。这里与现实中稍有不同的是：因为刚刚打开房门了，需要重新抢占 128 | //（重申：代码层面没有“门”的概念），抢占成功，则返回指示“继续”的标识，否则返回指示“离开”的标识。 129 | && tryLock() // 具体实现，前面已经分析过（字面上也形成闭环了）。 130 | } 131 | ``` 132 | * `Snatcher`整个代码里没有类似`synchronized`、`ReentrantLock`之类的同步锁机制，仅用了`cas`操作实现了两个信号的原子切换，所以，是无锁（`lock-free`）的，事实上，还是`wait-free`的。 133 | > 这就是无锁吗？总感觉哪里不对（我读书少，你不要骗我 😂）。 134 | 135 | #### 四、无锁为什么 136 | 137 | > 现在大概理解了`Snatcher`的实现逻辑或原理。但跟`Reflow`的无锁有什么关系？这样就能说`Reflow`是无锁的吗？ 138 | 139 | 众所周知，`lock-free/wait-free`是多线程/并发编程的最高境界，是高性能程序的基石，也是程序员不懈追求的目标。`lock-free`的程序意味着每个线程都能够畅通无阻地跑满（或空闲），而对共享对象的访问操作依然井然有序，不会对它们造成破坏性影响；而不用时不时等待（暂停，工作不饱和）其它线程释放锁，从而自己能够获得锁，进而得以执行某些原子操作。而且有锁的代码，有潜在死锁的风险。 140 | * 关于`lock-free/wait-free`更专业的说法，一搜一大把，就不赘述了。 141 | * 为什么要尽可能跑满？就像工人的工作量不饱和，就会导致整体效率低下。对于特定的一台设备，`CPU`等硬件资源是固定的，如果不能使线程跑满，想要达到相同的效果，就需要启动更多的线程，但又会消耗内存和`CPU`时间分片，显然不如跑满。 142 | 143 | 前面说了： 144 | > `Snatcher`贯穿于`Reflow`框架的各个角落，是`Reflow`无锁（`lock-free`）的基础。 145 | 146 | `Reflow`里有许多类似上述 _打卡地`P`房间_ 的操作：某些事只需要一个人（线程）干就好了，其它人（线程）可以马上略过去干其它事情，没必要在这干等着。 147 | * 线程的调度完全可以类比为 **某 leader 指挥某 team 的所有人干活**。如果团队里总有一些人偷懒，或由于某些原因 _需要等待别人完成某任务后_ 自己才能继续，显然工作量不饱和，效率较低。 148 | 149 | 例如，`Reflow`里有这样一个场景：_**从四个优先级桶中取出某桶里的某个任务丢进线程池**_，需要有人（线程）干这件事，那派谁呢？事实上，有启动`Reflow`工作流的线程做这件事；也有线程池中干完了某 _任务_ 的线程做这件事；可能线程池里的多个线程同时干完自己当前的 _任务_，然后都同时干这件事。像极了 _**`P`房间**_ 的场景，`Snatcher`组件就派上用场了，再也不用纠结到底派哪个线程干，调用`Snatcher`的`tryOn()`方法即可。在这里，各个线程就像`工厂模型`里面的工人，到达房间后只需 **按灯S0、看灯S1**，或进入房间，或直接略过。畅通无阻，丝毫不需等待停留。写法如下： 150 | ```Scala 151 | // 用法特别简单 152 | snatcher.tryOn { 153 | // “从四个优先级桶中取出某桶里的某个任务丢进线程池”的一系列操作 154 | } 155 | ``` 156 | 157 | 但假如用传统的多线程编程方式，一般有如下写法： 158 | ```Java 159 | void xxx() { 160 | … 161 | synchronized(object) { 162 | // “从四个优先级桶中取出某桶里的某个任务丢进线程池”的一系列操作 163 | } 164 | … 165 | } 166 | // 或： 167 | def xxx() { 168 | … 169 | lock.lock() 170 | try { 171 | // “从四个优先级桶中取出某桶里的某个任务丢进线程池”的一系列操作 172 | } finally { 173 | lock.unlock() 174 | } 175 | … 176 | } 177 | ``` 178 | 无论哪种写法，当某线程到达`synchronized`或`lock.lock()`时，都要看其它线程是否同时到达这里或其内部（还未出`synchronized`代码块，或`lock.unlock()`），没有很好（该线程直接进入），但如果有，则需要等那个线程出`synchronized`代码块，或`lock.unlock()`。显然，线程的工作量没有无锁的饱和，效率没有最大化。就相当于在`工厂模型`中，_**每个打卡交付物品的工人都得自己进入`P`房间，处理自己的物品，然后出来，另一个人再进房间**_ 这样一个逻辑，显然效率极低。 179 | 180 | * 顺便说一句，同步块语句（或方法）保证`原子性`的底层逻辑，是 [happens-before](https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.5) 原则。而该原则又基于`内存可见性`概念（用`visible`解释`happens-before`）。 181 | 182 | 最后的重要又关键的问题：**`snatcher.tryOn()`有没有`原子性`或`happens-before`保证，又是如何保证的？** 183 | - **有！！！** 不过`原子性`取决于用法和界定，类比锁机制来说，也是如此（必须公用同一个锁对象）。 184 | - 对于`happens-before`，两个信号灯就是保证，具体来说： 185 | ``` Scala 186 | // 对应信号灯`S0` 187 | val signature = new AtomicBoolean(false) 188 | // 对应信号灯`S1` 189 | val scheduling = new AtomicBoolean(false) 190 | ``` 191 | `AtomicBoolean`内部有个`volatile`变量，信号灯就是修改的这个变量。`volatile`在`java 内存模型`中，有明确的`happens-before`支持：原生的 **可见性** 保证。但这还不够，因为仅仅是针对访问`volatile`变量的。我要的是对`snatcher.tryOn(doWork)`代码块也有`happens-before`保证，这恰好涉及到`volatile`的另一个能力：**禁止指令重排序**，配合可见性，就实现了该保证。具体来说： 192 | > 以信号`S0`开始，保证了 **工人把物品放入传送带后打卡（按灯）** 的操作结果（程序中的数据或对象）对之后的工作`T'`（`doWork`）可见，而以信号`S1`收尾又保证了工作`T'`的结果对所有人（线程）可见。 193 | 194 | 195 | #### 五、缺点 196 | 197 | 对于`Snatcher`的使用，需要特别注意是： 198 | - `tryOn()`的参数`doWork`（即：具体工作内容的代码块），不能是带上下文参数的，必须是固定的操作（字面量是相同的，且对于任何线程都一样）。如果需要带上下文参数，可使用 [Snatcher.ActionQueue](https://github.com/bdo-cash/reflow/blob/master/src/main/scala/hobby/wei/c/tool/Snatcher.scala#L160)。 199 | 200 | `Snatcher`实现起来比较简单，也有较大的缺点，没理解透彻就很容易出错，这可能是没有把它纳入到 java 标准库的原因。 201 | 202 | 203 | --------------------------------------------------------------------------------