├── FAQ.md ├── README.md ├── TermsOfUse.md └── assets ├── jupyter_landing.png ├── jupyter_signin.png ├── landingpage.png ├── signup_1.png ├── signup_2.png ├── webui_1.png └── webui_2.png /FAQ.md: -------------------------------------------------------------------------------- 1 | # Frequently Asked Questions 2 | 3 | By Julius Technologies Inc (email: info@juliustech.co) 4 | 5 | Julius Graph Engine is an auto-scaling, low-code and visual graph computing solution. It can 6 | build and run sophisticated data and analytical pipelines as directed acyclic graphs (DAGs) 7 | with very little coding. Readers are referred to: 8 | - [this blog](https://www.juliustech.co/blog/why-graph-computing-is-stellar) for a high 9 | level overview of the benefits of graph computing, 10 | - [this repo](https://github.com/JuliusTechCo/JuliusGraph) for instructions to sign up for 11 | free developer access, and 12 | - [these tutorials](https://juliustechco.github.io/JuliusGraph) for a compiled version of 13 | tutorials. 14 | 15 | This FAQ page answers some of the most common questions from our users and clients. They are 16 | organized by subjects. 17 | 18 | ### Graph Computing and RuleDSL 19 | 20 | 1. **What is graph computing?** 21 | 22 | Conceptually, any computation, from the simplest formulas in a spreadsheet to the most 23 | complex enterprise systems, reduces to a directed acyclic graph (DAG). Graph computing, 24 | at its core, is to define, orchestrate and optimize computations using the generic DAG 25 | representation. Please refer to [this blog](https://www.juliustech.co/blog/amazing-graph) 26 | for an introduction to graph computing. 27 | 28 | 2. **What are the benefits of graph computing and why do we need computational DAGs?** 29 | 30 | Graph computing solves the most fundamental and challenging problems for building 31 | complex data and analytical pipelines and systems. These challenges include scalability, 32 | transparency, explainability, lineage, adaptability and reproducibility. Not only 33 | computational DAG is a generic representation of any data and analytical pipeline, it is 34 | also an ideal representation for building generic solutions for these common challenges. 35 | Please refer to [this blog](https://www.juliustech.co/blog/why-graph-computing-is-stellar) 36 | for a full explanation. 37 | 38 | Julius Graph Engine is the first solution that delivers the full benefits of graph 39 | computing using a low-code Domain Specific Language (named `RuleDSL`), allowing a small 40 | development team to build sophisticated, scalable and transparent pipelines and systems 41 | with very little code. 42 | 43 | 3. **How does Julius build data and analytical pipelines as computational DAGs?** 44 | 45 | `Atom` and `RuleDSL` are the two fundamental constructs in Julius for creating 46 | computational DAGs. 47 | 48 | `Atom` stands for atomic operation, it encapsulates the functions being performed at 49 | individual nodes in Julius' computational DAG. Atom is the minimal unit of distribution 50 | and caching in Julius' computational DAG. An `Atom` can be implemented in any mainstream 51 | programming language, such as C++, Python, Java, Julia, .Net and R, by inheriting from 52 | an Atom base class or interface. Existing libraries written in these languages can be 53 | easily wrapped up under the `Atom` interface. 54 | 55 | `RuleDSL` is a high level declarative domain specific language (DSL) for defining 56 | computational DAGs. `RuleDSL` features an easy to use and low-code syntax that connects 57 | individual Atoms to create the computational DAG, which can represent the entire data 58 | and analytics pipeline or workflow. 59 | 60 | Please refer to Julius [tutorials](https://juliustechco.github.io/JuliusGraph) for more 61 | detailed explanations and examples of using Atom and RuleDSL. 62 | 63 | 4. **How does Julius Graph Engine access data?** 64 | 65 | Julius Graph Engine creates and runs computational DAGs in two main stages: 66 | 67 | * First, it builds the data and analytical pipeline as a computational DAG (directed 68 | acyclic graph) from `RuleDSL`, 69 | * Then, it run the computational DAG to process the data and produce the desired 70 | results. 71 | 72 | Julius can access data in both stages, from any data source, such as databases, web URLs, 73 | static files, etc. 74 | 75 | The key advantage of the two stage data feeds is that the first stage usually requires a 76 | small amount of data for constructing the DAG. Once the DAG is built, it can be 77 | distributed to multiple computers, so that the heavy lifting of data processing in the 78 | second stage can be done in parallel. This achieves much better performance for both 79 | graph creation and execution. A developer has full control over which types of data are 80 | fed at each point in time through `RuleDSL`. 81 | 82 | 5. **How does `RuleDSL` differ from ordinary programming languages such as Python/C++/Java?** 83 | 84 | General purpose programming languages like Python, C/C++ and Java are low-level and 85 | imperative, a developer has to specify the program's entire execution step by step, 86 | which requires a lot of coding. In comparison, `RuleDSL` is a high level declarative 87 | domain specific language (DSL) for constructing computational DAGs. Using `RuleDSL`, a 88 | developer doesn't need to explicitly specify every step of a program's execution, 89 | instead they only need to declare the key data or analytical concepts and their logical 90 | dependencies. Then, Julius Graph Engine automatically creates an executable 91 | computational DAG from the `RuleDSL` declarations. 92 | 93 | Therefore, `RuleDSL` requires much less code to specify the same data or analytical 94 | logic compared to traditional programming languages, as most of the boilerplate flow 95 | control code is automatically generated by the Julius Graph Engine. Therefore, `RuleDSL` 96 | doesn't require any of the complex control flow syntax found at traditional programming 97 | languages, such as variables, loops, branches, functions, classes, inheritances, etc. 98 | This makes it much easier for non-programmers to learn and use. Furthermore, RuleDSL's 99 | declarative syntax gives the Julius Graph Engine a lot more freedom to optimize its 100 | execution at run time, which allows for efficient auto-scaling and parallelization over 101 | multiple machines. 102 | 103 | Julius `RuleDSL` is an exemplification of how less can be more: less coding in `RuleDSL` 104 | actually facilitates much better scalability and performance. 105 | 106 | 6. **How can a computational DAG, which is acyclic, support loops?** 107 | 108 | It is true that there can't be any loops in the DAG by construction. However, it doesn't 109 | mean that we can't express looping logic in a DAG. There are multiple ways to do so in a 110 | DAG: 111 | 112 | * loops can be created inside an Atom (i.e., within a single node in the DAG), 113 | * create a set of nodes, where each node represents a batch of iterations in the loop 114 | (for example, if a Monte Carlo simulation runs 100,000 paths, we can create 100 115 | nodes, each running 1,000 MC paths), 116 | * use recursion in RuleDSL if there are dependencies between loops, 117 | * run the computational DAG multiple times or in streaming mode. 118 | 119 | Any of these options can be easily implemented using RuleDSL, without the need for any 120 | special syntax for looping. There are examples of recursion and Monte Carlo via 121 | streaming at the [tutorials](https://juliustechco.github.io/JuliusGraph). 122 | 123 | 5. **How can you express conditional branches in RuleDSL and computational DAGs?** 124 | 125 | Similar to looping, conditional branches can also be expressed in multiple ways: 126 | 127 | * via rule polymorphism in RuleDSL, where a different rule is invoked depending on the 128 | type or values of the rule parameter during DAG creation, 129 | * write an Atom with a select logic, where one of the inputs is selected as output 130 | according to certain conditions at run time. 131 | 132 | The first approach is more efficient as the branch condition is resolved during DAG 133 | construction. The second approach is required when branching depends on data at run 134 | time. Either of these options can be easily expressed in RuleDSL, without the need for 135 | any special branching syntax. 136 | 137 | ### Capabilities and Use Cases 138 | 139 | 8. **What are the ideal use cases for graph computing and Julius Graph Engine?** 140 | 141 | The Julius Graph Engine is well suited for building complex data transformation and ML 142 | workflows. Julius adds value to enterprise systems with complex data and analytical 143 | pipelines which tend to suffer more from poor transparency, lineage, scalability and 144 | visibility. Business decisioning that requires complex rules, regular updates, and quick 145 | responses are ideal applications for Julius as well. Opaque processes that demand 146 | explainability and auditability to minimize operational risk are also applicable. In 147 | addition, Julius excels at processing large data sets and heavy workloads because of 148 | its auto-scaling capabilities. 149 | 150 | 1. **Can I use Julius for building machine learning applications and workflows?** 151 | 152 | Definitely. A ML pipeline or workflow is by definition a DAG, and Julius can 153 | automatically create and execute them. Julius' computational DAG offers tremendous 154 | transparency, explainability and auditability for building ML pipelines and 155 | applications. It can incorporate all the steps in a typical ML pipeline, including data 156 | cleansing, feature engineering, training, inference and hyper parameter tuning. It also 157 | offers great ML Ops support, such as easy persisting and recovery of ML experiments. 158 | Please refer to Julius [tutorials](https://juliustechco.github.io/JuliusGraph) for ML 159 | examples. 160 | 161 | 1. **Can Julius Graph Engine handle large data sets and heavy computational loads?** 162 | 163 | Absolutely. Julius can automatically distribute the computational DAG to many computers, 164 | without the need for any code changes. The ability to scale without code change is a 165 | huge advantage of Julius, as it allows for efficient parallel processing of large data 166 | set and heavy computations. This can facilitate the quick migration of a model from 167 | development to production. The distributed machine learning in Julius 168 | [tutorials](https://juliustechco.github.io/JuliusGraph) shows how easy it is to 169 | automatically distribute computational DAGs. 170 | 171 | 1. **Does Julius Graph Engine support streaming and live use cases?** 172 | 173 | Absolutely, the same computation DAG can run in either batch or streaming mode, without 174 | much code change. Please see the [tutorials](https://juliustechco.github.io/JuliusGraph) 175 | for examples of streaming use cases. 176 | 177 | 1. **How big can Julius' computational DAG be?** 178 | 179 | Julius Graph Engine can quickly create DAGs with tens of millions of nodes from a single 180 | computer. Julius also supports parallelism for DAG creation, so graphs with even 181 | billions of nodes can be efficiently created in parallel. Once created, Julius can 182 | automatically distribute the computations to hundreds of computers for efficient 183 | execution. 184 | 185 | 1. **Are the intermediate results in the computational DAG cached by Julius?** 186 | 187 | Yes, Julius automatically caches all the intermediate results for every node in the 188 | computational DAG. The cached results are tremendously useful to both developers and 189 | users. This includes the ability to: 190 | 191 | * explain, audit and verify the correctness of the entire process, 192 | * debug and track issues, 193 | * enable fast scenario runs, as only dependent nodes needs to be re-computed from any 194 | input changes, 195 | * facilitate adjoint algorithmic differentiation (AAD). 196 | 197 | The storage cost to cache all intermediate node level results in memory is small, as 198 | Julius can automatically distribute the computational DAG to a large number of computers 199 | and access an almost unlimited amount of memory. 200 | 201 | 1. **How do I access and visualize intermediate results in the computational DAG?** 202 | 203 | Julius comes with an intuitive and easy-to-use Web UI, where a user can navigate the 204 | computational DAG and visualize all the intermediate data by simply clicking on 205 | individual nodes. All the data is accessible via Julius API as well. Please refer to the 206 | [tutorials](https://juliustechco.github.io/JuliusGraph) for examples of using the web UI 207 | and Julius API. 208 | 209 | 1. **Does Julius Graph Engine support adjoint algorithmic differentiation (AAD) ?** 210 | 211 | Yes, Julius computational DAGs support AAD. The best part is that it doesn't need a 212 | tape, as all the primal calculations results are already cached in the DAG. Please refer 213 | to the [tutorials](https://juliustechco.github.io/JuliusGraph) for more information and 214 | examples on AAD. 215 | 216 | ### Integration 217 | 218 | 16. **What programming languages can I use with Julius Graph Engine?** 219 | 220 | Atoms can be written in any mainstream programming language such as C/C++, Python, Java, 221 | .Net, R, Julia, etc. Existing data and analytical libraries written in these languages 222 | can be wrapped up as `Atoms` and used by Julius Graph Engine. 223 | 224 | The Julius Graph Engine also features easy-to-use APIs that can be called from any 225 | programming language. 226 | 227 | 1. **How do I convert my Python/C++/Java/.Net/R/Julia/SQL applications to Julius, and get 228 | the benefits of Graph?** 229 | 230 | The great news is that your existing data and analytical libraries in these languages 231 | can be reused. You don't have to abandon your battle tested libraries and restart from 232 | scratch! The migration of an application or system to Julius involves two easy steps: 233 | 234 | * wrap the existing data and analytical classes and functions using the Julius' `Atom` 235 | interface. Julius offers generic wrappers that work out of the box for modern 236 | programming languages that support reflection, such as Python, Java, R, .Net and 237 | Julia. This generic reflection based Atom wrapper works automatically for any 238 | classes and functions in these languages. For older languages that do not support 239 | reflection such as C/C++, there is some additional effort needed to write Atom 240 | wrappers for individual functions and classes. But this work is largely mechanical 241 | and can be completed quickly by experienced developers. 242 | * declare the entire data and analytical pipeline using Julius RuleDSL, which can 243 | reference and use the existing libraries via the `Atom` wrappers. This step only 244 | requires minimal effort and a small amount of coding, thanks to the low-code nature 245 | of `RuleDSL`. A complex data and analytical pipeline with hundreds or thousands of 246 | nodes can be built with a few dozen lines of rules in RuleDSL, as shown in Julius' 247 | [tutorials](https://juliustechco.github.io/JuliusGraph). 248 | 249 | The conversion is straight forward if the existing system is built in a modern 250 | programming language. Our experience shows that a complex real world system can be fully 251 | converted in a few weeks by developers who are knowledgeable with the system's business 252 | logic and code base. For C/C++ applications and systems, it would require some more work 253 | for writing and testing the wrappers manually. 254 | 255 | 2. **Can I mix and match Atoms written in different programming languages in the same 256 | computational DAG?** 257 | 258 | Yes! Julius automatically takes care of the interop between them if these Atoms' inputs 259 | and outputs are all of standard types. Additional development work, such as 260 | serialization, is only required if you want to pass a non-standard object from one 261 | language to another. 262 | 263 | 3. **How do I integrate Julius Graph Engine with an existing system or application?** 264 | 265 | Existing systems and applications can access Julius' Graph Engine via Julius' 266 | easy-to-use APIs. Julius supports a low level API for Julia, C/C++ and Python clients, 267 | as well as a high level REST API that can be called from any programming language. The 268 | low-level API is faster and more granular, which is recommended for building high 269 | performing systems. The high-level API is easy to use and integrate, which is 270 | recommended for general use cases. 271 | 272 | 4. **What databases and data sources does Julius support?** 273 | 274 | Julius comes with a rich set of data connectors out of box, including relational 275 | databases, NoSQL databases (Hadoop, Spark, etc), data files, web data sources, etc. If 276 | you need additional data connectors that Julius does not yet support, you can easily 277 | write your own connectors through the Atom interface, using any mainstream programming 278 | language. 279 | 280 | 5. **Can I use data visualization tools such as Tableau and Qlik with Julius?** 281 | 282 | Absolutely. Results from Julius graph can be fed into Tableau and Qlik for visualization 283 | and reporting. 284 | 285 | 286 | ### Installation 287 | 288 | 22. **How do I install Julius?** 289 | 290 | Julius installation is super easy: Julius ships with a single docker image. A client can 291 | simply install this docker image on whatever number of computers he/she chooses to run, 292 | which can be on a public cloud or private on-premise cluster. Julius only requires 293 | minimal configurations after installation, as most of the orchestration and 294 | configuration is handled automatically within the Julius docker container. 295 | 296 | Julius can also configure and host the installation in the cloud on behalf of the client. 297 | 298 | 1. **What operating systems does Julius Graph Engine support?** 299 | 300 | Both Linux and Windows. 301 | 302 | 1. **What hardware environment can Julius Graph Engine run on?** 303 | 304 | The Julius Graph Engine can be deployed to any cloud computer provider or private 305 | on-premise clusters in an organization. 306 | 307 | 1. **How can a client protect its intellectual property when running the Julius Graph Engine 308 | in the cloud?** 309 | 310 | The Julius Graph Engine can be integrated to a client's internal authentication process, 311 | so that only authorized personnel have access. Julius can be deployed on premise or in a 312 | single tenant environment on the cloud, providing additional protection. 313 | 314 | 1. **How much does it cost to license Julius?** 315 | 316 | Developers can access and use Julius for free. The instructions to sign up for developer 317 | access can be found [here](https://github.com/JuliusTechCo/JuliusGraph). The cost of 318 | commercial licenses depend on the use case and the level of support required. Please 319 | email us at info@juliustech.co for licensing questions. Julius does offer educational 320 | discounts for universities and other research institutions. 321 | 322 | ### Comparables 323 | 324 | 27. **What is the main difference between the Julius Graph Engine from a collaborative 325 | development platform like Beacon?** 326 | 327 | Platforms like Beacon offer integrated and collaborative development/deployment 328 | environments so that firms can be more efficient in managing and deploying complex code 329 | bases. In contrast, Julius Graph Engine attacks the problem from its root by 330 | dramatically reducing the amount of coding and efforts required for building complex 331 | systems. While using Julius, there is much less need for sophisticated collaborative 332 | development tools, as there is much less code to write and manage. 333 | 334 | 1. **How does Julius Graph Engine's RuleDSL and computation DAG differ from Excel's 335 | worksheet formula and its dependency graph?** 336 | 337 | Julius RuleDSL is much more generic and expressive than Excel's worksheet formulae. The 338 | RuleDSL supports both type and value based polymorphism, allowing sophisticated logic to 339 | be expressed succinctly. In comparison, Excel formulas are hard coded with specific 340 | cells or ranges. An Excel workbook often has thousands of formulae created by 341 | copy/paste, which is error prone and difficult to audit and change. 342 | 343 | In addition, the Julius Graph Engine is much more performant and scalable than Excel, as 344 | it can handle large volume of data and computation by running in parallel on many 345 | computers. That allows Julius to be used as a production solution for enterprise 346 | systems, while Excel is mainly a personal and office application. 347 | 348 | 1. **How does Julius Graph Engine differ from other software packages that also use 349 | computational DAGs, such as Dask, Tensorflow and Dagger.jl?** 350 | 351 | Julius RuleDSL is specifically designed for efficient creation and execution of 352 | computational DAGs. It offers much better performance and scalability, as shown in our 353 | [tutorials](https://juliustechco.github.io/JuliusGraph), specially at this 354 | [benchmark](https://juliustechco.github.io/JuliusGraph/dev/pages/t007_benchmark.html). 355 | In addition, Julius RuleDSL is low-code, and requires much less effort to implement the 356 | same functionality than other tools. This makes future adaptations and changes much 357 | easier as well. 358 | 359 | 2. **The Julius Graph Engine sounds almost too good to be true, why hasn't anyone else done 360 | this before?** 361 | 362 | The answer is twofold: the first is that there are a myriad of technological hurdles to 363 | overcome in order to design and implement such a generic and scalable graph computing 364 | solution; secondly the solution leverages many new technologies that have only become 365 | mature in recent years, such as cloud computing. 366 | 367 | ### Development Environment 368 | 369 | 31. **What IDE does Julius support?** 370 | 371 | Since RuleDSL is just regular code in plain text, developers are free to user their 372 | preferred IDEs. Jupyter lab/notebook and VScode are the most popular choices when 373 | working with Julius Graph Engine. 374 | 375 | 1. **How do I debug a large computational DAG?** 376 | 377 | Julius offers efficient and easy-to-use tools for debugging. With the Julius web UI, a 378 | developer can quickly navigate and visualize every intermediate result in the DAG, 379 | allowing them to quickly track down bugs to individual nodes. Julius provides an API 380 | call for developers to download the entire state (including data and analytical logic) 381 | of any node in the distributed DAG, to recreate the error conditions on their local 382 | computer. Then the developer can use a debugger on a local computer to step through the 383 | execution and pin down the error. 384 | 385 | 1. **What if there is an error in the computational DAG?** 386 | 387 | Julius offers great error handling and recovery capabilities. When an error occurs, the 388 | web UI will highlight the affected node and bring the user directly there with one 389 | click. By default, the DAG execution will halt until the error is corrected. Julius also 390 | supports a trigger mechanism that allows errors to be automatically reported and 391 | handled, so that the DAG execution can proceed even with errors. 392 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | | **Rapid Access** | 2 | |:--------------------------------------------------------------------------------------------------------- | 3 | | [](https://JuliusTechCo.github.io/JuliusGraph/dev/) | 4 | 5 | # Welcome to the New World of Graph Computing! 👋 6 | 7 | We are a group of researchers and engineers passionate about [graph computing](https://www.juliustech.co/blog/why-graph-computing-is-stellar). 8 | We believe graph technology solves some of the most challenging problems in computing. We spent the 9 | past few years at [Julius Technologies](https://www.juliustech.co/) building a truly 10 | innovative graph computing solution: the Julius Graph Engine. We are excited to offer 11 | developers free access to Julius' online development environment, where you can learn and 12 | test drive graph computing and graph programming using Julius Graph Engine. 13 | 14 | In this repository, you will find instructions to [register for free developer 15 | access](https://github.com/JuliusTechCo/JuliusGraph#register-for-developer-access) to 16 | Julius' development environment, and a number of tutorials that can help you get started. 17 | These tutorials illustrate the benefits of graph computing in solving real world problems. 18 | Once you go through these tutorials, you will appreciate the power, simplicity and elegance of graph 19 | computing and graph programming, you will never look at programming the same way again! 20 | 21 | Please raise a github issue in this repo for bugs or feature requests, or email us at info@juliustech.co 22 | for general questions or inquiries. 23 | 24 | ## Tutorials 25 | 26 | Our tutorials are available in two formats: 27 | 28 | - As interactive jupyter notebooks hosted in Julius' development environment. It is the best 29 | format for you to learn and experiment with graph computing interactively. To gain access 30 | to these online tutorials notebooks, please [register for developer 31 | access](https://github.com/JuliusTechCo/JuliusGraph#register-for-developer-access), it 32 | only takes a minute! 33 | 34 | - As static HTML pages, which contains the same descriptions, source code and results as the 35 | notebooks, but they are not interactive. If you want to read about Julius Graph Engine and 36 | its capabilities without register for developer access, please click 37 | [here](https://JuliusTechCo.github.io/JuliusGraph/dev/) to access the tutorials in HTML 38 | format. The HTML is also a good format to share with your colleagues or fellow developers. 39 | 40 | ## Register for Developer Access 41 | 42 | Register for developer access is super easy and it only takes a minute. Please proceed to 43 | [www.juliusgraph.com/user/signup](https://www.juliusgraph.com/user/signup). The web site uses Google authentication, 44 | you will be prompted to log in using your Google account credentials if you are not already 45 | signed in with Google: 46 | 47 |
48 |
49 |
56 |
57 |
63 |
64 |
71 |
72 |