├── .gitignore ├── .travis.yml ├── CONTRIBUTING.md ├── LICENSE ├── Makefile ├── README.md ├── docs ├── README.md ├── build.md ├── examples.md ├── img │ ├── image1.emf │ └── image2.png ├── java.jpg └── whitepaper.md ├── javakdb-examples ├── pom.xml └── src │ └── main │ └── java │ └── com │ └── kx │ └── examples │ ├── Feed.java │ ├── GridViewer.java │ ├── QueryResponse.java │ ├── SerializationOnly.java │ ├── Server.java │ ├── Subscriber.java │ ├── TypesMapping.java │ └── package-info.java ├── javakdb ├── pom.xml └── src │ ├── main │ └── java │ │ └── com │ │ └── kx │ │ ├── c.java │ │ └── package-info.java │ └── test │ └── java │ └── com │ └── kx │ └── CTest.java └── pom.xml /.gitignore: -------------------------------------------------------------------------------- 1 | **/.* 2 | build/ 3 | target/ 4 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: java 2 | os: linux 3 | dist: jammy 4 | jdk: 5 | - openjdk17 6 | addons: 7 | sonarcloud: 8 | organization: "kxsystems" 9 | token: "$SONAR_KEY" 10 | script: 11 | - mvn clean verify sonar:sonar -Dsonar.projectKey=KxSystems_javakdb 12 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to contribute 2 | 3 | Thanks for choosing to contribute to this project. 4 | 5 | If you haven't already, please view the [README](README.md) for an introduction to the aims and intended use of this project. 6 | 7 | ## Contributing as a user (non-development) 8 | 9 | As a non-developer, you can still contribute in many ways. We are especially interesting in hearing from people using the project in production/working environments. 10 | 11 | If you spot any problems, please raise an issue within the project and provide as much information as possible. This should enable us to recreate the issue and check that any fix is appropriate. 12 | 13 | Feature requests can also be raised in the project as a new issue. Please detail what you may wish to be added, why you would benefit from it and any other information that may be relevant. 14 | 15 | You may also wish to 'star' the project (click the star link on the project main page) to show your appreciation. 16 | 17 | ## Contributing as a developer 18 | 19 | ### Getting Started 20 | 21 | If you are looking to lend your development skills to the project, you can check the projects issue list to see what people may be looking for. Developers who already have an idea of what they might like to change should consider creating an issue to indicate to others of the work they are undertaking and commence any discussion on the possible solutions. 22 | 23 | ### Making Changes 24 | 25 | Please fork the projects master branch to your own account. Using your fork, you are free to create changes or create branches in isolation. 26 | 27 | The following provides a good general guide ['Beginners Guide To Contributing'](https://akrabat.com/the-beginners-guide-to-contributing-to-a-github-project/) 28 | 29 | Please keep your fork up-to date to reduce the risk of conflicts occurring when your come to submit any changes. 30 | 31 | It can also be worth considering if your changes may also require changes to documentation or dependent systems. 32 | 33 | ### Submitting Changes 34 | 35 | When committing changes, please provide a descriptive commit comment of why the change was made (e.g. 'fixed bug' is not a suitable comment as it doesn't describe which bug). 36 | 37 | You can link to a relevant issue in a commit comment by referencing the issue number prefixed with a '#'. 38 | 39 | After pushing to your fork, submit a pull request against the main projects master branch. 40 | 41 | In order to have your pull request approved in a timely manner, please provide comments on the pull request that details what was changed & for which reasons. The complexity or size of the change indicate the size of the descriptions required, in order for the reviewer to get up to speed as quick as possible. 42 | 43 | ### Additional Resources 44 | 45 | [KDB+ and q documentation](https://code.kx.com/q/) 46 | 47 | [KDB+ data types](https://code.kx.com/q/basics/datatypes/) 48 | 49 | [KDB+ serialization](https://code.kx.com/q/kb/serialization/) -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | mkdir -p build/classes 3 | javac -Xlint:deprecation -d build/classes src/main/java//kx/c.java src/main/java//kx/examples/Feed.java src/main/java//kx/examples/QueryResponse.java src/main/java//kx/examples/Subscriber.java src/main/java//kx/examples/Server.java src/main/java//kx/examples/TypesMapping.java src/main/java//kx/examples/GridViewer.java 4 | # jar cf build/c.jar build/kx/*.class 5 | doc: 6 | javadoc -d docs src/main/java//kx/c.java 7 | Feed: 8 | java -cp build/classes kx.examples.Feed 9 | Subscriber: 10 | java -cp build/classes kx.examples.Subscriber 11 | QueryResponse: 12 | java -cp build/classes kx.examples.QueryResponse 13 | Server: 14 | java -cp build/classes kx.examples.Server 15 | TypesMapping: 16 | java -cp build/classes kx.examples.TypesMapping 17 | 18 | GridViewer: 19 | java -cp build/classes kx.examples.GridViewer 20 | 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Java](docs/java.jpg) 2 | 3 | # javakdb 4 | 5 | ![Travis (.com) branch](https://img.shields.io/travis/com/kxsystems/javakdb/master) [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=KxSystems_javakdb&metric=coverage)](https://sonarcloud.io/dashboard?id=KxSystems_javakdb) 6 | 7 | 8 | kdb+ IPC interface for the Java programming language. This will allow your application to 9 | 10 | - query kdb+ 11 | - subscribe to a kdb+ publisher 12 | - publish to a kdb+ consumer 13 | - serialize/deserialize kdb+ formatted data 14 | - act as a server for a kdb+ instance 15 | 16 | ## Releases 17 | 18 | Latest release can be downloaded [here](https://github.com/KxSystems/javakdb/releases). The github master branch will contain the latest development version for testing prior to release (may contain planned major version changes). 19 | 20 | Releases are also available from the Maven Central Repo, at [https://central.sonatype.com/namespace/com.kx](https://central.sonatype.com/namespace/com.kx). A guide to integrating with your build system can be found [here](https://central.sonatype.org/consume/). 21 | 22 | ## Documentation 23 | 24 | :point_right: Documentation is in the [`docs`](docs/README.md) folder. 25 | 26 | ## API Reference 27 | 28 | HTML docs can be generated via running 29 | ``` 30 | mvn -pl javakdb javadoc:javadoc 31 | ``` 32 | 33 | ## Building From Source 34 | 35 | :point_right: [`Building guide`](docs/build.md) 36 | 37 | ## Code Examples 38 | 39 | :point_right: [`Examples`](docs/examples.md) 40 | 41 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | ![Java](java.jpg) 2 | 3 | # Using Java with kdb+ 4 | 5 | Javakdb is the original Java driver, a.k.a `c.java`, from KX for interfacing [Java](https://www.java.com/en/) with kdb+ via TCP/IP. This driver allows Java applications to 6 | 7 | - query kdb+ 8 | - subscribe to a kdb+ publisher 9 | - publish to a kdb+ consumer 10 | 11 | using a straightforward and compact API. The four methods of the single class `c` of immediate interest are 12 | 13 | method | purpose 14 | --------- | -------- 15 | `c`. | the constructor 16 | `c.ks` | send an async message 17 | `c.k`. | send a sync message 18 | `c.close` | close the connection 19 | 20 | To establish a connection to a kdb+ process listening on the localhost on port 12345, invoke the relevant constructor of the `c` class 21 | 22 | ```java 23 | c c=new c("localhost",12345,System.getProperty("user.name")+":mypasswordhere"); 24 | ``` 25 | 26 | A KException will be thrown if the kdb+ process rejects the connection attempt. 27 | 28 | Then, to issue a query and read the response, use 29 | 30 | ```java 31 | Object result=c.k("2+3"); 32 | System.out.println("result is "+result); // expect to see 5 printed 33 | ``` 34 | 35 | or to subscribe to a kdb+ publisher, here kdb+tick, use 36 | 37 | ```java 38 | c.k(".u.sub","mytable",x); 39 | while(true) 40 | System.out.println("Received "+c.k()); 41 | ``` 42 | 43 | or to publish to a kdb+ consumer, here a kdb+ ticker plant, use 44 | 45 | ```java 46 | // Assuming a remote schema of 47 | // mytable:([]time:`timespan$();sym:`symbol$();price:`float$();size:`long$()) 48 | Object[]row={new c.Timespan(),"SYMBOL",new Double(93.5),new Long(300)}; 49 | c.k(".u.upd","mytable",row); 50 | ``` 51 | 52 | And to close a connection once it is no longer needed: 53 | 54 | ```java 55 | c.close(); 56 | ``` 57 | 58 | > **Closing unused connections** 59 | > 60 | > Closing unused connections is important to help avoid unnecessary resource usage on the remote process. 61 | 62 | The Java driver is effectively a data marshaller between Java and kdb+: sending an object to kdb+ typically results in kdb+ evaluating that object in some manner. The default message handlers on the kdb+ side are initialized to the kdb+ `value` operator, which means they will evaluate a string expression, e.g. 63 | 64 | ```java 65 | c.k("2+3") 66 | ``` 67 | 68 | or a list of (function; arg0; arg1; ...; argN), e.g. 69 | 70 | ```java 71 | c.k(new Object[]{'+',2,3}) 72 | ``` 73 | 74 | Usually when querying a database, one would receive a table as a result. This is indeed the common case with kdb+, and a table is represented in this Java interface as the `c.Flip` class. A flip has an array of column names, and an array of arrays containing the column data. 75 | 76 | The following is example code to iterate over a flip, printing each row to the console. 77 | 78 | ```java 79 | c.Flip flip=(c.Flip)c.k("([]sym:`MSFT`GOOG;time:0 1+.z.n;price:320.2 120.1;size:100 300)"); 80 | for(int col=0;col0?",":"")+flip.x[col]); 82 | System.out.println(); 83 | for(int row=0;row0?",":"")+c.at(flip.y[col],row)); 86 | System.out.println(); 87 | } 88 | ``` 89 | 90 | resulting in the following printing at the console 91 | 92 | ```csv 93 | sym,time,price,size 94 | MSFT,15:39:23.746172000,320.2,100 95 | GOOG,15:39:23.746172001,120.1,300 96 | ``` 97 | 98 | A keyed table is represented as a dictionary where both the key and the value of the dictionary are flips themselves. To obtain a table without keys from a keyed table, use the `c.td(d)` method. In the example below, note that the table is created with `sym` as the key, and the table is unkeyed using `c.td`. 99 | 100 | ```java 101 | c.Flip flip=c.td(c.k("([sym:`MSFT`GOOG]time:0 1+.z.n;price:320.2 120.1;size:100 300)")); 102 | ``` 103 | 104 | To create a table to send to kdb+, first construct a flip of a dictionary of column names with a list of column data. e.g. 105 | 106 | ```java 107 | c.Flip flip=new c.Flip(new c.Dict( 108 | new String[]{"time","sym","price","volume"}, 109 | new Object[]{new c.Timespan[]{new c.Timespan(),new c.Timespan()}, 110 | new String[]{"ABC","DEF"}, 111 | new double[]{123.456,789.012}, 112 | new long[]{100,200}})); 113 | ``` 114 | 115 | and then send it via a sync or async message 116 | 117 | ```java 118 | Object result=c.k("{x}",flip); // a sync msg, echos the flip back as result 119 | ``` 120 | 121 | ## Maximum Message Size 122 | 123 | The maximum transmissible message size is 2GB due to a limitation with the maximum array size in Java, therefore [capability 3](https://code.kx.com/q/basics/ipc/#handshake) will be used within the kdb+ handshake. 124 | 125 | ## Type mapping 126 | 127 | Kdb+ types are mapped to and from Java types by this driver, and the example [`TypesMapping.java`](javakdb-examples/src/kx/examples/TypesMapping.java) demonstrates the construction of atoms, vectors, a dictionary, and a table, sending them to kdb+ for echo back to Java, for comparison with the original type and value. The output is recorded here for clarity: 128 | 129 | | Java type| kdb+ type| value sent| kdb+ value| 130 | |--------------------:|--------------------:|-------------------------------------:|-------------------------------------:| 131 | | [Ljava.lang.Object| (0) list| | | 132 | | java.lang.Boolean| (-1)boolean| true| 1b| 133 | | [Z| (1)boolean vector| true| ,1b| 134 | | java.util.UUID| (-2)guid| f5889a7d-7c4a-4068-9767-a009c8ac46ef| f5889a7d-7c4a-4068-9767-a009c8ac46ef| 135 | | [Ljava.util.UUID| (2)guid vector| f5889a7d-7c4a-4068-9767-a009c8ac46ef| ,f5889a7d-7c4a-4068-9767-a009c8ac46ef| 136 | | java.lang.Byte| (-4)byte| 42| 0x2a| 137 | | [B| (4)byte vector| 42| ,0x2a| 138 | | java.lang.Short| (-5)short| 42| 42h| 139 | | [S| (5)short vector| 42| ,42h| 140 | | java.lang.Integer| (-6)int| 42| 42i| 141 | | [I| (6)int vector| 42| ,42i| 142 | | java.lang.Long| (-7)long| 42| 42| 143 | | [J| (7)long vector| 42| ,42| 144 | | java.lang.Float| (-8)real| 42.42| 42.42e| 145 | | [F| (8)real vector| 42.42| ,42.42e| 146 | | java.lang.Double| (-9)float| 42.42| 42.42| 147 | | [D| (9)float vector| 42.42| ,42.42| 148 | | java.lang.Character| (-10)char| a| "a"| 149 | | [C| (10)char vector| a| ,"a"| 150 | | java.lang.String| (-11)symbol| 42| `42| 151 | | [Ljava.lang.String| (11)symbol vector| 42| ,`42| 152 | | java.time.Instant| (-12)timestamp| 2017-07-07 15:22:38.976| 2017.07.07D15:22:38.976000000| 153 | | [Ljava.time.Instant| (12)timestamp vector| 2017-07-07 15:22:38.976| ,2017.07.07D15:22:38.976000000| 154 | | kx.c\$Month| (-13)month| 2000-12| 2000.12m| 155 | | [Lkx.c\$Month| (13)month vector| 2000-12| ,2000.12m| 156 | | java.time.LocalDate| (-14)date| 2017-07-07| 2017.07.07| 157 | | [Ljava.time.LocalDate| (14)date vector| 2017-07-07| ,2017.07.07| 158 | | java.util.LocalDateTime| (-15)datetime| Fri Jul 07 15:22:38 GMT+03:00 2017| 2017.07.07T15:22:38.995| 159 | | [Ljava.util.LocalDateTime| (15)datetime vector| Fri Jul 07 15:22:38 GMT+03:00 2017| ,2017.07.07T15:22:38.995| 160 | | kx.c\$Timespan| (-16)timespan| 15:22:38.995000000| 0D15:22:38.995000000| 161 | | [Lkx.c\$Timespan| (16)timespan vector| 15:22:38.995000000| ,0D15:22:38.995000000| 162 | | kx.c\$Minute| (-17)minute| 12:22| 12:22| 163 | | [Lkx.c\$Minute| (17)minute vector| 12:22| ,12:22| 164 | | kx.c\$Second| (-18)second| 12:22:38| 12:22:38| 165 | | [Lkx.c\$Second| (18)second vector| 12:22:38| ,12:22:38| 166 | | java.time.LocalTime| (-19)time| 15:22:38| 15:22:38.995| 167 | | [Ljava.time.LocalTime| (19)time vector| 15:22:38| ,15:22:38.995| 168 | 169 | 170 | ## Timezone 171 | 172 | For global data capture, it is common practice to store events using a GMT timestamp. To minimize confusion, it is easiest to set the current timezone to GMT, either explicitly in the `c` class as 173 | 174 | ```java 175 | c.tz=TimeZone.getTimeZone("GMT"); 176 | ``` 177 | 178 | or from the environment, e.g. 179 | 180 | ```bash 181 | export TZ=GMT;... 182 | ``` 183 | 184 | otherwise kdb+ will use the default timezone from the environment, and adjust values between local and GMT during serialization. 185 | 186 | 187 | ## Message types 188 | 189 | There are three message types in kdb+ 190 | 191 | |Msg Type|Description| 192 | |--------|-----------| 193 | | async| send via `c.ks(…)`. This call blocks until the message has been fully sent. There is no guarantee that the server has processed this message by the time the call returns.| 194 | | sync| send via `c.k(…)`. This call blocks until a response message has been received, and returns the response which could be either data or an error.| 195 | |response| this should _only_ ever be sent as a response to a sync message. If your Java process is acting as a server, processing incoming sync messages, a response message can be sent with `c.kr(responseObject)`. If the response should indicate an error, use `c.ke("error string here")`.| 196 | 197 | If `c.k()` is called with no arguments, the call will block until a message is received of _any_ type. This is useful for subscribing to a tickerplant, to receive incoming async messages published by the ticker plant. 198 | 199 | 200 | ## Sending sync/async messages 201 | 202 | The methods for sending sync/async messages are overloaded as follows: 203 | 204 | - Methods which send async messages do not return a value: 205 | 206 | ```java 207 | public void ks(String s) throws IOException 208 | public void ks(String s, Object x) throws IOException 209 | public void ks(String s, Object x, Object y) throws IOException 210 | public void ks(String s, Object x, Object y, Object z) throws IOException 211 | ``` 212 | 213 | - Methods which send sync messages return an Object, the result from the remote processing the sync message: 214 | 215 | ```java 216 | public Object k(Object x) throws KException, IOException 217 | public Object k(String s) throws KException, IOException 218 | public Object k(String s, Object x) throws KException, IOException 219 | public Object k(String s, Object x, Object y) throws KException, IOException 220 | public Object k(String s, Object x, Object y, Object z) throws KException, IOException 221 | ``` 222 | - If no argument is given, the `k` call will block until a message is received, deserialized to an Object. 223 | 224 | ```java 225 | public Object k() throws KException, IOException 226 | ``` 227 | 228 | 229 | ## Exceptions 230 | 231 | The `c` class throws IOExceptions for network errors such as read/write failures and throws KExceptions for higher-level cases, such as remote execution errors arising during the query at hand. 232 | 233 | 234 | ## Accessing items of lists 235 | 236 | List items can be accessed using the `at` method of the utility class `c`: 237 | 238 | ```java 239 | Object c.at(Object x, int i) // Returns the object at x[i] or null 240 | ``` 241 | 242 | and set them with `set`: 243 | 244 | ```java 245 | void c.set(Object x, int i, Object y) // Set x[i] to y, or the appropriate q null value if y is null 246 | ``` 247 | 248 | 249 | ## Creating null values 250 | 251 | For each type suffix, "hijefcspmdznuvt", we can get a reference to a null q value by indexing into the `NULL` Object array using the `NULL` utility method. Note the q null values are not the same as Java’s null. 252 | 253 | An example of creating an object array containing a null integer and a null long: 254 | 255 | ```java 256 | Object[] twoNullIntegers = {NULL('i'), NULL('j')}; // i - int, j - long 257 | ``` 258 | 259 | 260 | ## Testing for null 261 | 262 | An object can be tested where it is a q null using the `c` utility method 263 | 264 | ```java 265 | public static boolean qn(Object x); 266 | ``` 267 | 268 | 269 | ## SSL/TLS 270 | 271 | Secure, encrypted connections may be established using SSL/TLS, by specifying the `useTLS` argument to the `c` constructor as true, e.g. 272 | 273 | ```java 274 | c c=new c("localhost",12345,System.getProperty("user.name"),true); 275 | ``` 276 | 277 | The kdb+ process [must be enabled](https://code.kx.com/q/kb/ssl) to accept TLS connections. 278 | 279 | Prior to using SSL/TLS, ensure that the required certificates and keys have been imported into your keystore. 280 | The Java [keytool](https://docs.oracle.com/javase/10/tools/keytool.htm) application provides the facility to manage certificates/keys/etc. 281 | 282 | Consult the [JSSE reference guide](https://docs.oracle.com/en/java/javase/11/security/java-secure-socket-extension-jsse-reference-guide.html) for details on configuring a Java application for SSL/TLS communication. 283 | 284 | ### Example 285 | 286 | This example uses the certificates and keys generated from the following [script](https://code.kx.com/q/kb/ssl/#checking-configuration). 287 | Passwords used are for example only. 288 | 289 | The private key (`client-private-key.pem`) and client certificate (`client-cert.pem`) can be converted to PKCS12 format using OpenSSL. 290 | The folling command is used to create a keystore file `keystore.p12`. When prompted for a password, we will use `kdbkdbpass`. 291 | ```bash 292 | openssl pkcs12 -export -inkey client-private-key.pem -in client-cert.pem -out keystore.p12 -name client-alias 293 | ``` 294 | 295 | Convert the CA certificate to a Java Truststore (JKS format). The example will use `ca-cert.pem` to generate the truststore file `truststore.jks` with a password `changeit`. 296 | ```bash 297 | keytool -importcert -trustcacerts -file ca-cert.pem -keystore truststore.jks -storepass changeit -alias ca-alias 298 | ``` 299 | 300 | When running the application, set the standard Java JSSE properties so the SSL/TLS connection has access to the keystores, for example: 301 | ```bash 302 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.TypesMapping" -Djavax.net.ssl.keyStore=keystore.p12 -Djavax.net.ssl.keyStorePassword=kdbkdbpass -D=javax.net.ssl.trustStore=truststore.jks -D=javax.net.ssl.trustStorePassword=changeit 303 | ``` 304 | 305 | To troubleshoot, supply `-Djavax.net.debug=ssl` on the command line when invoking your Java application. 306 | 307 | ## UDS (unix domain sockets) 308 | 309 | kdb+ can use UDS for comms, see [here](https://code.kx.com/q/basics/listening-port/#unix-domain-socket) for details. 310 | 311 | Java ipc requires java version 16 or greater, OS support & client/server residing on same machine. 312 | Java reference [here](https://inside.java/2021/02/03/jep380-unix-domain-sockets-channels/) 313 | 314 | example of client connection when kdb+ listening on 5010 315 | ```java 316 | c=new c("/tmp/kx.5010",System.getProperty("user.name")+":mypassword"); 317 | ``` 318 | 319 | example of creating server when kdb+ connecting with h:hopen`:unix://1234 320 | ```java 321 | java.net.UnixDomainSocketAddress address = java.net.UnixDomainSocketAddress.of("/tmp/kx.1234"); 322 | ServerSocketChannel serverChannel = ServerSocketChannel.open(java.net.StandardProtocolFamily.UNIX); 323 | serverChannel.bind(address); 324 | // pass serverChannel to c contructor to wait til new client connection occurs 325 | ``` 326 | 327 | 328 | 329 | -------------------------------------------------------------------------------- /docs/build.md: -------------------------------------------------------------------------------- 1 | # Building Java kdb+ ipc interface from source 2 | 3 | Java 1.8 (and above) is recommended. Please ensure that your `JAVA_HOME` environment variable is set to the version of Java you have installed (or the one preferred if you have multiple versions). 4 | 5 | You will also need [Apache Maven](https://maven.apache.org/) installed. Run the following to check you have it set up and configured correctly 6 | 7 | ```bash 8 | mvn -version 9 | ``` 10 | 11 | In order to build the library, run the following within the directory where the `pom.xml` file is located (from the downloaded source). 12 | 13 | ```bash 14 | mvn clean compile 15 | ``` 16 | 17 | If you wish to deploy the library to your machines local repository, in order to be used by other maven projects on your machine, run the following 18 | 19 | ```bash 20 | mvn clean install 21 | ``` 22 | 23 | Please refer to the [Apache Maven documentation](https://maven.apache.org/guides/index.html) for further details 24 | -------------------------------------------------------------------------------- /docs/examples.md: -------------------------------------------------------------------------------- 1 | # java ipc examples for kdb+ 2 | 3 | Supplied with the code is a series of code examples. The following describes each with an example of how to run from Maven (note: Maven is not required to run the applications, but used here for convenience). 4 | 5 | `mvn clean install` should be performed prior to running. 6 | 7 | ## GridViewer 8 | 9 | Creates a Swing GUI that presents the contents of a KDB+ table (Flip). It shows the mapping of the Flip class to a Swing TableModel. The contents of the table are some random data that we instruct KDB+ to generate. 10 | 11 | Prerequisite: 12 | 13 | - a kdb+ server running on port 5010 on your machine i.e. `q -p 5010` 14 | 15 | 16 | Run command: 17 | 18 | ```bash 19 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.GridViewer" 20 | ``` 21 | 22 | ## QueryResponse 23 | 24 | Instructs the remote kdb+ process to execute q code (kdb+ native language) and receives the result. The same principle can be used to execute q functions. Example of a sync request. 25 | 26 | Prerequisite: 27 | 28 | - a kdb+ server running on port 5010 on your machine i.e. `q -p 5010` 29 | 30 | 31 | Run command: 32 | 33 | ```bash 34 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.QueryResponse" 35 | ``` 36 | 37 | 38 | ## SerializationOnly 39 | 40 | Example of code that can be used to serialize/deserialize a Java type (array of ints) to kdb+ format. 41 | 42 | Run command: 43 | 44 | ```bash 45 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.SerializationOnly" 46 | ``` 47 | 48 | ## Server 49 | 50 | Creates a Java app that listens on TCP port 5010, which a kdb+ process can communicate with. It will echo back sync messages and discard async messages. The following is an example of running kdb+ from the same machine (i.e. running the q executable and typing commands at the q prompt) that will communicate with the Java server. 51 | 52 | ```q 53 | q 54 | q)h:hopen `::5010 55 | q)h"hello" 56 | q)neg[h]"hello" 57 | ``` 58 | 59 | Run command: 60 | 61 | ```bash 62 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.Server" 63 | ``` 64 | 65 | ## Feed 66 | 67 | Example of creating an update function remotely (to capture table inserts), along with table creation and population of the table. 68 | Table population has an example of single-row inserts (lower latency) and bulk inserts (better throughput and resource utilization). 69 | 70 | Prerequisites: 71 | 72 | - a kdb+ server running on port 5010 on your machine i.e. `q -p 5010`. 73 | - as this example depends on a `.u.upd` function being defined and a table name `mytable` pre-existing, you may wish to run the following within the kdb+ server (in normal environments, these table and function definitions should be pre-created by your kdb+ admin). 74 | 75 | ```q 76 | q).u.upd:{[tbl;row] insert[tbl](row)} 77 | q)mytable:([]time:`timespan$();sym:`symbol$();price:`float$();size:`long$()) 78 | ``` 79 | 80 | 81 | Run command: 82 | 83 | ```bash 84 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.Feed" 85 | ``` 86 | 87 | ## TypesMapping 88 | 89 | Example app that creates each of the kdb+ types in Java, and communicates with kdb+ to check that the type has been correctly matched with its q type (kdb+ default language). Prints the Java type and corresponding q type. 90 | 91 | Prerequisite: 92 | 93 | - a kdb+ server running on port 5010 on your machine i.e. `q -p 5010`. 94 | 95 | 96 | Run command: 97 | 98 | ```bash 99 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.TypesMapping" 100 | ``` 101 | 102 | ## Subscriber 103 | 104 | Example app that subscribes to real-time updates from a table that is maintained in kdb+. 105 | 106 | Prerequisite: 107 | 108 | - a kdb+ server running on port 5010 on your machine. The instance must have the `.u.sub` function defined. An example of `.u.sub` can be found in [KxSystems/kdb-tick](https://github.com/KxSystems/kdb-tick), an example tickerplant. You can execute this tickerplant process by running `q tick.q` (the default port is set to 5010). 109 | 110 | Run command: 111 | 112 | ```bash 113 | mvn exec:java -pl javakdb-examples -Dexec.mainClass="com.kx.examples.Subscriber" 114 | ``` 115 | 116 | -------------------------------------------------------------------------------- /docs/img/image1.emf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KxSystems/javakdb/acfe75a25c88580244119fed68230661095d306e/docs/img/image1.emf -------------------------------------------------------------------------------- /docs/img/image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KxSystems/javakdb/acfe75a25c88580244119fed68230661095d306e/docs/img/image2.png -------------------------------------------------------------------------------- /docs/java.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KxSystems/javakdb/acfe75a25c88580244119fed68230661095d306e/docs/java.jpg -------------------------------------------------------------------------------- /docs/whitepaper.md: -------------------------------------------------------------------------------- 1 | # Java API for kdb+ 2 | 3 | by [Peter Lyness](#author) 4 | 5 | > **Abstract** 6 | > 7 | > Illustrates how the Java API for kdb+ can be used to enable a Java program to interact with a kdb+ process. 8 | 9 | *Originally appeared in 2018 as a KX whitepaper.* 10 | 11 | 12 | The Java programming language has been consistently popular for two decades, and is important in many development environments. Its longevity, and the compatibility of code between versions and operating systems, leaves the landscape of Java applications in many industries very much divided between new offerings and long-established legacy code. 13 | 14 | Financial technology is no exception. 15 | Competition in this risk-averse domain drives it to push against boundaries. 16 | Production systems inevitably mix contemporary and legacy code. 17 | Because of this, developers need tools for communication and integration. 18 | Implementation risks must be kept to a strict minimum. 19 | KX technology is well-equipped for this issue. 20 | By design kdb+’s communication with external processes is kept simple, and reinforced with interface libraries for other languages. 21 | 22 | The Java API for kdb+ is a Java library. 23 | It fits easily in any Java application as an interface to kdb+ processes. 24 | As with any API, potential use cases are many. 25 | To introduce kdb+ gradually into a wider system, such an interface is essential for any interaction with Java processes, upstream or downstream. 26 | The straightforward implementation keeps changes to legacy code lightweight, reducing the risk of wider system issues arising as kdb+ processes are introduced. 27 | 28 | This paper illustrates how the Java API for kdb+ can be used to enable a Java program to interact with a kdb+ process. 29 | It first explores the API itself: how it is structured, and how it might 30 | be included in a development project. 31 | Examples are then provided for core use cases for the API in a standard setup. 32 | Particular consideration is given to how the API facilitates subscription and publication to a kdb+ tickerplant process, a core component of any kdb+ tick-capture system. 33 | 34 | The examples presented here form a set of practical templates complementary to the [primary source of information](../README.md). 35 | These templates can be combined and adapted to apply kdb+ across a 36 | broad range of problem domains. 37 | 38 | 39 | ## API overview 40 | 41 | The API is contained in a [single source file](https://github.com/KxSystems/javakdb/blob/master/javakdb/src/main/java/com/kx/c.java). 42 | Inclusion in a development project is, therefore, a straightforward matter 43 | of including the file with other source code under the package `kx`, and 44 | ensuring it is properly imported and referenced by other classes. If 45 | preferred, it can be compiled separately into a class or JAR file to be 46 | included in the classpath for use as an external library or uploaded to 47 | a local repository for build integration. 48 | 49 | As the API is provided as source, it is perfectly possible to customize code to meet specific requirements. 50 | However, without prior knowledge of how the interactions work, this is not advised unless the solution to these requirements or issues are known. 51 | It is also possible, and in some contexts encouraged, to wrap the 52 | functionality of this class within a model suitable for your framework. 53 | An example might be the open-source [qJava library](https://github.com/exxeleron/qJava). 54 | Although it is not compatible with the most recent kdb+ version at the time of writing, it shows how to use `c.java` as a core over which an object-oriented framework of q types and functionality has been applied. 55 | 56 | The source file is structured as a single outer class, `c`. 57 | Within it, a number of constants and inner classes together model an 58 | environment for sending and receiving data from a kdb+ process. 59 | This section explores the fundamentals of the class to provide context and understanding of practical use-cases for the API. 60 | 61 | 62 | ### Connection and interface logic 63 | 64 | The highly-recommended means of connecting to a kdb+ process using the API is through instantiation of the `c` object itself. 65 | Three constructors provide for this purpose: 66 | ```java 67 | public c(String host,int port,String usernamepassword) 68 | public c(String host,int port,String usernamepassword,boolean useTLS) 69 | public c(String host,int port) 70 | ``` 71 | These constructors are straightforward to use. 72 | The host and port specify a socket-object connection, with the username/password string serialized and passed to the remote instance for authorization. 73 | The core logic is the same for all; the host/port-only constructor attempts to retrieve the user string from the Java properties, and the constructor with the `useTLS` boolean will, when flagged true, attempt to use an SSL socket instead of an ordinary socket. 74 | 75 | It is also possible to set up the object to accept incoming connections 76 | from kdb+ processes rather than just making them. There are two 77 | constructors which, when passed a server socket reference, will allow a 78 | q session to establish a handle against the `c` object: 79 | ```java 80 | public c(ServerSocket s) 81 | public c(ServerSocket s,IAuthenticate a) 82 | ``` 83 | `IAuthenticate` is an interface within the `c` class that can be 84 | implemented to emulate kdb+ server-side authentication, allowing the 85 | establishment of authentication rules similar to that which might be 86 | done through the q function [`.z.pw`](https://code.kx.com/q/ref/dotz#zpw-validate-user). 87 | 88 | Both of these constructor families represent two ‘modes’ in which 89 | the `c` object can be instantiated. The first, and ultimately most 90 | widely used, is for making connections to kdb+ processes, which 91 | naturally would be used for queries, subscriptions and any task that 92 | requires the reception of or sending of data to said processes. The 93 | second, which sees Java act as the server, would see utility in 94 | management and aggregation of kdb+ clients, perhaps as a data sink or 95 | an intermediary interface for another technology. 96 | 97 | Interactions between Java and kdb+ through these connections are 98 | largely handled by what might be called the ‘k’ family of methods in 99 | the `c` class. There are thirteen combined methods and overloads that 100 | fall under this group. They can be divided roughly into four groups: 101 | 102 | 103 | ### Synchronous query methods 104 | 105 | ```java 106 | public Object k(String expr) 107 | public Object k(String s,Object x) 108 | public Object k(String s,Object x,Object y) 109 | public void k(String s,Object x,Object y,Object z) 110 | public synchronized Object k(Object x) 111 | ``` 112 | These methods are responsible for handling synchronous queries to a kdb+ 113 | process. The String parameter will represent either the entire q 114 | expression or the function name; in the case of the latter, the Object 115 | parameters may be used to pass values into that function. In all 116 | instances, the String/Object combinations are merged into a single 117 | object to be passed to the synchronized `k(Object)` method. 118 | 119 | 120 | ### Asynchronous query methods 121 | 122 | ```java 123 | public void ks(String expr) 124 | public void ks(String s,Object x) 125 | public void ks(String s,Object x,Object y) 126 | public void ks(String s,Object x,Object y,Object z) 127 | public void ks(Object obj) 128 | ``` 129 | These methods are responsible for handling asynchronous queries to a 130 | kdb+ process. They operate logically in a similar manner to the 131 | synchronous query method, with the exception that they are, of course, 132 | void methods in that they neither wait for nor return any response from 133 | the process. 134 | 135 | 136 | ### Incoming message method 137 | 138 | ```java 139 | public Object k() 140 | ``` 141 | This method waits on the class input stream and will deserialize the 142 | next incoming kdb+ message. It is used by the `c` synchronous methods in 143 | order to capture and return response objects, and is also used in 144 | server-oriented applications in order to capture incoming messages from 145 | client processes. 146 | 147 | 148 | ### Response message methods 149 | 150 | ```java 151 | public void kr(Object obj) 152 | public void ke(String text) 153 | ``` 154 | These methods are typically used in server-oriented applications to 155 | serialize and write response messages to the class output stream. 156 | `kr(Object)` will act much like any synchronous response, while `ke(String)` 157 | will format and output an error message. 158 | 159 | The use of these constructors and methods will be treated in more 160 | practical detail through the use-case examples below. 161 | 162 | 163 | ## Models and type mapping 164 | 165 | The majority of q data types are represented in the API through mapping 166 | to standard Java objects. This is best seen in the method 167 | `c.r()`, 168 | which reads bytes from an incoming message and converts those bytes into 169 | representative Java types. 170 | 171 | 172 | ### Basic types 173 | 174 | The method `c.r()` deserializes a stream of bytes within a certain range to point 175 | to further methods which return the appropriate typed object. These are 176 | largely self-explanatory, such as booleans and integer primitives 177 | mapping directly to one another, or q UUIDs mapping to `java.util.UUID`. 178 | There are some types with caveats, however: 179 | 180 | - The kdb+ float type (9) corresponds to `java.lang.Double` and _not_ `java.lang.Float`, which corresponds to the kdb+ real type (8). 181 | 182 | - Java strings map to the kdb+ symbol type (11). In terms of reading 183 | or passing in data, this means that passing `"String"` from Java to 184 | kdb would result in `` `String``. Conversely, passing `"String"` (type 10 185 | list) from kdb to Java would result in a six-index character array. 186 | 187 | 188 | ### Time-based types 189 | 190 | Of particular interest is how the mapping handles temporal types, of 191 | which there are eight: 192 | 193 | q type | id | Java type | note 194 | -------|----|-----------|------ 195 | datetime | 15 | `java.util.Date` | This Java class stores times as milliseconds passed since the Unix epoch. Therefore, like the q datetime, it can represent time information accurate to the millisecond. (This despite the default output format of the class). 196 | date | 14 | java.sql.Date | While this Java class extends the `java.util` date object it is used specifically for the date type as it restricts usage and output of time data. 197 | time | 19 | `java.sql.Time` | This also extends `java.util.Date`, restricting usage and output of date data this time. 198 | timestamp | 12 | `java.sql.Timestamp` | This comes yet again from the base date class, extended this time to include nanoseconds storage (which is done separately from the underlying date object, which only has millisecond accuracy). This makes it directly compatible with the q timestamp type. 199 | month | 13 | inner class `c.Month` | 200 | timespan | 16 | inner class `c.Timespan`| 201 | minute | 17 | inner class `c.Minute` | 202 | second | 18 | inner class `c.Second` | 203 | 204 | When manipulating date, time and datetime data from kdb+ it is important 205 | to note that while `java.sql.Date` and `Time` extend `java.util.Date`, and can 206 | be assigned to a `java.util` reference, that many of the methods from the 207 | original date class are overridden in these to throw exceptions if 208 | invoked. For example, in order to create a single date object for two 209 | separate SQL Date and Time objects, a `java.util.Date` object should be 210 | instantiated by adding the `getTime()` values from both SQL objects: 211 | ```java 212 | //Date value = datetime - time 213 | java.sql.Date sqlDate = (java.sql.Date)qconn.k(".z.d"); 214 | // Time value - datetime - date 215 | java.sql.Time sqlTime = (java.sql.Time)qconn.k(".z.t"); 216 | java.util.Date utilDate= new java.util.Date(sqlDate.getTime()+sqlTime.getTime()); 217 | ``` 218 | The four time types represented by inner classes are somewhat less 219 | prevalent than those modeled by Date and its subclasses. These classes 220 | exist as comparable models due to a lack of a clear representative 221 | counterpart in the standard Java library, although their modeling is for 222 | the large part fairly simple and the values can be easily implemented or 223 | extracted. 224 | 225 | 226 | ### Dictionaries and tables 227 | 228 | Kdb+ dictionaries (type 99) and tables (type 98) are represented by the 229 | internal classes Dict and Flip respectively. The makeup of these models 230 | is simple but effective, and useful in determining how best to 231 | manipulate them. 232 | 233 | [The Dict class](../javakdb/src/main/java/kx/c.java#L427) 234 | consists of two public `java.lang.Object` fields (`x` for keys, `y` for 235 | values) and a basic constructor, which allows any of the represented 236 | data types to be used. However, while from a Java perspective any object 237 | could be passed to the constructor, dictionaries in q are always 238 | structured as two lists. This means that if the object is being created 239 | to pass to a q session directly, the Object fields in a Dict object 240 | should be assigned arrays of a given representative type, as passing in 241 | an atomic object will result in an error. 242 | 243 | For example, the first of the following dictionary instantiation is 244 | legal with regards to the Java object, but because the pairs being 245 | passed in are atomic, it would signal a type error in q. Instead, the 246 | second example should be used, and can be seen as mirroring the practice 247 | of enlisting single values in q: 248 | ```java 249 | new c.Dict("Key","Value"); // not q-compatible 250 | new c.Dict(new String[] {"Key"}, new String[] {"Value"}); // q-compatible 251 | ``` 252 | As the logical extension of that, in order to represent a list as a 253 | single key or pair, multi-dimensional arrays should be used: 254 | ```java 255 | new c.Dict(new String[] {"Key"}, new String[][] {{"Value1","Value2","Value3"}}); 256 | ``` 257 | Flip (table) objects 258 | consist of a String array for columns, an Object array for values, a 259 | constructor and a method for returning the Object array for a given 260 | column. The constructor takes a dictionary as its parameter, which is 261 | useful for the conversion of one to the other should the dictionary in 262 | question consist of single symbol keys. Of course, with the fields of 263 | the class being public, the columns and values can be assigned manually. 264 | 265 | Keyed tables in q are dictionaries in terms of type, and therefore will 266 | be represented as a Dict object in Java. The method 267 | `td(Object)` 268 | will create a Flip object from a keyed table Dict, but will remove its 269 | keyed nature in the process. 270 | 271 | 272 | ### GUID 273 | 274 | The globally unique identifier (GUID) type was introduced into kdb+ with 275 | version 3.0 for the purpose of storing arbitrary 16-byte values, such as 276 | transaction IDs. Storing such values in this form allows for savings in 277 | tasks such as memory and storage usage, as well as improved performance 278 | in certain operations such as table lookups when compared with standard 279 | types such as Strings. 280 | 281 | Java has its own unique identifier type: `java.util.UUID` (universally 282 | unique identifier). In the API the kdb+ GUID type maps directly to this 283 | object through the extraction and provision of its most and least 284 | significant long values. Otherwise, the only high-level difference in 285 | how this type can be used when compared to other types handled by the 286 | API is that a `RuntimeException` will be thrown if an attempt is made to 287 | serialize and pass a UUID object to a kdb+ instance with a version lower 288 | than 3.0. 289 | 290 | More information on these identifier types can be found in the [KX documentation](https://code.kx.com/q/basics/datatypes#guid) as well as the 291 | [core Java documentation](https://docs.oracle.com/javase/7/docs/api/java/util/UUID.html). 292 | 293 | 294 | ### Null types 295 | 296 | Definitions for q null type representations in Java are held in the 297 | static Object array `NULL`, with index positions representing the q type. 298 | ```java 299 | public static Object[] NULL={ 300 | null, 301 | new Boolean(false), 302 | new UUID(0,0), 303 | null, 304 | new Byte((byte)0), 305 | new Short(Short.MIN_VALUE), 306 | new Integer(ni), 307 | new Long(nj), 308 | new Float(nf), 309 | new Double(nf), 310 | new Character(' '), 311 | "", 312 | new Timestamp(nj), 313 | new Month(ni) 314 | ,new Date(nj), 315 | new java.util.Date(nj), 316 | new Timespan(nj), 317 | new Minute(ni), 318 | new Second(ni), 319 | new Time(nj) 320 | }; 321 | ``` 322 | Of note are the integer types, as the null values for these are 323 | represented by the minimum possible value of each of the Java 324 | primitives. Shorts, for example, have a minimum value of -372768 in 325 | Java, but a minimum value of -372767 in q. The extra negative value in 326 | Java can therefore be used to signal a null value to the q connection 327 | logic in the `c` class. 328 | 329 | Float and real nulls are both represented in Java by the 330 | `java.lang.Double.NaN` constant. Time values, essentially being longs 331 | under the bonnet, are represented by the same null value as longs in 332 | Java. Month, minute, second and timespan, each with custom model 333 | classes, use the same null value as ints. 334 | 335 | The method 336 | `c.qn(Object)` 337 | can assist with checking and identifying null value representations, as 338 | it will check both the `Object` type and value against the `NULL` list. 339 | 340 | It is worth noting that infinity types are not explicitly mapped in 341 | Java, although kdb+ float and real infinities will correspond with the 342 | infinity constants in `java.lang.Double` and `java.lang.Float` 343 | respectively. 344 | 345 | 346 | ### Exceptions 347 | 348 | `KException` 349 | is the single custom exception defined and thrown by the API. It is 350 | fairly safe to assume that a thrown `KException` denotes a q error signal, 351 | which will be included in the exception message when thrown. 352 | 353 | Other common exceptions thrown in the API logic include: 354 | 355 | exception | origin 356 | ----------|------- 357 | `IOException` | Denotes issues with connecting to the kdb+ process. It is also thrown by `c.java` itself for such issues as authentication. 358 | `RuntimeException`| Thrown when certain type implementations are attempted on kdb+ versions prior to their introduction (such as the GUIDs prior to kdb+ 3.0) 359 | `UnsupportedEncodingException` | It is possible, through the method `setEncoding`, to specify character encoding different to the default (`ISO-859-1`). This exception will be thrown commonly if the default is changed to a charset format not implemented on the target Java platform. 360 | 361 | 362 | ## Practical use-case examples 363 | 364 | The examples that follow consist of common practical tasks that a Java 365 | developer might be expected to carry out when interfacing with kdb+. The 366 | inline examples take the form of extracted sections of key logic 367 | and output, and are available as example classes from 368 | this repository for use as starting points or templates. 369 | 370 | These examples assume, at minimum, a standard installation of 32-bit 371 | kdb+ on the local system, and a suitable Java development environment. 372 | 373 | 374 | ## Connecting to a kdb+ process 375 | 376 | 377 | ### Starting a local q server 378 | 379 | During development, it can be helpful to start a basic q server to which 380 | a Java process can connect. This requires the opening of a port, for 381 | which there are two basic methods: 382 | 383 | Example: Starting q with `–p` parameter 384 | ```bash 385 | q -p 10000 386 | ``` 387 | ```q 388 | q)\p // command to show the port that q is listening on 389 | 10000i 390 | ``` 391 | 392 | Example: Using the `\p` system command 393 | ```bash 394 | q 395 | ``` 396 | ```q 397 | q)\p 10000 // set the listening port to 10000 398 | q)\p 399 | 10000i 400 | ``` 401 | To close the port, it should be set to its default value of 0 i.e. `\p 0`. 402 | 403 | Setting up a q session in this manner will allow other processes to open 404 | handles to it on the specified port. The remainder of the examples in 405 | this paper assume an opened q session listening on port 10000, with 406 | no further configuration unless otherwise specified. 407 | 408 | 409 | ### Opening a socket connection 410 | 411 | As discussed in the previous section, the `c` class establishes 412 | connections via its constructors. 413 | 414 | For connecting to a listening q process, one useful mechanism might be 415 | to create a factory class with a method that returns a connected `c` 416 | object based on what is passed to it. This way, any number of credential 417 | combinations can be set whilst allowing the creation of multiple 418 | connections, say for reconnection purposes: 419 | 420 | 421 | Example: `QConnectionFactory.java` 422 | ```java 423 | public QConnectionFactory(String host, int port, 424 | String username, String password, boolean useTLS) { 425 | this.host=host; 426 | this.port=port; 427 | this.username=username; 428 | this.password=password; 429 | this.useTLS=useTLS; 430 | } 431 | 432 | //[…] 433 | 434 | public c getQConnection() throws KException, IOException { 435 | return new c(host,port,username+":"+password,useTLS); 436 | } 437 | ``` 438 | These constructors will always return a `c` object connected to the target 439 | session, and failure to do so will result in a thrown exception; 440 | `IOException` will denote the port not being open or available, and a 441 | `KException` will denote something wrong with the q process itself (such 442 | as `'access` for incorrect or incomplete credentials). 443 | 444 | For the remaining examples, connections will be made using a custom 445 | `QConnectionFactory` object returned from a static method `getDefault()`, 446 | which will instantiate the object with the host `localhost` and the port 447 | 10000: 448 | 449 | Example: `QConnectionFactory.java` 450 | ```java 451 | public static QConnectionFactory getDefault() { 452 | return new QConnectionFactory("localhost", 10000); 453 | } 454 | ``` 455 | Connection objects created using this will be given the variable name 456 | `qConnection` unless otherwise stated. 457 | 458 | 459 | ### Running queries using k methods 460 | 461 | Queries can be made using the ‘k’ family of methods in the `c` class. 462 | For synchronous queries, that might be used to retrieve data (or, more 463 | generally, to halt execution of the java process until a response 464 | is received), the k methods with parameter combinations of strings 465 | and objects might be used. 466 | For asynchronous queries, as might be used in a 467 | feed-handler process to push data to a tickerplant, the `ks` methods would 468 | be used. 469 | 470 | The methods `k()`, `kr()` and `ke()` would not see explicit use in the 471 | querying of a server q process, but are more significant when the Java 472 | process acts as the server, as will be touched upon below. 473 | 474 | The following examples demonstrate some of the means by which these 475 | synchronous and asynchronous queries may be called: 476 | 477 | 478 | Example: `SimpleQueryExamples.java` 479 | ```java 480 | //Object for storing the results of these queries 481 | Object result = null; 482 | 483 | //Basic synchronous q expression 484 | result = qConnection.k("{x+y}\[4;3\]"); 485 | System.out.println(result.toString()); 486 | 487 | //parameterized synchronous query 488 | result = qConnection.k("{x+y}",4,3); //Note autoboxing! 489 | System.out.println(result.toString()); 490 | 491 | //asynchronous assignment of function 492 | qConnection.ks("jFunc:{x-y+z}"); 493 | 494 | //synchronous calling of that function 495 | result = qConnection.k("jFunc",10,4,3); 496 | System.out.println(result); 497 | 498 | //asynchronous error - note no exception can be returned, so be careful! 499 | qConnection.ks("{x+y}\[4;3;2\]"); 500 | 501 | //Always close resources\! 502 | qConnection.close(); 503 | ``` 504 | 505 | ## Extracting data from returned objects 506 | 507 | 508 | ### Note on internal variables and casting 509 | 510 | The relationship between the kdb+ types and their Java counterparts has 511 | been discussed in the previous section. From a practical perspective, it 512 | is important to note that almost all objects and fields that might 513 | return from a given synchronous query will be of type Object, and will 514 | therefore more often than not require casting in order to be manipulated 515 | properly. Care must be taken, therefore, to ensure that the types that 516 | can be returned from a given query are known and handled appropriately 517 | so as to avoid unwanted exceptions. 518 | 519 | The exception to this might be the column names of a `flip` object (once 520 | cast itself) held in the field `flip.x`. This field is already typed as 521 | `String[]`, as column names must always be symbols in q. 522 | 523 | Kdb+ types that map to primitives (such as int) can be passed in Java to 524 | a `k` method as a primitive thanks to 525 | [autoboxing](https://docs.oracle.com/javase/tutorial/java/data/autoboxing.html), 526 | but will always be returned as the corresponding wrapper object (such as 527 | Integer). 528 | 529 | 530 | ### Extracting atoms from a list 531 | 532 | Lists will always be returned as an array of the given list type, or as 533 | `Object[]` if the list is generic. Extraction of atomic values from a 534 | list, therefore, is as simple as casting the return object to the 535 | appropriate array type and accessing the desired index: 536 | 537 | 538 | Example: `ExtractionExamples.java` 539 | ```java 540 | //Get a list from the q session 541 | Object result = qConnection.k("(1 2 3 4)"); 542 | 543 | //Cast the returned Object into long[], and retrieve the desired result. 544 | long[] castList = ((long[]) result); 545 | long extractedAtom = castList[0]; 546 | System.out.println(extractedAtom); 547 | ``` 548 | If the type of list is unknown, the method `c.t(Object)` can be used to 549 | derive the q type of the object, and theoretically could be useful in 550 | further casting efforts. 551 | 552 | 553 | ### Extracting lists from a nested list 554 | 555 | Accessing a list from a nested list is similar to accessing a value from 556 | any list. Here there are two casts required: a cast to `Object[]` for 557 | the parent list and then again to the appropriate typed array for the 558 | extracted list: 559 | 560 | 561 | Example: `ExtractionExamples.java` 562 | ```java 563 | // Start by casting the returned Object into Object[] 564 | Object[] resultArray = (Object[]) qConnection.k("((1 2 3 4); (1 2))"); 565 | 566 | //Iterate through the Object array 567 | for (Object resultElement : resultArray) { 568 | 569 | //Retrieve each list and cast to appropriate type 570 | long[] elementArray = (long[]) resultElement; 571 | 572 | //Iterate through these arrays to access values. 573 | for(long elementAtom : elementArray) { 574 | System.out.println(elementAtom); 575 | } 576 | } 577 | ``` 578 | 579 | 580 | ### Working with dictionaries 581 | 582 | The Dict inner class is used for all returned objects of q type 583 | dictionary (and therefore, by extension, keyed tables). Key values are 584 | stored in the field `Dict.x`, and values in `Dict.y`, both of which will 585 | generally be castable as an array. 586 | 587 | Aside from matching the index positions of `x` and `y`, there is no 588 | intrinsic key-value pairing between the two, meaning that alteration of 589 | either of the array structures can compromise the key-value 590 | relationship. The following example illustrates operations that might be 591 | performed on a returned dictionary object: 592 | 593 | Example: `ExtractionExamples.java` 594 | ```java 595 | //Retrieve Dictionary 596 | c.Dict dict = (c.Dict) qConnection.k("`a`b`c!((1 2 3);\"Second\"; (`x`y`z))"); 597 | //Retrieve keys from dictionary 598 | String[] keys = (String[]) dict.x; 599 | System.out.println(Arrays.toString(keys)); 600 | //Retrieve values 601 | Object[] values = (Object[]) dict.y; 602 | //These can then be worked with similarly to nested lists 603 | long[] valuesLong = (long[]) values[0]; 604 | //[…] 605 | ``` 606 | 607 | 608 | ### Working with tables 609 | 610 | The inner class `c.Flip` used to represent tables operates in a similar 611 | manner to `c.Dict`. The primary difference, as previously mentioned, is 612 | that `Flip.x` is already typed as `String[]`, while `Flip.y` will still 613 | require casting. The following example shows how the data from a 614 | returned `Flip` object might be used to print the table to console: 615 | 616 | 617 | Example: `ExtractionExamples.java` 618 | ```java 619 | // (try to load trade.q first for this (create a table manually if not possible) 620 | qConnection.ks("system \"l trade.q\""); 621 | //Retrieve table 622 | c.Flip flip = (c.Flip) qConnection.k("select from trade where sym = `a"); 623 | 624 | //Retrieve columns and data 625 | String[] columnNames = flip.x; 626 | Object[] columnData = flip.y; 627 | //Extract row data into typed arrays 628 | java.sql.Timestamp[] time = (java.sql.Timestamp[]) columnData[0]; 629 | String[] sym = (String[]) columnData[1]; 630 | double[] price = (double[]) columnData[2]; 631 | int[] size = (int[]) columnData[3]; 632 | int rows = time.length; 633 | 634 | //Print the table now - columns first: 635 | for (String columnName : columnNames) 636 | { 637 | System.out.print(columnName + "\t\t\t"); 638 | } 639 | System.out.println("\n-----------------------------------------------------"); 640 | //Then rows: 641 | for (int i = 0; i < rows; i++) 642 | { 643 | System.out.print(time[i]+"\t"+sym[i]+"\t\t\t"+price[i]+"\t\t\t"+size[i]+"\n"); 644 | } 645 | ``` 646 | 647 | 648 | ## Creating and passing data objects 649 | 650 | 651 | When passing objects to q via the `c` class, there is less emphasis on how a given object is created. Rather, such an operation 652 | is subject to the common pitfalls associated with passing values to a q 653 | expression; those of type and rank. 654 | 655 | The k family of methods, regardless of its return protocol, will take 656 | either the String of a q expression or the String of a q operator or 657 | function, complemented by Object parameters. Given the nature of q as an 658 | interpreted language, all of these are serialized and sent to the q 659 | session with little regard for logical correctness. 660 | 661 | It is important, therefore, that any expressions passed to a query 662 | method are syntactically accurate and refer to variables that actually 663 | exist in the target session. It is also important that any passed 664 | objects are mapped to a relevant q type, and function within the context 665 | that they are sent. `KException` messages to look out for while 666 | implementing these operations are `'type` and `'rank`, as these will 667 | generally denote basic type and rank issues respectively. 668 | 669 | 670 | ### Creating and passing a simple list 671 | 672 | The following method might be applied to all direct type mappings in 673 | the API; for simple lists (lists in which all elements are of the same 674 | type), it is enough to pass a Java array of the appropriate type. 675 | 676 | The following example invokes the q `set` function, which allows for the 677 | passing of a variable name as well as an object with which the variable 678 | might be set: 679 | 680 | 681 | Example: `CreateAndSendExamples.java` 682 | ```java 683 | //Create typed array 684 | int[] simpleList = {10, 20, 30}; 685 | //Pass array to q using set function. 686 | qConnection.k("set", "simpleList", simpleList) 687 | ``` 688 | 689 | 690 | #### Creating and passing a mixed list 691 | 692 | Mixed lists should always be passed to kdb+ through an Object array, 693 | `Object[]`. This array may then hold any number of mapped types, 694 | including, if appropriate, other typed or Object arrays: 695 | 696 | 697 | Example: `CreateAndSendExamples.java` 698 | ```java 699 | //Create generic Object array. 700 | Object[] mixedList = {new String[] {"first", "second"}, new double[] {1.0, 2.0}}; 701 | //Pass to q in the same way as a simple list. 702 | qConnection.k("set", "mixedList", mixedList); 703 | ``` 704 | 705 | 706 | ### Creating and passing dictionaries 707 | 708 | `c.Dict` objects are instantiated by setting its `x` and `y` objects in the 709 | constructor, and these objects should always be arrays. Once created, 710 | the Dict can be passed to kdb+ like any other object: 711 | 712 | 713 | Example: `CreateAndSendExamples.java` 714 | ```java 715 | //Create keys and values 716 | Object[] keys = {"a", "b", "c"}; 717 | int[] values = {100, 200, 300}; 718 | //Set in dict constructor 719 | c.Dict dict = new c.Dict(keys, values); 720 | //Set in q session 721 | qConnection.k("set","dict",dict); 722 | ``` 723 | 724 | ### Creating and passing tables 725 | 726 | `c.Flip` objects are created slightly differently; it is best to 727 | instantiate these by passing a `c.Dict` object into the constructor. This 728 | is because tables are essentially collections of dictionaries in kdb+, 729 | and therefore using this constructor helps ensure that the Flip object 730 | is set up correctly. 731 | 732 | It is worth noting that for this method to work correctly, the passed 733 | Dict object must use String keys, as these will map into the Flip 734 | object’s typed `String[]` columns: 735 | 736 | Example: `CreateAndSendExamples.java` 737 | ```java 738 | //Create rows and columns 739 | int[] values = {1, 2, 3}; 740 | Object[] data = new Object[] {values}; 741 | String[] columnNames = new String[] {"column"}; 742 | //Wrap values in dictionary 743 | c.Dict dict = new c.Dict(columnNames, data); 744 | //Create table using dict 745 | c.Flip table = new c.Flip(dict); 746 | //Send to q using 'insert' method 747 | qConnection.ks("insert", "t1", table); 748 | ``` 749 | 750 | ### Creating and passing GUID objects 751 | 752 | Globally universal identifier objects are represented in Java by 753 | `java.util.UUID` objects, and are passed to kdb+ in an identical manner as 754 | other basic types. The Java object has a useful static method for 755 | generating random identifiers, which further streamlines this process 756 | and can see utility in some use cases where only a certain number of 757 | arbitrary identifiers are required: 758 | 759 | 760 | Example: `CreateAndSendExamples.java` 761 | ```java 762 | //Generate random UUID object 763 | java.util.UUID uuid = java.util.UUID.randomUUID(); 764 | System.out.println(uuid.toString()); 765 | 766 | //Pass object to q using set function 767 | qConnection.k("set","randomGUID",uuidj); 768 | System.out.println(qConnection.k("randomGUID").toString()); 769 | ``` 770 | Of course, it should be remembered that kdb+ version 3.0 or higher is 771 | required to work with GUIDs, and running the above code connected to an 772 | older version will cause a `RuntimeException` to be thrown. 773 | 774 | 775 | ## Reconnecting to a q process automatically 776 | 777 | Requirements will often dictate that while q processes will need to be 778 | bounced (such as for End-of-Day processing), that a Java process will 779 | need to be able to handle loss and reacquisition of said processes 780 | without being restarted itself. A simple example might be a graphical 781 | user interface, where the forced shutdown of the entire application due 782 | to a dropped connection, or the lack of ability to reconnect, would be 783 | very poor design indeed. 784 | 785 | Use of patterns such as factories can help with the task of setting up a 786 | reconnection mechanism, as it allows for the simple creation of a 787 | preconfigured object. For `c` Objects, given that they connect on 788 | instantiation, means that a connection can be re-established simply by 789 | calling the relevant factory method. 790 | 791 | In order to handle longer periods of potential downtime, either loops or 792 | recursion should be used. The danger with recursive methodology here is 793 | that, given an extended without a timeout limitation, there is a risk of 794 | overflowing the method-call stack, as each failed attempt will invoke a 795 | new method onto the stack. 796 | 797 | For mechanisms that may need to wait indefinitely, it might be 798 | considered safer to use an indefinite while-loop that makes use of catch 799 | blocks, continue and break statements. This averts the danger of 800 | `StackOverflowError` occurring and is easily modified to implement a 801 | maximum number of tries: 802 | 803 | Example: `ReconnectionExample.java` 804 | ```java 805 | //initiate reconnect loop (possibly within a catch block). 806 | while (true) { 807 | try { 808 | System.err.println("Connection failed - retrying.."); 809 | //Wait a bit before trying to reconnect 810 | Thread.sleep(5000); 811 | qConnection = qConnFactory.getQConnection(); 812 | System.out.println("Connection re-established! Resuming.."); 813 | //Exit loop 814 | break; 815 | } catch (IOException | KException e1) { 816 | //resume loop if it fails 817 | continue; 818 | } 819 | … 820 | } 821 | ``` 822 | 823 | 824 | ## Kdb+ tickerplant overview 825 | 826 | A kdb+ tickerplant is a q process specifically designed to handle 827 | incoming high-frequency data feeds from publishing process. Its primary 828 | responsibility is the management of subscription requests and the fast 829 | publication of data to subscribers. The following diagram illustrates a 830 | simple dataflow of a potential kdb+ tick system: 831 | 832 | ![Simple dataflow of a potential kdb+ tick system](img/image2.png) 833 | 834 | :point_right: 835 | [_Building Real-time Tick Subscribers_](https://code.kx.com/q/wp/rt-tick) regarding the above vanilla setup 836 | 837 | Of interest in this whitepaper are the Java publisher and subscriber processes. As the kdb+ tick system is very widely used, both of these kinds of processes are highly likely to come up in development tasks involving kdb+ interfacing. 838 | 839 | 840 | ### Test tickerplant and feedhandler setup 841 | 842 | To facilitate the testing of Java subscriber processes we can implement 843 | example q processes freely available in the KX repository. Simulation of 844 | a tickerplant can be achieved with 845 | [`tick.q`](https://github.com/KxSystems/kdb-tick/blob/master/tick.q); 846 | Trade data, using the trade schema defined in `sym.q`, can then be 847 | published to this tickerplant using the definition for the file `feed.q` 848 | given here: 849 | ```q 850 | // q feed.q / with a default port of 5010 and default timer of 1000 851 | // q feed.q -port 10000 / with a default timer of 1000 852 | // q feed.q -port 10000 -t 2000 853 | 854 | tph:hopen $[0=count .z.x;5010;"J"$first .Q.opt\[.z.x]`port] 855 | if[not system"t";system"t 1000"] 856 | 857 | publishTradeToTickerPlant:{ 858 | nRows:first 1?1+til 3; 859 | tph(".u.upd";`trade;(nRows#.z.N;nRows?`IBM`FB`GS`JPM;nRows?150.35;nRows?1000)); 860 | } 861 | 862 | .z.ts:{ 863 | publishTradeToTickerPlant[]; 864 | } 865 | ``` 866 | The tickerplant and feed handlers can then be started by executing the 867 | following commands consecutively: 868 | ```bash 869 | q tick.q sym -t 2000 870 | q feed.q 871 | ``` 872 | Once the feedhandler is publishing to the tickerplant, processes can 873 | connect to it in order either to publish or subscribe to it. 874 | 875 | It should be noted that in this example and below we are using a 876 | Java process to subscribe to a tickerplant being fed directly 877 | by a simulated feed. While we are doing this here in order to 878 | facilitate a simple example setup, in production this is not 879 | usually encouraged. Processes such as Java subscribers would generally 880 | connect to derivative kdb+ processes such as chained tickerplants (as in the above diagram), 881 | for which standard publishing and subscription logic should be 882 | the same as that covered here. 883 | 884 | 885 | ## Tickerplant subscription 886 | 887 | 888 | ### Extracting the table schema 889 | 890 | Typical subscriber processes are required to make an initial subscription request to the tickerplant in order to receive data. 891 | See the [publish and subscribe](https://code.kx.com/q/kb/publish-subscribe) Knowledge Base article for details. 892 | This request involves calling the `.u.sub` function with two 893 | parameters. The first parameter is the table name and the second is a 894 | list of symbols for subscription. (Specifying a backtick in any of the 895 | parameters means all tables and/or all symbols). 896 | 897 | 898 | Example: `TickSubscriberExample.java` 899 | ```java 900 | // Run sub function and store result 901 | Object[] response = (Object[]) qConnection.k(".u.sub[`trade;`]"); 902 | ``` 903 | If the `.u.sub` function is called synchronously, the tickerplant will 904 | return the table schema. If subscribing to one table, the returned 905 | object will be a generic Object array, with the table name in 906 | `object[0]` and a `c.Flip` representation of the schema in `object[1]`: 907 | 908 | 909 | Example: `TickSubscriberExample.java` 910 | ```java 911 | // first index is table name 912 | System.out.println("table name: " + response[0]); 913 | 914 | // second index is flip object 915 | c.Flip table = (c.Flip) response[1]; 916 | 917 | // Retrieve column names 918 | String[] columnNames = table.x; 919 | for (int i = 0; i < columnNames.length; i++) { 920 | System.out.printf("Column %d is named %s\n", i, columnNames[i]); 921 | } 922 | ``` 923 | If more than one table is being subscribed to, the returned object will 924 | be an Object array consisting of the above object arrays; therefore, in 925 | order to retrieve each individual Flip object, this should be iterated 926 | against: 927 | 928 | 929 | Example: `TickSubscriberExample.java` 930 | ```java 931 | // Run sub function and store result 932 | Object[] response = (Object[]) qConnection.k(".u.sub[`;`]"); 933 | 934 | // iterate through Object array 935 | for (Object tableObjectElement : response) { 936 | 937 | // From here, it is similar to the one-table schema extraction 938 | Object[] tableData = (Object[]) tableObjectElement; 939 | System.out.println("table name: " + tableData[0]); 940 | c.Flip table = (c.Flip) tableData[1]; 941 | String[] columnNames = table.x; 942 | for (int i = 0; i < columnNames.length; i++) { 943 | System.out.printf("Column %d is named %s\n", i, columnNames[i]); 944 | } 945 | } 946 | ``` 947 | 948 | 949 | ### Subscribing to a tickerplant data feed 950 | 951 | Upon calling `.u.sub` and retrieving the schema, the tickerplant process 952 | will start to publish data to the Java process. The data it sends can be 953 | retrieved through the parameter-free `k()` method, which will wait for a 954 | response and return an Object (a `c.Flip` of the passed data) on 955 | publication: 956 | 957 | 958 | Example: `TickSubscriberExample.java` 959 | ```java 960 | while (true) { 961 | 962 | //wait on k() 963 | Object response = qConnection.k(); 964 | 965 | if(response != null) { 966 | Object[] data = (Object[]) response; 967 | 968 | //Slightly different.. table is in data[2]\! 969 | c.Flip table = (c.Flip) data[2]; 970 | //[…] 971 | } 972 | } 973 | ``` 974 | With the data in this form, it can be manipulated in a number of 975 | meaningful ways. To iterate through the columns, `c.n` can be called on 976 | individual `flip.y` columns in order to provide a row count: 977 | 978 | 979 | Example: `TickSubscriberExample.java` 980 | ```java 981 | String[] columnNames = table.x; 982 | Object[] columnData = table.y; 983 | 984 | //Get row count for looping 985 | int rowCount = c.n(columnData[0]); 986 | 987 | //Print out the table! 988 | System.out.printf("%s\t\t\t%s\t%s\t%s\n", 989 | columnNames[0], columnNames[1], columnNames[2], columnNames[3]); 990 | System.out.println("--------------------------------------------"); 991 | for (int i = 0; i < rowCount; i++) { 992 | 993 | //[Printing logic] 994 | 995 | } 996 | ``` 997 | This mechanism might be then enveloped in an indefinite loop, such as a 998 | `while(true)` loop. Each iteration waits on the `k()` method returning 999 | published data, which will continue until one of the contributing 1000 | processes fails (at which point an exception is caught and handled 1001 | appropriately). 1002 | 1003 | 1004 | ## Tickerplant publishing 1005 | 1006 | Publishing data to a tickerplant is almost always a necessity for a kdb+ 1007 | feed-handler process. Java, as a common language of choice for 1008 | third-party API development (e.g. Reuters, Bloomberg, MarkIT), is a popular language for feedhandler development, within which `c.java` is 1009 | used to handle the asynchronous invocation of a publishing function. 1010 | 1011 | 1012 | ### Publishing rows 1013 | 1014 | In general, publishing values to a tickerplant will require an 1015 | asynchronous query much like the following: 1016 | ```java 1017 | qConnection.ks(".u.upd", "trade", data); //Where data is an Object[] 1018 | ``` 1019 | The parameters for this can be defined as follows: 1020 | 1021 | - The update function name (`.u.upd`) 1022 | 1023 | This is the function executed on the tickerplant which enables the data insertion. As per the norm with this API, this is passed as a string. 1024 | 1025 | - Table name 1026 | 1027 | A String representation of the name of the table that receives the data. 1028 | 1029 | - Data 1030 | 1031 | An Object that will form the row/s to be appended to the table. This parameter is typically passed as an object array, each index representing a table column. 1032 | 1033 | In order to publish a single row to a tickerplant, typed arrays 1034 | consisting of single values might be instantiated. These are then 1035 | encapsulated in an Object array and passed to the `ks` method: 1036 | 1037 | 1038 | Example: `TickPublisherExamples.java` 1039 | ```java 1040 | //Create typed arrays for holding data 1041 | String[] sym = new String[] {"IBM"}; 1042 | double[] bid = new double[] {100.25}; 1043 | double[] ask = new double[] {100.26}; 1044 | int[] bSize = new int[]{1000}; 1045 | int[] aSize = new int[]{1000}; 1046 | //Create Object[] for holding typed arrays 1047 | Object[] data = new Object[] {sym, bid, ask, bSize, aSize}; 1048 | //Call .u.upd asynchronously 1049 | qConnection.ks(".u.upd", "quote", data); 1050 | ``` 1051 | Publishing multiple rows is then just a case of increased length of each 1052 | of the typed arrays: 1053 | 1054 | 1055 | Example: `TickPublisherExamples.java` 1056 | ```java 1057 | String[] sym = new String[] {"IBM", "GE"}; 1058 | double[] bid = new double[] {100.25, 120.25}; 1059 | double[] ask = new double[] {100.26, 120.26}; 1060 | int[] bSize = new int[]{1000, 2000}; 1061 | int[] aSize = new int[]{1000, 2000}; 1062 | ``` 1063 | In order to maximize tickerplant throughput and efficiency, it is 1064 | generally recommended to publish multiple rows in one go. 1065 | 1066 | :point_right: 1067 | White paper [_Kdb+tick Profiling for Throughput Optimization_](https://code.kx.com/q/wp/tick-profiling). 1068 | 1069 | Care has to be taken here to ensure that all typed arrays maintain 1070 | the same length, as failure to do so will likely result in a kdb+ 1071 | type error. Such errors are especially troublesome when using 1072 | asynchronous methods, which will not return `KExceptions` in the same 1073 | manner as sync methods! It is also worth noting that the order of the 1074 | typed arrays within the object array should match that of the table 1075 | schema. 1076 | 1077 | 1078 | ### Adding a timespan column 1079 | 1080 | It is standard tickerplant functionality to append a timespan column to 1081 | each row received from a feed handler if not included with the data 1082 | passed, which is used to record when the data was received by the 1083 | tickerplant. It’s possible for the publisher to create the timespan 1084 | column to prevent the tickerplant from adding one: 1085 | 1086 | 1087 | Example: `TickPublisherExamples.java` 1088 | ```java 1089 | //Timespan can be added here 1090 | c.Timespan[] time = new c.Timespan[] {new c.Timespan()}; 1091 | String[] sym = new String[] {"GS"}; 1092 | double[] bid = new double[] {100.25}; 1093 | double[] ask = new double[] {100.26}; 1094 | int[] bSize = new int[]{1000}; 1095 | int[] aSize = new int[]{1000}; 1096 | //Timespan array is then added at beginning of Object array 1097 | Object[] data = new Object[] {time, sym, bid, ask, bSize, aSize}; 1098 | qConnection.ks(".u.upd", "quote", data); 1099 | ``` 1100 | This might be done, for example, to allow the feedhandler to define the 1101 | time differently than simply logging the time at which the tickerplant 1102 | receives the data. 1103 | 1104 | 1105 | ## Connecting from kdb+ to a Java process 1106 | 1107 | The examples thus far have emphasized interfacing between Java and kdb+ 1108 | very much from the perspective of a Java client connecting to a kdb+ 1109 | server, using the constructors relevant to this purpose. It is very much 1110 | possible to reverse these roles using the `c(Serversocket)` constructor, 1111 | which enables a Java process to listen for incoming kdb+ messages on the 1112 | specified port. 1113 | 1114 | While the use cases for this ‘server’ mode of operation are not as 1115 | common as they might be for ‘client’-mode connections, it is nevertheless 1116 | available to developers as a means of implementing communication between 1117 | Java and kdb+ processes. The following examples demonstrate the 1118 | basic mechanisms by which this can be done. 1119 | 1120 | 1121 | ### Handling a single connection 1122 | 1123 | To set this up, a `c` object is instantiated using the ‘server’ mode constructor. 1124 | This will listen to the incoming connection of a single kdb+ process: 1125 | 1126 | Example: `IncomingConnectionExample.java` 1127 | ```java 1128 | //Wait for incoming connection 1129 | System.out.println("Waiting for incoming connection on port 5001.."); 1130 | c incomingConnection = new c(new ServerSocket(5001)); 1131 | ``` 1132 | In a manner similar to tickerplant subscription, the method `k()` (without 1133 | parameters) can be used to wait on and listen to any connecting q 1134 | session. In this example, the object is retrieved in this fashion and 1135 | deciphered, either to return an error when passed the 1136 | symbol `` `returnError`` or to return a message describing what was sent: 1137 | 1138 | 1139 | Example: `IncomingConnectionExample.java` 1140 | ```java 1141 | while(true) { 1142 | //k() method will wait until the kdb+ process sends an object. 1143 | Object incoming = incomingConnection.k(); 1144 | try { 1145 | // check the incoming object and return something based on what it is 1146 | if (incoming instanceof String && ((String)incoming).equals("returnError")) { 1147 | incomingConnection.ke("ReturningError!"); 1148 | } else if(incoming.getClass().isArray()) { 1149 | // if list, use Arrays toString method 1150 | incomingConnection.kr("The incoming list values are: " + Arrays.toString((Object[])incoming)); 1151 | } else { 1152 | incomingConnection.kr(("The incoming message was: " + incoming.toString()).toCharArray()); 1153 | } 1154 | } catch(IOException | KException e) { 1155 | //return error responses too 1156 | incomingConnection.ke(e.getMessage()); 1157 | } 1158 | } 1159 | ``` 1160 | 1161 | 1162 | ### Handling multiple connections 1163 | 1164 | In the above example, the server `c` object is instantiated with a 1165 | new ServerSocket being created in its constructor. This is acceptable in 1166 | this instance because we cared only about the handling of one 1167 | connection. 1168 | 1169 | In general, ServerSocket objects should not be used in this manner, as 1170 | they are designed to handle more than a single incoming connection. 1171 | Instead, the ServerSocket should be passed as a reference. With the 1172 | addition of some simple threading, an application capable of handling 1173 | messages from multiple q sessions can be created: 1174 | 1175 | 1176 | Example: `IncomingConnectionsExample.java` 1177 | ```java 1178 | //Create server socket reference beforehand.. 1179 | ServerSocket serverSocket = new ServerSocket(5001); 1180 | 1181 | //Set up connection loop 1182 | while(true) { 1183 | //Create c object with reference to server socket 1184 | final c incomingConnection = new c(serverSocket); 1185 | 1186 | //Create thread for handling this connection 1187 | new Thread(new Runnable() { 1188 | @Override 1189 | public void run() { 1190 | while(true) { 1191 | //Logic in this loop is similar to single connection 1192 | //[...] 1193 | } 1194 | } 1195 | //Run thread and restart loop. 1196 | }).start(); 1197 | } 1198 | ``` 1199 | This will allow any number of connections to be established, with factors 1200 | such as connection limitation and load balancing left up to how the 1201 | process is implemented. As in any case where threading is used, take 1202 | care that such a method does not enable race conditions or concurrency 1203 | issues; if necessary, steps can be taken to reduce the risk of such 1204 | operations, such as synchronized blocks and methods. 1205 | 1206 | 1207 | ## Conclusion 1208 | 1209 | This document has covered a variety of topics concerning 1210 | the mechanics and application of the `c.java` interface for kdb+. Of the 1211 | workings and examples shown, the most common use case for this interface 1212 | will be connecting to a q process, executing queries and functions and 1213 | managing any result objects. However, this document has also displayed 1214 | the versatile nature of `c.java` as a tool, providing a handful of 1215 | solutions to a given problem and able to fulfill server as well as 1216 | client functions. 1217 | 1218 | The practical examples should also help demonstrate that tasks required 1219 | as part of a standard kdb+ toolset can be handled easily from the 1220 | perspective of both Java developers interfacing with kdb+ for the first 1221 | time, or kdb+ developers who are required to venture into Java 1222 | development, for example, to help complete development of a feed 1223 | handler. The benefit of such interfaces is felt keenly through the 1224 | common role of these developers in helping to reconcile longstanding 1225 | applications with contemporary technologies, often to the benefit of 1226 | both. 1227 | 1228 | 1229 | 1230 | ## Author 1231 | 1232 | **Peter Lyness** joined First Derivatives as a software engineer in 2015. Since then he has implemented a number of Java-based technical solutions for clients, including kdb+ interface logic for upstream static and real time data feeds. 1233 | 1234 | 1235 | -------------------------------------------------------------------------------- /javakdb-examples/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 4.0.0 6 | 7 | com.kx 8 | javakdb-examples 9 | 2.0 10 | jar 11 | 12 | 13 | com.kx.multi 14 | JavaKdbModules 15 | 2.0 16 | 17 | 18 | JavaKDBExamples 19 | KDB+ IPC interface examples for the Java programming language 20 | https://github.com/KxSystems/javakdb 21 | 22 | 23 | 24 | Apache License, Version 2.0 25 | https://www.apache.org/licenses/LICENSE-2.0 26 | repo 27 | 28 | 29 | 30 | 31 | 32 | Fusion Developer 33 | fusion@kx.com 34 | KX Systems 35 | http://www.kx.com 36 | 37 | 38 | 39 | 40 | scm:git:git://github.com/KxSystems/javakdb.git 41 | scm:git:ssh://github.com:KxSystems/javakdb.git 42 | https://github.com/KxSystems/javakdb/tree/master 43 | 44 | 45 | 46 | 47 | com.kx 48 | javakdb 49 | 2.0 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | org.apache.maven.plugins 59 | maven-compiler-plugin 60 | 3.1 61 | 62 | ${java.version} 63 | ${java.version} 64 | 65 | -Xlint:deprecation 66 | 67 | 68 | 69 | 70 | org.apache.maven.plugins 71 | maven-javadoc-plugin 72 | 3.2.0 73 | 74 | 8 75 | true 76 | 77 | 78 | 79 | attach-javadocs 80 | package 81 | 82 | jar 83 | 84 | 85 | 86 | 87 | 88 | org.apache.maven.plugins 89 | maven-source-plugin 90 | 3.2.1 91 | 92 | 93 | attach-sources 94 | 95 | jar 96 | 97 | 98 | 99 | 100 | 101 | maven-assembly-plugin 102 | 103 | 104 | package 105 | 106 | single 107 | 108 | 109 | 110 | 111 | 112 | jar-with-dependencies 113 | 114 | 115 | 116 | 117 | net.ju-n.maven.plugins 118 | checksum-maven-plugin 119 | 1.3 120 | 121 | 122 | checksum-maven-plugin-files 123 | package 124 | 125 | files 126 | 127 | 128 | 129 | 130 | 131 | 132 | ${project.build.directory} 133 | 134 | *.jar 135 | 136 | 137 | 138 | 139 | SHA-1 140 | MD5 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | release-sign-artifacts 149 | 150 | 151 | mvnRelease 152 | 153 | 154 | 155 | 156 | 157 | org.apache.maven.plugins 158 | maven-gpg-plugin 159 | 1.5 160 | 161 | 162 | sign-artifacts 163 | verify 164 | 165 | sign 166 | 167 | 168 | 169 | 170 | 171 | org.sonatype.central 172 | central-publishing-maven-plugin 173 | 0.4.0 174 | true 175 | 176 | central 177 | true 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | -------------------------------------------------------------------------------- /javakdb-examples/src/main/java/com/kx/examples/Feed.java: -------------------------------------------------------------------------------- 1 | package com.kx.examples; 2 | import com.kx.c; 3 | import java.security.SecureRandom; 4 | import java.util.logging.Logger; 5 | import java.util.logging.Level; 6 | /** 7 | * Example of creating an update function remotely (to capture table inserts), 8 | * along with table creation and population of the table. 9 | * Table population has an example of single row inserts (lower latency) and 10 | * bulk inserts (better throughput and resource utilization). 11 | */ 12 | public class Feed{ 13 | private static final Logger LOGGER = Logger.getLogger(Feed.class.getName()); 14 | private static final String QFUNC = ".u.upd"; 15 | private static final String TABLENAME = "mytable"; 16 | private Feed(){} 17 | /** 18 | * Example of 10 single row inserts to a table 19 | * @param kconn Connection that will be sent the inserts 20 | * @throws java.io.IOException when issue with KDB+ comms 21 | */ 22 | static void rowInserts(c kconn) throws java.io.IOException{ 23 | // Single row insert - not as efficient as bulk insert 24 | LOGGER.log(Level.INFO,"Populating 'mytable' on kdb server with 10 rows..."); 25 | for(int i=0;i<10;i++){ 26 | // Assumes a remote schema of mytable:([]time:`timespan$();sym:`symbol$();price:`float$();size:`long$()) 27 | Object[] row={new c.Timespan(),"SYMBOL",Double.valueOf(93.5),Long.valueOf(300L)}; 28 | kconn.ks(QFUNC,TABLENAME,row); 29 | } 30 | } 31 | 32 | /** 33 | * Example of bulk inserts to a table to improve throughput 34 | * @param kconn Connection that will be sent the inserts 35 | * @throws java.io.IOException when issue with KDB+ comms 36 | * @throws c.KException if request evaluation resulted in an error 37 | */ 38 | static void bulkInserts(c kconn) throws java.io.IOException, c.KException{ 39 | // Bulk row insert - more efficient 40 | String[]syms=new String[]{"ABC","DEF","GHI","JKL"}; // symbols to randomly choose from 41 | // Allocate one array per column 42 | c.Timespan[] time=new c.Timespan[10]; 43 | String[] sym=new String[10]; 44 | double[] price=new double[10]; 45 | long[] size=new long[10]; 46 | SecureRandom random = new SecureRandom(); 47 | // populate the arrays with sample data 48 | for(int i=0;i<10;i++){ 49 | time[i]=new c.Timespan(); 50 | sym[i]=syms[random.nextInt(syms.length)]; // choose a random symbol 51 | price[i]=i; 52 | size[i]=i*10L; 53 | } 54 | // Note that we don't need to supply a flip with columns names for .u.upd. 55 | // Just the column data in the correct order is sufficient. 56 | LOGGER.log(Level.INFO,"Populating 'mytable' with a 10 row bulk insert (without column names)..."); 57 | kconn.ks(QFUNC,TABLENAME,new Object[]{time,sym,price,size}); 58 | // if we did want to supply a flip, it can be done as 59 | LOGGER.log(Level.INFO,"Populating 'mytable' with a 10 row bulk insert (using Flip with column names)..."); 60 | kconn.ks(QFUNC,TABLENAME,new c.Flip(new c.Dict(new String[]{"time","sym","price","size"},new Object[]{time,sym,price,size}))); 61 | kconn.k(""); // sync chase ensures the remote has processed all msgs 62 | } 63 | 64 | /** 65 | * Run example tick feed 66 | * Requires a KDB+ server running on port 5010 on your machine i.e. q -p 5010 67 | * Depends on a .u.upd function being defined and a table name 'mytable' pre-existing 68 | * @param args not used 69 | */ 70 | public static void main(String[] args){ 71 | c c=null; 72 | try{ 73 | c=new c("localhost",5010,System.getProperty("user.name")+":mypassword"); 74 | rowInserts(c); 75 | bulkInserts(c); 76 | } 77 | catch(Exception e){ 78 | LOGGER.log(Level.SEVERE,e.toString()); 79 | }finally{ 80 | if(c!=null) 81 | try{c.close();}catch(java.io.IOException e){ 82 | // ingnore exception 83 | } 84 | } 85 | } 86 | } 87 | -------------------------------------------------------------------------------- /javakdb-examples/src/main/java/com/kx/examples/GridViewer.java: -------------------------------------------------------------------------------- 1 | package com.kx.examples; 2 | 3 | import java.awt.BorderLayout; 4 | import java.awt.Color; 5 | import java.io.IOException; 6 | import java.lang.reflect.Array; 7 | import java.util.logging.Level; 8 | import java.util.logging.Logger; 9 | import javax.swing.JFrame; 10 | import javax.swing.JScrollPane; 11 | import javax.swing.JTable; 12 | import javax.swing.table.AbstractTableModel; 13 | import com.kx.c; 14 | /** 15 | * Creates a Swing GUI that presents the contents of a KDB+ table (Flip). 16 | * It shows the mapping of the Flip class to a Swing TableModel. 17 | * The contents of the table are some random data that we instruct KDB+ to generate. 18 | */ 19 | public class GridViewer { 20 | private GridViewer(){} 21 | static class KxTableModel extends AbstractTableModel { 22 | /** kdb table to display in gui as result of query */ 23 | private c.Flip flip; 24 | void setFlip(c.Flip data) { 25 | this.flip = data; 26 | } 27 | 28 | public int getRowCount() { 29 | return Array.getLength(flip.y[0]); 30 | } 31 | 32 | public int getColumnCount() { 33 | return flip.y.length; 34 | } 35 | 36 | public Object getValueAt(int rowIndex, int columnIndex) { 37 | return c.at(flip.y[columnIndex], rowIndex); 38 | } 39 | 40 | @Override 41 | public String getColumnName(int columnIndex) { 42 | return flip.x[columnIndex]; 43 | } 44 | } 45 | 46 | /** 47 | * Creates a GUI to show contents of a table from KDB+ 48 | * Requires a KDB+ server running on port 5010 on your machine i.e. q -p 5010 49 | * @param args not used 50 | */ 51 | public static void main(String[] args) { 52 | KxTableModel model = new KxTableModel(); 53 | c c = null; 54 | try { 55 | c = new c("localhost", 5010,System.getProperty("user.name")+":mypassword"); 56 | String query="([]date:.z.D;time:.z.T;sym:10?`8;price:`float$10?500.0;size:10?100)"; 57 | model.setFlip((c.Flip) c.k(query)); 58 | JTable table = new JTable(model); 59 | table.setGridColor(Color.BLACK); 60 | String title = "kdb+ Example - "+model.getRowCount()+" Rows"; 61 | JFrame frame = new JFrame(title); 62 | frame.setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE); 63 | frame.getContentPane().add(new JScrollPane(table), BorderLayout.CENTER); 64 | frame.setSize(300, 300); 65 | frame.setVisible(true); 66 | } catch (Exception ex) { 67 | Logger.getLogger(GridViewer.class.getName()).log(Level.SEVERE, null, ex); 68 | } finally { 69 | if (c != null) { 70 | try { 71 | c.close(); 72 | } 73 | catch (IOException ex) { 74 | // ingnore exception 75 | } 76 | } 77 | } 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /javakdb-examples/src/main/java/com/kx/examples/QueryResponse.java: -------------------------------------------------------------------------------- 1 | package com.kx.examples; 2 | import java.util.logging.Logger; 3 | import java.util.logging.Level; 4 | import com.kx.c; 5 | /** 6 | * Instructs the remote KDB+ process to execute 'q' code (KDB+ native language) and 7 | * receives the result. The same principle can be used to execute q functions 8 | */ 9 | public class QueryResponse{ 10 | private static final Logger LOGGER = Logger.getLogger(QueryResponse.class.getName()); 11 | private QueryResponse(){} 12 | /** 13 | * Runs a calcuation on remote KDB+ server and prints result to console 14 | * Requires a KDB+ server running on port 5010 on your machine i.e. q -p 5010 15 | * @param args not used 16 | */ 17 | public static void main(String[] args){ 18 | c c=null; 19 | try{ 20 | c=new c("localhost",5010,System.getProperty("user.name")+":mypassword"); 21 | Object result=c.k("2+3"); 22 | LOGGER.log(Level.INFO,"result is {0}",result); 23 | }catch(Exception e){ 24 | LOGGER.log(Level.SEVERE,e.toString()); 25 | }finally{ 26 | try{if(c!=null)c.close();}catch(java.io.IOException e){ 27 | // ingnore exception 28 | } 29 | } 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /javakdb-examples/src/main/java/com/kx/examples/SerializationOnly.java: -------------------------------------------------------------------------------- 1 | package com.kx.examples; 2 | import java.util.Arrays; 3 | import java.util.logging.Level; 4 | import java.util.logging.Logger; 5 | import com.kx.c; 6 | /** 7 | * Example of code that can be used to serialize/dezerialise a Java type (array of ints) to KDB+ format. 8 | * @author charlie 9 | */ 10 | public class SerializationOnly{ 11 | private static final Logger LOGGER = Logger.getLogger(SerializationOnly.class.getName()); 12 | private SerializationOnly(){} 13 | /** 14 | * Runs program and prints whether serialization/deserialization of an 15 | * integer array match. 16 | * @param s not used 17 | */ 18 | public static void main(String[]s){ 19 | c c=new c(); 20 | int[]input=new int[50000]; 21 | for(int i=0;i0?" vector":""); 30 | else if(i==99) 31 | result+="dictionary"; 32 | else if(i==98) 33 | result+="table"; 34 | return result; 35 | } 36 | /** 37 | * Runs program to check Java types against q types on KDB+. 38 | * Prints results to console. 39 | * Requires KDB+ server running on port 5010 on your machine i.e. q -p 5010 40 | * @param args not used 41 | */ 42 | public static void main(String[] args){ 43 | c c=null; 44 | try{ 45 | c=new c("localhost",5010,System.getProperty("user.name")+":mypassword"); 46 | Object[]vectors=new Object[]{ 47 | new boolean[]{true}, 48 | new UUID[]{UUID.randomUUID()}, 49 | new byte[]{42}, 50 | new short[]{42}, 51 | new int[]{42}, 52 | new long[]{42}, 53 | new float[]{42.42f}, 54 | new double[]{42.42d}, 55 | new char[]{'a'}, 56 | new String[]{"42"}, 57 | new Instant[]{Instant.now().plusNanos(1)}, 58 | new c.Month[]{new c.Month(11)}, 59 | new LocalDate[]{LocalDate.now()}, 60 | new LocalDateTime[]{LocalDateTime.now().truncatedTo(ChronoUnit.MILLIS)}, // datetime 61 | new c.Timespan[]{new c.Timespan()}, 62 | new c.Minute[]{new c.Minute(LocalDateTime.now().getMinute())}, 63 | new c.Second[]{new c.Second(LocalDateTime.now().getSecond())}, 64 | new LocalTime[]{LocalTime.now().truncatedTo(ChronoUnit.MILLIS)}}; 65 | String format="|%21s|%21s|%38s|%38s|%5s|\n"; 66 | LOGGER.log(Level.INFO,"{0}",String.format(format,"Java Type","kdb+ Type","Value Sent","kdb+ Value","Match")); 67 | LOGGER.log(Level.INFO,"{0}",String.format(format,"","","","","").replace(' ', '-')); 68 | for(Object vector:vectors){ 69 | for(int i=0;i<2;i++){ 70 | boolean asArray=i>0; 71 | Object arg=asArray?vector:Array.get(vector,0); 72 | Object[]result=(Object[])c.k("{(-3!x;type x;x)}",arg); // returns a 3 element list of (stringified x; the type number of x; x) 73 | LOGGER.log(Level.INFO,"{0}",String.format(format, 74 | arg.getClass().toString().substring(6).replace(";",""), // strip leading "class " and trailing semi colon 75 | getKTypeAsString((short)result[1]), 76 | asArray?Array.get(arg,0):arg, 77 | new String((char[])result[0]), 78 | result[2].getClass().equals(arg.getClass()) 79 | && asArray?Arrays.deepEquals(new Object[]{result[2]},new Object[]{arg}):result[2].equals(arg))); 80 | } 81 | } 82 | LOGGER.log(Level.INFO,"{0}",String.format(format,"","","","","").replace(' ', '-')); 83 | Object result=c.k("{x}",vectors); 84 | LOGGER.log(Level.INFO,"List Roundtrip match: {0}",Arrays.deepEquals(new Object[]{result},new Object[]{vectors})); 85 | 86 | c.Dict dict=new c.Dict(new String[]{"Andrew","James"},new int[]{24,35}); 87 | result=c.k("{x}",dict); 88 | LOGGER.log(Level.INFO,"Dict Roundtrip match: {0}",(Arrays.deepEquals(new Object[]{dict.x},new Object[]{((c.Dict)result).x}) 89 | && Arrays.deepEquals(new Object[]{dict.y},new Object[]{((c.Dict)result).y}))); 90 | 91 | c.Flip flip=new c.Flip(new c.Dict(new String[]{"time","sym","price","volume"}, 92 | new Object[]{new c.Timespan[]{new c.Timespan(),new c.Timespan()}, 93 | new String[]{"ABC","DEF"}, 94 | new double[]{123.456,789.012}, 95 | new long[]{100,200} 96 | })); 97 | result=c.k("{x}",flip); 98 | LOGGER.log(Level.INFO,"Flip Roundtrip match: {0}",(Arrays.deepEquals(new Object[]{flip.x},new Object[]{((c.Flip)result).x}) 99 | && Arrays.deepEquals(flip.y,((c.Flip)result).y))); 100 | }catch(Exception e){ 101 | LOGGER.log(Level.SEVERE,e.toString()); 102 | }finally{ 103 | try{if(c!=null)c.close();}catch(java.io.IOException e){ 104 | // ingnore exception 105 | } 106 | } 107 | } 108 | } 109 | -------------------------------------------------------------------------------- /javakdb-examples/src/main/java/com/kx/examples/package-info.java: -------------------------------------------------------------------------------- 1 | /** 2 | * This package provides examples of KDB+ IPC and serialization/deserialization. 3 | */ 4 | package com.kx.examples; 5 | -------------------------------------------------------------------------------- /javakdb/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 4.0.0 6 | 7 | com.kx 8 | javakdb 9 | 2.0 10 | jar 11 | 12 | 13 | com.kx.multi 14 | JavaKdbModules 15 | 2.0 16 | 17 | 18 | JavaKDB 19 | KDB+ IPC interface for the Java programming language 20 | https://github.com/KxSystems/javakdb 21 | 22 | 23 | 24 | Apache License, Version 2.0 25 | https://www.apache.org/licenses/LICENSE-2.0 26 | repo 27 | 28 | 29 | 30 | 31 | 32 | Fusion Developer 33 | fusion@kx.com 34 | KX Systems 35 | http://www.kx.com 36 | 37 | 38 | 39 | 40 | scm:git:git://github.com/KxSystems/javakdb.git 41 | scm:git:ssh://github.com:KxSystems/javakdb.git 42 | https://github.com/KxSystems/javakdb/tree/master 43 | 44 | 45 | 46 | 47 | junit 48 | junit 49 | [4.13.1,) 50 | test 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | org.apache.maven.plugins 60 | maven-compiler-plugin 61 | 3.1 62 | 63 | ${java.version} 64 | ${java.version} 65 | 66 | -Xlint:deprecation 67 | 68 | 69 | 70 | 71 | org.apache.maven.plugins 72 | maven-javadoc-plugin 73 | 3.2.0 74 | 75 | 8 76 | true 77 | 78 | 79 | 80 | attach-javadocs 81 | package 82 | 83 | jar 84 | 85 | 86 | 87 | 88 | 89 | org.apache.maven.plugins 90 | maven-source-plugin 91 | 3.2.1 92 | 93 | 94 | attach-sources 95 | 96 | jar 97 | 98 | 99 | 100 | 101 | 102 | org.jacoco 103 | jacoco-maven-plugin 104 | ${jacoco.version} 105 | 106 | ${maven.test.skip} 107 | ${basedir}/target/coverage-reports/jacoco-unit.exec 108 | ${basedir}/target/coverage-reports/jacoco-unit.exec 109 | file 110 | false 111 | 112 | 113 | 114 | jacoco-initialize 115 | 116 | prepare-agent 117 | 118 | test-compile 119 | 120 | 121 | jacoco-site 122 | verify 123 | 124 | report 125 | 126 | 127 | 128 | 129 | 130 | net.ju-n.maven.plugins 131 | checksum-maven-plugin 132 | 1.3 133 | 134 | 135 | checksum-maven-plugin-files 136 | package 137 | 138 | files 139 | 140 | 141 | 142 | 143 | 144 | 145 | ${project.build.directory} 146 | 147 | *.jar 148 | 149 | 150 | 151 | 152 | SHA-1 153 | MD5 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | release-sign-artifacts 163 | 164 | 165 | mvnRelease 166 | 167 | 168 | 169 | 170 | 171 | org.apache.maven.plugins 172 | maven-gpg-plugin 173 | 1.5 174 | 175 | 176 | sign-artifacts 177 | verify 178 | 179 | sign 180 | 181 | 182 | 183 | 184 | 185 | org.sonatype.central 186 | central-publishing-maven-plugin 187 | 0.4.0 188 | true 189 | 190 | central 191 | true 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | -------------------------------------------------------------------------------- /javakdb/src/main/java/com/kx/package-info.java: -------------------------------------------------------------------------------- 1 | /** 2 | * This package provides KDB+ IPC and serialization/deserialization. 3 | */ 4 | package com.kx; 5 | -------------------------------------------------------------------------------- /javakdb/src/test/java/com/kx/CTest.java: -------------------------------------------------------------------------------- 1 | package com.kx; 2 | 3 | import static org.junit.Assert.assertTrue; 4 | 5 | import java.util.UUID; 6 | import java.util.Arrays; 7 | import java.time.LocalTime; 8 | import java.time.Instant; 9 | import java.time.LocalDate; 10 | import java.time.LocalDateTime; 11 | import java.time.ZoneId; 12 | import java.time.format.DateTimeFormatter; 13 | import org.junit.Test; 14 | import org.junit.Assert; 15 | 16 | /** 17 | * Unit test for c.java. 18 | */ 19 | public class CTest 20 | { 21 | @Test 22 | public void testGetNullValuesFromArray() 23 | { 24 | Assert.assertNull(c.NULL[0]); 25 | Assert.assertEquals(false,c.NULL[1]); 26 | Assert.assertEquals(new UUID(0,0),c.NULL[2]); 27 | Assert.assertNull(c.NULL[3]); 28 | Assert.assertEquals(Byte.valueOf((byte)0),c.NULL[4]); 29 | Assert.assertEquals(Short.MIN_VALUE,c.NULL[5]); 30 | Assert.assertEquals(Integer.MIN_VALUE,c.NULL[6]); 31 | Assert.assertEquals(Long.MIN_VALUE,c.NULL[7]); 32 | Assert.assertEquals(Float.valueOf((float)Double.NaN),c.NULL[8]); 33 | Assert.assertEquals(Double.NaN,c.NULL[9]); 34 | Assert.assertEquals(' ',c.NULL[10]); 35 | Assert.assertEquals("",c.NULL[11]); 36 | Assert.assertEquals(Instant.MIN,c.NULL[12]); 37 | Assert.assertEquals(new c.Month(Integer.MIN_VALUE),c.NULL[13]); 38 | Assert.assertEquals(LocalDate.MIN,c.NULL[14]); 39 | Assert.assertEquals(LocalDateTime.MIN,c.NULL[15]); 40 | Assert.assertEquals(new c.Timespan(Long.MIN_VALUE),c.NULL[16]); 41 | Assert.assertEquals(new c.Minute(Integer.MIN_VALUE),c.NULL[17]); 42 | Assert.assertEquals(new c.Second(Integer.MIN_VALUE),c.NULL[18]); 43 | Assert.assertEquals(com.kx.c.LOCAL_TIME_NULL,c.NULL[19]); 44 | } 45 | 46 | @Test 47 | public void testGetNullValues() 48 | { 49 | Assert.assertNull(c.NULL(' ')); 50 | Assert.assertEquals(false, c.NULL('b')); 51 | Assert.assertEquals(new UUID(0,0), c.NULL('g')); 52 | Assert.assertEquals(Byte.valueOf((byte)0), c.NULL('x')); 53 | Assert.assertEquals(Short.MIN_VALUE, c.NULL('h')); 54 | Assert.assertEquals(Integer.MIN_VALUE, c.NULL('i')); 55 | Assert.assertEquals(Long.MIN_VALUE, c.NULL('j')); 56 | Assert.assertEquals(Float.valueOf((float)Double.NaN), c.NULL('e')); 57 | Assert.assertEquals(Double.NaN, c.NULL('f')); 58 | Assert.assertEquals(' ', c.NULL('c')); 59 | Assert.assertEquals("", c.NULL('s')); 60 | Assert.assertEquals(Instant.MIN, c.NULL('p')); 61 | Assert.assertEquals(new c.Month(Integer.MIN_VALUE), c.NULL('m')); 62 | Assert.assertEquals(LocalDate.MIN, c.NULL('d')); 63 | Assert.assertEquals(LocalDateTime.MIN, c.NULL('z')); 64 | Assert.assertEquals(new c.Timespan(Long.MIN_VALUE), c.NULL('n')); 65 | Assert.assertEquals(new c.Minute(Integer.MIN_VALUE), c.NULL('u')); 66 | Assert.assertEquals(new c.Second(Integer.MIN_VALUE), c.NULL('v')); 67 | Assert.assertEquals(com.kx.c.LOCAL_TIME_NULL, c.NULL('t')); 68 | } 69 | 70 | @Test 71 | public void testIncorrectNullType() 72 | { 73 | try { 74 | c.NULL('a'); 75 | Assert.fail("Expected an ArrayIndexOutOfBoundsException to be thrown"); 76 | } catch (ArrayIndexOutOfBoundsException e) { 77 | // do nothing 78 | } 79 | } 80 | 81 | @Test 82 | public void testValueIsNull() 83 | { 84 | assertTrue( c.qn("") ); 85 | Assert.assertEquals(false, c.qn(" ")); 86 | assertTrue( c.qn(Instant.MIN)); 87 | assertTrue( c.qn(new c.Month(Integer.MIN_VALUE))); 88 | assertTrue( c.qn(LocalDate.MIN)); 89 | assertTrue( c.qn(LocalDateTime.MIN)); 90 | assertTrue( c.qn(new c.Timespan(Long.MIN_VALUE))); 91 | assertTrue( c.qn(new c.Minute(Integer.MIN_VALUE))); 92 | assertTrue( c.qn(new c.Second(Integer.MIN_VALUE))); 93 | assertTrue( c.qn(com.kx.c.LOCAL_TIME_NULL) ); 94 | assertTrue( c.qn(com.kx.c.NULL('g'))); 95 | } 96 | 97 | @Test 98 | public void testValueIsNotNull() 99 | { 100 | Assert.assertEquals(false, c.qn(" ")); 101 | Assert.assertEquals(false, c.qn(new StringBuffer())); 102 | } 103 | 104 | @Test 105 | public void testGetAtomType() 106 | { 107 | Assert.assertEquals(-1, c.t(Boolean.FALSE)); 108 | Assert.assertEquals(-2, c.t(new UUID(0,0))); 109 | Assert.assertEquals(-4, c.t(Byte.valueOf("1"))); 110 | Assert.assertEquals(-5, c.t(Short.valueOf("1"))); 111 | Assert.assertEquals(-6, c.t(Integer.valueOf(1111))); 112 | Assert.assertEquals(-7, c.t(Long.valueOf(1111))); 113 | Assert.assertEquals(-8, c.t(Float.valueOf(1.2f))); 114 | Assert.assertEquals(-9, c.t(Double.valueOf(1.2))); 115 | Assert.assertEquals(-10, c.t(Character.valueOf(' '))); 116 | Assert.assertEquals(-11, c.t("")); 117 | Assert.assertEquals(-14, c.t(LocalDate.MIN)); 118 | Assert.assertEquals(-19, c.t(com.kx.c.LOCAL_TIME_NULL)); 119 | Assert.assertEquals(-12, c.t(Instant.MIN)); 120 | Assert.assertEquals(-15, c.t(LocalDateTime.MIN)); 121 | Assert.assertEquals(-16, c.t(new c.Timespan(Long.MIN_VALUE))); 122 | Assert.assertEquals(-13, c.t(new c.Month(Integer.MIN_VALUE))); 123 | Assert.assertEquals(-17, c.t(new c.Minute(Integer.MIN_VALUE))); 124 | Assert.assertEquals(-18, c.t(new c.Second(Integer.MIN_VALUE))); 125 | } 126 | 127 | @Test 128 | public void testGetType() 129 | { 130 | Assert.assertEquals(1, c.t(new boolean[2])); 131 | Assert.assertEquals(2, c.t(new UUID[2])); 132 | Assert.assertEquals(4, c.t(new byte[2])); 133 | Assert.assertEquals(5, c.t(new short[2])); 134 | Assert.assertEquals(6, c.t(new int[2])); 135 | Assert.assertEquals(7, c.t(new long[2])); 136 | Assert.assertEquals(8, c.t(new float[2])); 137 | Assert.assertEquals(9, c.t(new double[2])); 138 | Assert.assertEquals(10, c.t(new char[2])); 139 | Assert.assertEquals(11, c.t(new String[2])); 140 | Assert.assertEquals(14, c.t(new LocalDate[2])); 141 | Assert.assertEquals(19, c.t(new LocalTime[2])); 142 | Assert.assertEquals(12, c.t(new Instant[2])); 143 | Assert.assertEquals(15, c.t(new LocalDateTime[2])); 144 | Assert.assertEquals(16, c.t(new c.Timespan[2])); 145 | Assert.assertEquals(13, c.t(new c.Month[2])); 146 | Assert.assertEquals(17, c.t(new c.Minute[2])); 147 | Assert.assertEquals(18, c.t(new c.Second[2])); 148 | c.Dict dict = new c.Dict(new String[] {"Key"}, new String[][] {{"Value1","Value2","Value3"}}); 149 | Assert.assertEquals(98, c.t(new c.Flip(dict))); 150 | Assert.assertEquals(99, c.t(dict)); 151 | } 152 | 153 | @Test 154 | public void testGetUnknownType() 155 | { 156 | Assert.assertEquals(0, c.t(new StringBuffer())); 157 | } 158 | 159 | @Test 160 | public void testDictConstructor() 161 | { 162 | String[] x = new String[] {"Key"}; 163 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 164 | c.Dict dict = new c.Dict(x, y); 165 | Assert.assertEquals(x, dict.x); 166 | Assert.assertEquals(y, dict.y); 167 | } 168 | 169 | @Test 170 | public void testFlipConstructor() 171 | { 172 | String[] x = new String[] {"Key"}; 173 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 174 | c.Dict dict = new c.Dict(x, y); 175 | c.Flip flip = new c.Flip(dict); 176 | Assert.assertArrayEquals(x, flip.x); 177 | Assert.assertArrayEquals(y, flip.y); 178 | } 179 | 180 | @Test 181 | public void testFlipColumnPosition() 182 | { 183 | String[] x = new String[] {"Key"}; 184 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 185 | c.Dict dict = new c.Dict(x, y); 186 | c.Flip flip = new c.Flip(dict); 187 | Assert.assertEquals(y[0], flip.at("Key")); 188 | } 189 | 190 | @Test 191 | public void testFlipRemoveKeyWithFlip() 192 | { 193 | try { 194 | String[] x = new String[] {"Key"}; 195 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 196 | c.Dict dict = new c.Dict(x, y); 197 | c.Flip flip = new c.Flip(dict); 198 | c.Flip newflip = c.td(flip); 199 | Assert.assertEquals(flip, newflip); 200 | } catch (Exception e) { 201 | Assert.fail(e.toString()); 202 | } 203 | 204 | try { 205 | String[] x = new String[] {"Key"}; 206 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 207 | c.Dict dict = new c.Dict(x, y); 208 | c.Flip flip = new c.Flip(dict); 209 | c.Dict dictOfFlips = new c.Dict(flip, flip); 210 | c.Flip newflip = c.td(dictOfFlips); 211 | Assert.assertArrayEquals(new String[] {"Key","Key"}, newflip.x); 212 | Assert.assertArrayEquals(new String[][] {{"Value1","Value2","Value3"},{"Value1","Value2","Value3"}}, newflip.y); 213 | } catch (Exception e) { 214 | Assert.fail(e.toString()); 215 | } 216 | } 217 | 218 | @Test 219 | public void testFlipUnknownColumn() 220 | { 221 | String[] x = new String[] {"Key"}; 222 | String[][] y = new String[][] {{"Value1","Value2","Value3"}}; 223 | c.Dict dict = new c.Dict(x, y); 224 | c.Flip flip = new c.Flip(dict); 225 | try { 226 | flip.at("RUBBISH"); 227 | Assert.fail("Expected an ArrayIndexOutOfBoundsException to be thrown"); 228 | } catch (ArrayIndexOutOfBoundsException e) { 229 | // do nothing 230 | } 231 | } 232 | 233 | @Test 234 | public void testSerializeDeserializeBool() 235 | { 236 | com.kx.c c=new com.kx.c(); 237 | Boolean input=Boolean.valueOf(true); 238 | try{ 239 | Assert.assertEquals(input,(Boolean)c.deserialize(c.serialize(1,input,false))); 240 | Assert.assertEquals(input,(Boolean)c.deserialize(c.serialize(1,input,true))); 241 | } catch (Exception e) { 242 | Assert.fail(e.toString()); 243 | } 244 | } 245 | 246 | @Test 247 | public void testSerializeDeserializeUUID() 248 | { 249 | com.kx.c c=new com.kx.c(); 250 | UUID input=new UUID(0,0); 251 | try{ 252 | Assert.assertEquals(input,(UUID)c.deserialize(c.serialize(1,input,false))); 253 | Assert.assertEquals(input,(UUID)c.deserialize(c.serialize(1,input,true))); 254 | } catch (Exception e) { 255 | Assert.fail(e.toString()); 256 | } 257 | c.ipcVersion=2; 258 | try{ 259 | c.serialize(1,input,false); 260 | Assert.fail("Expected a RuntimeException to be thrown"); 261 | } catch(RuntimeException e) { 262 | // expected 263 | } 264 | catch (Exception e) { 265 | Assert.fail(e.toString()); 266 | } 267 | } 268 | 269 | @Test 270 | public void testSerializeDeserializeByte() 271 | { 272 | com.kx.c c=new com.kx.c(); 273 | Byte input=Byte.valueOf((byte)1); 274 | try{ 275 | Assert.assertEquals(input,(Byte)c.deserialize(c.serialize(1,input,false))); 276 | Assert.assertEquals(input,(Byte)c.deserialize(c.serialize(1,input,true))); 277 | } catch (Exception e) { 278 | Assert.fail(e.toString()); 279 | } 280 | } 281 | 282 | @Test 283 | public void testSerializeDeserializeShort() 284 | { 285 | com.kx.c c=new com.kx.c(); 286 | Short input=Short.valueOf((short)1); 287 | try{ 288 | Assert.assertEquals(input,(Short)c.deserialize(c.serialize(1,input,false))); 289 | Assert.assertEquals(input,(Short)c.deserialize(c.serialize(1,input,true))); 290 | } catch (Exception e) { 291 | Assert.fail(e.toString()); 292 | } 293 | } 294 | 295 | @Test 296 | public void testSerializeDeserializeInteger() 297 | { 298 | com.kx.c c=new com.kx.c(); 299 | Integer input=Integer.valueOf(77); 300 | try{ 301 | Assert.assertEquals(input,(Integer)c.deserialize(c.serialize(1,input,false))); 302 | Assert.assertEquals(input,(Integer)c.deserialize(c.serialize(1,input,true))); 303 | } catch (Exception e) { 304 | Assert.fail(e.toString()); 305 | } 306 | } 307 | 308 | @Test 309 | public void testSerializeDeserializeLong() 310 | { 311 | com.kx.c c=new com.kx.c(); 312 | Long input=Long.valueOf(77); 313 | try{ 314 | Assert.assertEquals(input,(Long)c.deserialize(c.serialize(1,input,false))); 315 | Assert.assertEquals(input,(Long)c.deserialize(c.serialize(1,input,true))); 316 | } catch (Exception e) { 317 | Assert.fail(e.toString()); 318 | } 319 | } 320 | 321 | @Test 322 | public void testSerializeDeserializeFloat() 323 | { 324 | com.kx.c c=new com.kx.c(); 325 | Float input=Float.valueOf((float)77.7); 326 | try{ 327 | Assert.assertEquals(input,(Float)c.deserialize(c.serialize(1,input,false))); 328 | Assert.assertEquals(input,(Float)c.deserialize(c.serialize(1,input,true))); 329 | } catch (Exception e) { 330 | Assert.fail(e.toString()); 331 | } 332 | } 333 | 334 | @Test 335 | public void testSerializeDeserializeDouble() 336 | { 337 | com.kx.c c=new com.kx.c(); 338 | Double input=Double.valueOf(77.7); 339 | try{ 340 | Assert.assertEquals(input,(Double)c.deserialize(c.serialize(1,input,false))); 341 | Assert.assertEquals(input,(Double)c.deserialize(c.serialize(1,input,true))); 342 | } catch (Exception e) { 343 | Assert.fail(e.toString()); 344 | } 345 | } 346 | 347 | @Test 348 | public void testSerializeDeserializeCharacter() 349 | { 350 | com.kx.c c=new com.kx.c(); 351 | Character input=Character.valueOf('a'); 352 | try{ 353 | Assert.assertEquals(input,(Character)c.deserialize(c.serialize(1,input,false))); 354 | Assert.assertEquals(input,(Character)c.deserialize(c.serialize(1,input,true))); 355 | } catch (Exception e) { 356 | Assert.fail(e.toString()); 357 | } 358 | } 359 | 360 | @Test 361 | public void testSerializeDeserializeString() 362 | { 363 | com.kx.c c=new com.kx.c(); 364 | String input=new String("hello"); 365 | try{ 366 | Assert.assertEquals(input,(String)c.deserialize(c.serialize(1,input,false))); 367 | Assert.assertEquals(input,(String)c.deserialize(c.serialize(1,input,true))); 368 | input=""; 369 | Assert.assertEquals(input,(String)c.deserialize(c.serialize(1,input,true))); 370 | com.kx.c.setEncoding("US-ASCII"); 371 | Assert.assertEquals(input,(String)c.deserialize(c.serialize(1,input,true))); 372 | } catch (Exception e) { 373 | Assert.fail(e.toString()); 374 | } 375 | } 376 | 377 | @Test 378 | public void testSerializeDeserializeLocalDate() 379 | { 380 | com.kx.c c=new com.kx.c(); 381 | LocalDate input=LocalDate.ofEpochDay(Integer.MAX_VALUE); 382 | try{ 383 | Assert.assertEquals(input,(LocalDate)c.deserialize(c.serialize(1,input,false))); 384 | Assert.assertEquals(input,(LocalDate)c.deserialize(c.serialize(1,input,true))); 385 | } catch (Exception e) { 386 | Assert.fail(e.toString()); 387 | } 388 | input=LocalDate.MIN; 389 | try{ 390 | Assert.assertEquals(input,(LocalDate)c.deserialize(c.serialize(1,input,false))); 391 | Assert.assertEquals(input,(LocalDate)c.deserialize(c.serialize(1,input,true))); 392 | } catch (Exception e) { 393 | Assert.fail(e.toString()); 394 | } 395 | try{ 396 | Assert.assertEquals(com.kx.c.NULL[14],(LocalDate)c.deserialize(c.serialize(1,com.kx.c.NULL[14],true))); 397 | } catch (Exception e) { 398 | Assert.fail(e.toString()); 399 | } 400 | input=LocalDate.ofEpochDay(Integer.MAX_VALUE+1); 401 | try{ 402 | c.serialize(1,input,true); 403 | Assert.fail("Expected a RuntimeException to be thrown"); 404 | } catch(RuntimeException e) { 405 | // expected 406 | } catch (Exception e) { 407 | Assert.fail(e.toString()); 408 | } 409 | input=LocalDate.ofEpochDay(Integer.MIN_VALUE-1L-com.kx.c.DAYS_BETWEEN_1970_2000); 410 | try{ 411 | c.serialize(1,input,true); 412 | Assert.fail("Expected a RuntimeException to be thrown"); 413 | } catch(RuntimeException e) { 414 | // expected 415 | } catch (Exception e) { 416 | Assert.fail(e.toString()); 417 | } 418 | } 419 | 420 | @Test 421 | public void testSerializeDeserializeTime() 422 | { 423 | com.kx.c c=new com.kx.c(); 424 | LocalTime input=LocalTime.of(12,10,1,1000000*5);; 425 | try{ 426 | Assert.assertEquals(input,(LocalTime)c.deserialize(c.serialize(1,input,false))); 427 | Assert.assertEquals(input,(LocalTime)c.deserialize(c.serialize(1,input,true))); 428 | } catch (Exception e) { 429 | Assert.fail(e.toString()); 430 | } 431 | input=com.kx.c.LOCAL_TIME_NULL; 432 | try{ 433 | Assert.assertEquals(input,(LocalTime)c.deserialize(c.serialize(1,input,false))); 434 | Assert.assertEquals(input,(LocalTime)c.deserialize(c.serialize(1,input,true))); 435 | } catch (Exception e) { 436 | Assert.fail(e.toString()); 437 | } 438 | } 439 | 440 | @Test 441 | public void testSerializeDeserializeInstant() 442 | { 443 | com.kx.c c=new com.kx.c(); 444 | Instant input=Instant.ofEpochMilli(55); 445 | try{ 446 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,false))); 447 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,true))); 448 | } catch (Exception e) { 449 | Assert.fail(e.toString()); 450 | } 451 | input=Instant.ofEpochMilli(86400000L*10957L+10); 452 | try{ 453 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,false))); 454 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,true))); 455 | } catch (Exception e) { 456 | Assert.fail(e.toString()); 457 | } 458 | input=Instant.MIN; 459 | try{ 460 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,false))); 461 | Assert.assertEquals(input,(Instant)c.deserialize(c.serialize(1,input,true))); 462 | } catch (Exception e) { 463 | Assert.fail(e.toString()); 464 | } 465 | c.ipcVersion=0; 466 | try{ 467 | c.serialize(1,input,false); 468 | Assert.fail("Expected a RuntimeException to be thrown"); 469 | } catch(RuntimeException e) { 470 | // expected 471 | } 472 | catch (Exception e) { 473 | Assert.fail(e.toString()); 474 | } 475 | } 476 | 477 | @Test 478 | public void testSerializeDeserializeUtilDate() 479 | { 480 | com.kx.c c=new com.kx.c(); 481 | try{ 482 | DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd/MM/yyyy"); 483 | LocalDateTime input = LocalDate.parse("01/01/1990", formatter).atStartOfDay(); 484 | Assert.assertEquals(input,(LocalDateTime)c.deserialize(c.serialize(1,input,false))); 485 | Assert.assertEquals(input,(LocalDateTime)c.deserialize(c.serialize(1,input,true))); 486 | Assert.assertEquals(com.kx.c.NULL[15],(LocalDateTime)c.deserialize(c.serialize(1,com.kx.c.NULL[15],true))); 487 | } catch (Exception e) { 488 | Assert.fail(e.toString()); 489 | } 490 | } 491 | 492 | @Test 493 | public void testSerializeDeserializeTimespan() 494 | { 495 | com.kx.c c=new com.kx.c(); 496 | com.kx.c.Timespan input=new com.kx.c.Timespan(java.util.TimeZone.getDefault()); 497 | try{ 498 | Assert.assertEquals(input,(com.kx.c.Timespan)c.deserialize(c.serialize(1,input,false))); 499 | Assert.assertEquals(input,(com.kx.c.Timespan)c.deserialize(c.serialize(1,input,true))); 500 | } catch (Exception e) { 501 | Assert.fail(e.toString()); 502 | } 503 | c.ipcVersion=0; 504 | try{ 505 | c.serialize(1,input,false); 506 | Assert.fail("Expected a RuntimeException to be thrown"); 507 | } catch(RuntimeException e) { 508 | // expected 509 | } 510 | catch (Exception e) { 511 | Assert.fail(e.toString()); 512 | } 513 | } 514 | 515 | @Test 516 | public void testSerializeDeserializeMonth() 517 | { 518 | com.kx.c c=new com.kx.c(); 519 | com.kx.c.Month input=new com.kx.c.Month(55); 520 | try{ 521 | Assert.assertEquals(input,(com.kx.c.Month)c.deserialize(c.serialize(1,input,false))); 522 | Assert.assertEquals(input,(com.kx.c.Month)c.deserialize(c.serialize(1,input,true))); 523 | } catch (Exception e) { 524 | Assert.fail(e.toString()); 525 | } 526 | } 527 | 528 | @Test 529 | public void testSerializeDeserializeMinute() 530 | { 531 | com.kx.c c=new com.kx.c(); 532 | com.kx.c.Minute input=new com.kx.c.Minute(55); 533 | try{ 534 | Assert.assertEquals(input,(com.kx.c.Minute)c.deserialize(c.serialize(1,input,false))); 535 | Assert.assertEquals(input,(com.kx.c.Minute)c.deserialize(c.serialize(1,input,true))); 536 | } catch (Exception e) { 537 | Assert.fail(e.toString()); 538 | } 539 | } 540 | 541 | @Test 542 | public void testSerializeDeserializeSecond() 543 | { 544 | com.kx.c c=new com.kx.c(); 545 | com.kx.c.Second input=new com.kx.c.Second(55); 546 | try{ 547 | Assert.assertEquals(input,(com.kx.c.Second)c.deserialize(c.serialize(1,input,false))); 548 | Assert.assertEquals(input,(com.kx.c.Second)c.deserialize(c.serialize(1,input,true))); 549 | } catch (Exception e) { 550 | Assert.fail(e.toString()); 551 | } 552 | } 553 | 554 | @Test 555 | public void testSerializeDeserializeObjectArray() 556 | { 557 | com.kx.c c=new com.kx.c(); 558 | Object[]input=new Object[2]; 559 | input[0]=Long.valueOf(77); 560 | input[1]=Integer.valueOf(22); 561 | try{ 562 | assertTrue(Arrays.equals(input,(Object[])c.deserialize(c.serialize(1,input,false)))); 563 | assertTrue(Arrays.equals(input,(Object[])c.deserialize(c.serialize(1,input,true)))); 564 | } catch (Exception e) { 565 | Assert.fail(e.toString()); 566 | } 567 | } 568 | 569 | @Test 570 | public void testSerializeDeserializeBoolArray() 571 | { 572 | com.kx.c c=new com.kx.c(); 573 | boolean[]input=new boolean[500]; 574 | try{ 575 | assertTrue(Arrays.equals(input,(boolean[])c.deserialize(c.serialize(1,input,false)))); 576 | assertTrue(Arrays.equals(input,(boolean[])c.deserialize(c.serialize(1,input,true)))); 577 | } catch (Exception e) { 578 | Assert.fail(e.toString()); 579 | } 580 | } 581 | 582 | @Test 583 | public void testSerializeDeserializeUUIDArray() 584 | { 585 | com.kx.c c=new com.kx.c(); 586 | UUID[]input=new UUID[500]; 587 | for(int i=0;i 2 | 6 | 4.0.0 7 | 8 | 9 | com.kx.multi 10 | JavaKdbModules 11 | pom 12 | 2.0 13 | 14 | JavaKDBModules 15 | KDB+ IPC interface modules for the Java programming language 16 | https://github.com/KxSystems/javakdb 17 | 18 | 19 | 20 | Apache License, Version 2.0 21 | https://www.apache.org/licenses/LICENSE-2.0 22 | repo 23 | 24 | 25 | 26 | 27 | 28 | Fusion Developer 29 | fusion@kx.com 30 | KX Systems 31 | http://www.kx.com 32 | 33 | 34 | 35 | 36 | scm:git:git://github.com/KxSystems/javakdb.git 37 | scm:git:ssh://github.com:KxSystems/javakdb.git 38 | https://github.com/KxSystems/javakdb/tree/master 39 | 40 | 41 | 42 | UTF-8 43 | 1.8 44 | ${java.version} 45 | ${java.version} 46 | 0.8.11 47 | jacoco 48 | reuseReports 49 | java 50 | 51 | 52 | 53 | 54 | javakdb 55 | javakdb-examples 56 | 57 | 58 | 59 | 60 | release-sign-artifacts 61 | 62 | 63 | mvnRelease 64 | 65 | 66 | 67 | 68 | 69 | org.apache.maven.plugins 70 | maven-gpg-plugin 71 | 1.5 72 | 73 | 74 | sign-artifacts 75 | verify 76 | 77 | sign 78 | 79 | 80 | 81 | 82 | 83 | org.sonatype.central 84 | central-publishing-maven-plugin 85 | 0.4.0 86 | true 87 | 88 | central 89 | true 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | --------------------------------------------------------------------------------