├── .miss_hit ├── AUTHORS.txt ├── ChangeLog.txt ├── LICENSE_BSD-3.txt ├── LICENSE_GPLv3.txt ├── README.md ├── decodevarname.m ├── encodevarname.m ├── examples └── demo_eazyh5_basic.m ├── jdatadecode.m ├── jdataencode.m ├── jsonopt.m ├── loadh5.m ├── mergestruct.m ├── regrouph5.m ├── saveh5.m ├── transposemat.m └── varargin2struct.m /.miss_hit: -------------------------------------------------------------------------------- 1 | project_root 2 | 3 | suppress_rule: "redundant_brackets" 4 | suppress_rule: "line_length" 5 | suppress_rule: "file_length" 6 | suppress_rule: "copyright_notice" 7 | suppress_rule: "naming_functions" 8 | suppress_rule: "naming_parameters" 9 | suppress_rule: "naming_scripts" 10 | suppress_rule: "unicode" 11 | indent_function_file_body: false 12 | -------------------------------------------------------------------------------- /AUTHORS.txt: -------------------------------------------------------------------------------- 1 | EasyH5 is written by Qianqian Fang as part of the 2 | NeuroJSON project (https://neurojson.org). 3 | 4 | Qianqian is currently an Associate Professor in the 5 | Department of Bioengineering at Northeastern University. 6 | 7 | Address: Dept. of Bioengineering 8 | Northeastern University 9 | 360 Huntington Ave, ISEC 206, Boston, MA 02115, USA 10 | URL: http://fanglab.org 11 | Email: 12 | 13 | NeuroJSON Website : https://neurojson.org 14 | 15 | -------------------------------------------------------------------------------- /ChangeLog.txt: -------------------------------------------------------------------------------- 1 | = Change Log = 2 | 3 | Major updates are marked with a "*" 4 | 5 | == EasyH5 v0.9 (Daseot - Korean 5), FangQ == 6 | 7 | 2025-03-16*[4baab54] [feat] initial support of saveh5 using oct-hdf5 in octave 8 | 2025-03-16*[a2b62c1] [feat] support Octave low-level HDF5 IO via oct-hdf5 9 | 2025-03-15 [1fe742d] [test] pass test on MATLAB R2010b 10 | 2025-03-14 [39b3b07] [jsonlab] sync jsonlab with the latest version, update doc 11 | 2025-03-13 [e04da0c] [snirf] fix fNIRS/snirf-samples#13 12 | 2024-09-15 [2090320] [snirf] allow saving empty 1d vector as timeseries 13 | 2024-07-09 [4b4444f] [bug] fix #16, timeseries() can handle complex 1D vector 14 | 2024-07-09 [6f145c0] [snirf] fix probe.sourceLabels with timeseries(string({})), fNIRS/snirf-samples#8 15 | 2024-07-08*[df03a9c] [feature] use timeseries datatype for true 1d vector, fix #16, fNIRS/snirf-samples#12 16 | 2024-07-07 [01e5e53] [bug] allow to store true 1D vector with ndim=1, fix #16, fNIRS/snirf-samples#12 17 | 2023-11-26 [a88c2e1] scalar and variablelengthstring can not use with deflate on 18 | 2023-11-26 [322b798] reverse forcedim to fix transposed array after roundtrip 19 | 2023-11-26 [1c4ad33] sync units from jsonlab 20 | 2023-11-26*[b6d102f] automatic format with miss_hit 21 | 2023-11-26 [ffd5870] merge scalar and stringarray changes 22 | 2023-11-26 [2d38eda] save as scalar instead of length of 1 array 23 | 2023-11-26 [830bf24] support octave to load hdf5 file 24 | 2023-11-25 [3c8dc14] trim null ending string, transpose string array, octave support 25 | 2023-02-12 [e7fc01f] saving and loading string and cell char arrays, fix NeuroJSON/jsnirfy#1 26 | 2022-10-01 [172915b] fix sparse matrix saving error, close #14 27 | 2022-09-30 [96132b7] fix rootname issue, a regression due to #11, close #13 28 | 2022-08-10 [74a430c] Update README.md 29 | 2022-08-05*[c1d30f4] can write variable length string via VariableLengthString flag 30 | 2022-05-18 [c985103] upgrade all jsonlab files to the latest version, bump to v0.9.0 31 | 2022-05-18 [72deacf] allow appending data under 'rootname' path with 'append',1, fix #11, works in MATLAB R2010b 32 | 2022-05-18 [a11cc77] set default name, allow saving expressions to /data 33 | 2022-04-06 [6ab811a] clear opt to avoid error 34 | 2020-05-22 [94b7364] avoid crash in matlab R2014 or earlier when handling attributes 35 | 2020-05-15 [dfd1c49] transpose array by default both when saving and loading, fix #10 36 | 2020-05-14 [ef1ccdc] fix errors of saving empty matrices on matlab 2014 & older (hdf5 1.8.6) 37 | 2020-05-12 [9a1b459] fix regression for complex sparse matices due to #4 38 | 2020-05-05 [2951909] add parentheses for test condition, #8 39 | 2020-05-05 [600c2d1] fall back to alphabetic order on matlab 2014 and earlier, fix #8 40 | 2019-12-14 [e42564d] customizable complex numbers storage 41 | 2019-10-25 [1d45bc3] use jdata as input flag 42 | 2019-10-25*[a00940e] add jdataencode and jdatadecode functions 43 | 44 | == EasyH5 v0.8 (Go - Japanese 5), FangQ == 45 | 46 | 2019-10-01 [7c76642] rename to eazyh5 to avoid spelling confusions-already got questions 47 | 2019-09-30*[104a9ed] add regroup option for loadh5, change regexp, add demo 48 | 2019-09-30*[bb762a8] support saving and restoring non-ascii group and dataset names, like JSONLab 49 | 2019-09-29*[1486603] restore the original position for a grouped item 50 | 2019-09-29*[d420b3d] support data compression, close #2 51 | 2019-09-29*[04f12ee] transpose data when saving not loading, fix sparse array loading bug 52 | 2019-09-28*[77e0d47] support saving and restoring sparse array, both real and complex, close #3 53 | 2019-09-28 [f05e305] add helper functions copied from jsonlab 54 | 2019-09-28 [2e385ae] collapse a single numbered group with in the form of ...1 55 | 2019-09-23*[5e694b8] apply both order tracked and indexed when writing datasets 56 | 2019-09-22 [289d2b6] update readme 57 | 2019-09-22*[b09c80f] now reading data in creation order, fix #1, also reads specified node using rootpath 58 | 59 | == EasyH5 v0.5 (Cinco - Spanish 5), FangQ == 60 | 61 | 2019-09-19 [dc62ed5] update code name and version number 62 | 2019-09-19 [66de6e2] tracking creation order, need to use links to read in loadh5 63 | 2019-09-19*[69979e4] initial but fully working version 64 | -------------------------------------------------------------------------------- /LICENSE_BSD-3.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019,2022,2025, Qianqian Fang 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | * Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 10 | * Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution 13 | 14 | * Neither the name of Northeastern University nor the names of its 15 | contributors may be used to endorse or promote products derived from this 16 | software without specific prior written permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE 22 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 24 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 25 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 26 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | -------------------------------------------------------------------------------- /LICENSE_GPLv3.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019,2022,2025 Qianqian Fang 2 | 3 | ------------------------------------------------------------------------------- 4 | 5 | This program is free software: you can redistribute it and/or modify 6 | it under the terms of the GNU General Public License as published by 7 | the Free Software Foundation, either version 3 of the License, or 8 | (at your option) any later version. 9 | 10 | This program is distributed in the hope that it will be useful, 11 | but WITHOUT ANY WARRANTY; without even the implied warranty of 12 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 13 | GNU General Public License for more details. 14 | 15 | You should have received a copy of the GNU General Public License 16 | along with this program. If not, see . 17 | 18 | ------------------------------------------------------------------------ 19 | 20 | GNU GENERAL PUBLIC LICENSE 21 | Version 3, 29 June 2007 22 | 23 | Copyright (C) 2007 Free Software Foundation, Inc. 24 | Everyone is permitted to copy and distribute verbatim copies 25 | of this license document, but changing it is not allowed. 26 | 27 | Preamble 28 | 29 | The GNU General Public License is a free, copyleft license for 30 | software and other kinds of works. 31 | 32 | The licenses for most software and other practical works are designed 33 | to take away your freedom to share and change the works. By contrast, 34 | the GNU General Public License is intended to guarantee your freedom to 35 | share and change all versions of a program--to make sure it remains free 36 | software for all its users. We, the Free Software Foundation, use the 37 | GNU General Public License for most of our software; it applies also to 38 | any other work released this way by its authors. You can apply it to 39 | your programs, too. 40 | 41 | When we speak of free software, we are referring to freedom, not 42 | price. Our General Public Licenses are designed to make sure that you 43 | have the freedom to distribute copies of free software (and charge for 44 | them if you wish), that you receive source code or can get it if you 45 | want it, that you can change the software or use pieces of it in new 46 | free programs, and that you know you can do these things. 47 | 48 | To protect your rights, we need to prevent others from denying you 49 | these rights or asking you to surrender the rights. Therefore, you have 50 | certain responsibilities if you distribute copies of the software, or if 51 | you modify it: responsibilities to respect the freedom of others. 52 | 53 | For example, if you distribute copies of such a program, whether 54 | gratis or for a fee, you must pass on to the recipients the same 55 | freedoms that you received. You must make sure that they, too, receive 56 | or can get the source code. And you must show them these terms so they 57 | know their rights. 58 | 59 | Developers that use the GNU GPL protect your rights with two steps: 60 | (1) assert copyright on the software, and (2) offer you this License 61 | giving you legal permission to copy, distribute and/or modify it. 62 | 63 | For the developers' and authors' protection, the GPL clearly explains 64 | that there is no warranty for this free software. For both users' and 65 | authors' sake, the GPL requires that modified versions be marked as 66 | changed, so that their problems will not be attributed erroneously to 67 | authors of previous versions. 68 | 69 | Some devices are designed to deny users access to install or run 70 | modified versions of the software inside them, although the manufacturer 71 | can do so. This is fundamentally incompatible with the aim of 72 | protecting users' freedom to change the software. The systematic 73 | pattern of such abuse occurs in the area of products for individuals to 74 | use, which is precisely where it is most unacceptable. Therefore, we 75 | have designed this version of the GPL to prohibit the practice for those 76 | products. If such problems arise substantially in other domains, we 77 | stand ready to extend this provision to those domains in future versions 78 | of the GPL, as needed to protect the freedom of users. 79 | 80 | Finally, every program is threatened constantly by software patents. 81 | States should not allow patents to restrict development and use of 82 | software on general-purpose computers, but in those that do, we wish to 83 | avoid the special danger that patents applied to a free program could 84 | make it effectively proprietary. To prevent this, the GPL assures that 85 | patents cannot be used to render the program non-free. 86 | 87 | The precise terms and conditions for copying, distribution and 88 | modification follow. 89 | 90 | TERMS AND CONDITIONS 91 | 92 | 0. Definitions. 93 | 94 | "This License" refers to version 3 of the GNU General Public License. 95 | 96 | "Copyright" also means copyright-like laws that apply to other kinds of 97 | works, such as semiconductor masks. 98 | 99 | "The Program" refers to any copyrightable work licensed under this 100 | License. Each licensee is addressed as "you". "Licensees" and 101 | "recipients" may be individuals or organizations. 102 | 103 | To "modify" a work means to copy from or adapt all or part of the work 104 | in a fashion requiring copyright permission, other than the making of an 105 | exact copy. The resulting work is called a "modified version" of the 106 | earlier work or a work "based on" the earlier work. 107 | 108 | A "covered work" means either the unmodified Program or a work based 109 | on the Program. 110 | 111 | To "propagate" a work means to do anything with it that, without 112 | permission, would make you directly or secondarily liable for 113 | infringement under applicable copyright law, except executing it on a 114 | computer or modifying a private copy. Propagation includes copying, 115 | distribution (with or without modification), making available to the 116 | public, and in some countries other activities as well. 117 | 118 | To "convey" a work means any kind of propagation that enables other 119 | parties to make or receive copies. Mere interaction with a user through 120 | a computer network, with no transfer of a copy, is not conveying. 121 | 122 | An interactive user interface displays "Appropriate Legal Notices" 123 | to the extent that it includes a convenient and prominently visible 124 | feature that (1) displays an appropriate copyright notice, and (2) 125 | tells the user that there is no warranty for the work (except to the 126 | extent that warranties are provided), that licensees may convey the 127 | work under this License, and how to view a copy of this License. If 128 | the interface presents a list of user commands or options, such as a 129 | menu, a prominent item in the list meets this criterion. 130 | 131 | 1. Source Code. 132 | 133 | The "source code" for a work means the preferred form of the work 134 | for making modifications to it. "Object code" means any non-source 135 | form of a work. 136 | 137 | A "Standard Interface" means an interface that either is an official 138 | standard defined by a recognized standards body, or, in the case of 139 | interfaces specified for a particular programming language, one that 140 | is widely used among developers working in that language. 141 | 142 | The "System Libraries" of an executable work include anything, other 143 | than the work as a whole, that (a) is included in the normal form of 144 | packaging a Major Component, but which is not part of that Major 145 | Component, and (b) serves only to enable use of the work with that 146 | Major Component, or to implement a Standard Interface for which an 147 | implementation is available to the public in source code form. A 148 | "Major Component", in this context, means a major essential component 149 | (kernel, window system, and so on) of the specific operating system 150 | (if any) on which the executable work runs, or a compiler used to 151 | produce the work, or an object code interpreter used to run it. 152 | 153 | The "Corresponding Source" for a work in object code form means all 154 | the source code needed to generate, install, and (for an executable 155 | work) run the object code and to modify the work, including scripts to 156 | control those activities. However, it does not include the work's 157 | System Libraries, or general-purpose tools or generally available free 158 | programs which are used unmodified in performing those activities but 159 | which are not part of the work. For example, Corresponding Source 160 | includes interface definition files associated with source files for 161 | the work, and the source code for shared libraries and dynamically 162 | linked subprograms that the work is specifically designed to require, 163 | such as by intimate data communication or control flow between those 164 | subprograms and other parts of the work. 165 | 166 | The Corresponding Source need not include anything that users 167 | can regenerate automatically from other parts of the Corresponding 168 | Source. 169 | 170 | The Corresponding Source for a work in source code form is that 171 | same work. 172 | 173 | 2. Basic Permissions. 174 | 175 | All rights granted under this License are granted for the term of 176 | copyright on the Program, and are irrevocable provided the stated 177 | conditions are met. This License explicitly affirms your unlimited 178 | permission to run the unmodified Program. The output from running a 179 | covered work is covered by this License only if the output, given its 180 | content, constitutes a covered work. This License acknowledges your 181 | rights of fair use or other equivalent, as provided by copyright law. 182 | 183 | You may make, run and propagate covered works that you do not 184 | convey, without conditions so long as your license otherwise remains 185 | in force. You may convey covered works to others for the sole purpose 186 | of having them make modifications exclusively for you, or provide you 187 | with facilities for running those works, provided that you comply with 188 | the terms of this License in conveying all material for which you do 189 | not control copyright. Those thus making or running the covered works 190 | for you must do so exclusively on your behalf, under your direction 191 | and control, on terms that prohibit them from making any copies of 192 | your copyrighted material outside their relationship with you. 193 | 194 | Conveying under any other circumstances is permitted solely under 195 | the conditions stated below. Sublicensing is not allowed; section 10 196 | makes it unnecessary. 197 | 198 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 199 | 200 | No covered work shall be deemed part of an effective technological 201 | measure under any applicable law fulfilling obligations under article 202 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 203 | similar laws prohibiting or restricting circumvention of such 204 | measures. 205 | 206 | When you convey a covered work, you waive any legal power to forbid 207 | circumvention of technological measures to the extent such circumvention 208 | is effected by exercising rights under this License with respect to 209 | the covered work, and you disclaim any intention to limit operation or 210 | modification of the work as a means of enforcing, against the work's 211 | users, your or third parties' legal rights to forbid circumvention of 212 | technological measures. 213 | 214 | 4. Conveying Verbatim Copies. 215 | 216 | You may convey verbatim copies of the Program's source code as you 217 | receive it, in any medium, provided that you conspicuously and 218 | appropriately publish on each copy an appropriate copyright notice; 219 | keep intact all notices stating that this License and any 220 | non-permissive terms added in accord with section 7 apply to the code; 221 | keep intact all notices of the absence of any warranty; and give all 222 | recipients a copy of this License along with the Program. 223 | 224 | You may charge any price or no price for each copy that you convey, 225 | and you may offer support or warranty protection for a fee. 226 | 227 | 5. Conveying Modified Source Versions. 228 | 229 | You may convey a work based on the Program, or the modifications to 230 | produce it from the Program, in the form of source code under the 231 | terms of section 4, provided that you also meet all of these conditions: 232 | 233 | a) The work must carry prominent notices stating that you modified 234 | it, and giving a relevant date. 235 | 236 | b) The work must carry prominent notices stating that it is 237 | released under this License and any conditions added under section 238 | 7. This requirement modifies the requirement in section 4 to 239 | "keep intact all notices". 240 | 241 | c) You must license the entire work, as a whole, under this 242 | License to anyone who comes into possession of a copy. This 243 | License will therefore apply, along with any applicable section 7 244 | additional terms, to the whole of the work, and all its parts, 245 | regardless of how they are packaged. This License gives no 246 | permission to license the work in any other way, but it does not 247 | invalidate such permission if you have separately received it. 248 | 249 | d) If the work has interactive user interfaces, each must display 250 | Appropriate Legal Notices; however, if the Program has interactive 251 | interfaces that do not display Appropriate Legal Notices, your 252 | work need not make them do so. 253 | 254 | A compilation of a covered work with other separate and independent 255 | works, which are not by their nature extensions of the covered work, 256 | and which are not combined with it such as to form a larger program, 257 | in or on a volume of a storage or distribution medium, is called an 258 | "aggregate" if the compilation and its resulting copyright are not 259 | used to limit the access or legal rights of the compilation's users 260 | beyond what the individual works permit. Inclusion of a covered work 261 | in an aggregate does not cause this License to apply to the other 262 | parts of the aggregate. 263 | 264 | 6. Conveying Non-Source Forms. 265 | 266 | You may convey a covered work in object code form under the terms 267 | of sections 4 and 5, provided that you also convey the 268 | machine-readable Corresponding Source under the terms of this License, 269 | in one of these ways: 270 | 271 | a) Convey the object code in, or embodied in, a physical product 272 | (including a physical distribution medium), accompanied by the 273 | Corresponding Source fixed on a durable physical medium 274 | customarily used for software interchange. 275 | 276 | b) Convey the object code in, or embodied in, a physical product 277 | (including a physical distribution medium), accompanied by a 278 | written offer, valid for at least three years and valid for as 279 | long as you offer spare parts or customer support for that product 280 | model, to give anyone who possesses the object code either (1) a 281 | copy of the Corresponding Source for all the software in the 282 | product that is covered by this License, on a durable physical 283 | medium customarily used for software interchange, for a price no 284 | more than your reasonable cost of physically performing this 285 | conveying of source, or (2) access to copy the 286 | Corresponding Source from a network server at no charge. 287 | 288 | c) Convey individual copies of the object code with a copy of the 289 | written offer to provide the Corresponding Source. This 290 | alternative is allowed only occasionally and noncommercially, and 291 | only if you received the object code with such an offer, in accord 292 | with subsection 6b. 293 | 294 | d) Convey the object code by offering access from a designated 295 | place (gratis or for a charge), and offer equivalent access to the 296 | Corresponding Source in the same way through the same place at no 297 | further charge. You need not require recipients to copy the 298 | Corresponding Source along with the object code. If the place to 299 | copy the object code is a network server, the Corresponding Source 300 | may be on a different server (operated by you or a third party) 301 | that supports equivalent copying facilities, provided you maintain 302 | clear directions next to the object code saying where to find the 303 | Corresponding Source. Regardless of what server hosts the 304 | Corresponding Source, you remain obligated to ensure that it is 305 | available for as long as needed to satisfy these requirements. 306 | 307 | e) Convey the object code using peer-to-peer transmission, provided 308 | you inform other peers where the object code and Corresponding 309 | Source of the work are being offered to the general public at no 310 | charge under subsection 6d. 311 | 312 | A separable portion of the object code, whose source code is excluded 313 | from the Corresponding Source as a System Library, need not be 314 | included in conveying the object code work. 315 | 316 | A "User Product" is either (1) a "consumer product", which means any 317 | tangible personal property which is normally used for personal, family, 318 | or household purposes, or (2) anything designed or sold for incorporation 319 | into a dwelling. In determining whether a product is a consumer product, 320 | doubtful cases shall be resolved in favor of coverage. For a particular 321 | product received by a particular user, "normally used" refers to a 322 | typical or common use of that class of product, regardless of the status 323 | of the particular user or of the way in which the particular user 324 | actually uses, or expects or is expected to use, the product. A product 325 | is a consumer product regardless of whether the product has substantial 326 | commercial, industrial or non-consumer uses, unless such uses represent 327 | the only significant mode of use of the product. 328 | 329 | "Installation Information" for a User Product means any methods, 330 | procedures, authorization keys, or other information required to install 331 | and execute modified versions of a covered work in that User Product from 332 | a modified version of its Corresponding Source. The information must 333 | suffice to ensure that the continued functioning of the modified object 334 | code is in no case prevented or interfered with solely because 335 | modification has been made. 336 | 337 | If you convey an object code work under this section in, or with, or 338 | specifically for use in, a User Product, and the conveying occurs as 339 | part of a transaction in which the right of possession and use of the 340 | User Product is transferred to the recipient in perpetuity or for a 341 | fixed term (regardless of how the transaction is characterized), the 342 | Corresponding Source conveyed under this section must be accompanied 343 | by the Installation Information. But this requirement does not apply 344 | if neither you nor any third party retains the ability to install 345 | modified object code on the User Product (for example, the work has 346 | been installed in ROM). 347 | 348 | The requirement to provide Installation Information does not include a 349 | requirement to continue to provide support service, warranty, or updates 350 | for a work that has been modified or installed by the recipient, or for 351 | the User Product in which it has been modified or installed. Access to a 352 | network may be denied when the modification itself materially and 353 | adversely affects the operation of the network or violates the rules and 354 | protocols for communication across the network. 355 | 356 | Corresponding Source conveyed, and Installation Information provided, 357 | in accord with this section must be in a format that is publicly 358 | documented (and with an implementation available to the public in 359 | source code form), and must require no special password or key for 360 | unpacking, reading or copying. 361 | 362 | 7. Additional Terms. 363 | 364 | "Additional permissions" are terms that supplement the terms of this 365 | License by making exceptions from one or more of its conditions. 366 | Additional permissions that are applicable to the entire Program shall 367 | be treated as though they were included in this License, to the extent 368 | that they are valid under applicable law. If additional permissions 369 | apply only to part of the Program, that part may be used separately 370 | under those permissions, but the entire Program remains governed by 371 | this License without regard to the additional permissions. 372 | 373 | When you convey a copy of a covered work, you may at your option 374 | remove any additional permissions from that copy, or from any part of 375 | it. (Additional permissions may be written to require their own 376 | removal in certain cases when you modify the work.) You may place 377 | additional permissions on material, added by you to a covered work, 378 | for which you have or can give appropriate copyright permission. 379 | 380 | Notwithstanding any other provision of this License, for material you 381 | add to a covered work, you may (if authorized by the copyright holders of 382 | that material) supplement the terms of this License with terms: 383 | 384 | a) Disclaiming warranty or limiting liability differently from the 385 | terms of sections 15 and 16 of this License; or 386 | 387 | b) Requiring preservation of specified reasonable legal notices or 388 | author attributions in that material or in the Appropriate Legal 389 | Notices displayed by works containing it; or 390 | 391 | c) Prohibiting misrepresentation of the origin of that material, or 392 | requiring that modified versions of such material be marked in 393 | reasonable ways as different from the original version; or 394 | 395 | d) Limiting the use for publicity purposes of names of licensors or 396 | authors of the material; or 397 | 398 | e) Declining to grant rights under trademark law for use of some 399 | trade names, trademarks, or service marks; or 400 | 401 | f) Requiring indemnification of licensors and authors of that 402 | material by anyone who conveys the material (or modified versions of 403 | it) with contractual assumptions of liability to the recipient, for 404 | any liability that these contractual assumptions directly impose on 405 | those licensors and authors. 406 | 407 | All other non-permissive additional terms are considered "further 408 | restrictions" within the meaning of section 10. If the Program as you 409 | received it, or any part of it, contains a notice stating that it is 410 | governed by this License along with a term that is a further 411 | restriction, you may remove that term. If a license document contains 412 | a further restriction but permits relicensing or conveying under this 413 | License, you may add to a covered work material governed by the terms 414 | of that license document, provided that the further restriction does 415 | not survive such relicensing or conveying. 416 | 417 | If you add terms to a covered work in accord with this section, you 418 | must place, in the relevant source files, a statement of the 419 | additional terms that apply to those files, or a notice indicating 420 | where to find the applicable terms. 421 | 422 | Additional terms, permissive or non-permissive, may be stated in the 423 | form of a separately written license, or stated as exceptions; 424 | the above requirements apply either way. 425 | 426 | 8. Termination. 427 | 428 | You may not propagate or modify a covered work except as expressly 429 | provided under this License. Any attempt otherwise to propagate or 430 | modify it is void, and will automatically terminate your rights under 431 | this License (including any patent licenses granted under the third 432 | paragraph of section 11). 433 | 434 | However, if you cease all violation of this License, then your 435 | license from a particular copyright holder is reinstated (a) 436 | provisionally, unless and until the copyright holder explicitly and 437 | finally terminates your license, and (b) permanently, if the copyright 438 | holder fails to notify you of the violation by some reasonable means 439 | prior to 60 days after the cessation. 440 | 441 | Moreover, your license from a particular copyright holder is 442 | reinstated permanently if the copyright holder notifies you of the 443 | violation by some reasonable means, this is the first time you have 444 | received notice of violation of this License (for any work) from that 445 | copyright holder, and you cure the violation prior to 30 days after 446 | your receipt of the notice. 447 | 448 | Termination of your rights under this section does not terminate the 449 | licenses of parties who have received copies or rights from you under 450 | this License. If your rights have been terminated and not permanently 451 | reinstated, you do not qualify to receive new licenses for the same 452 | material under section 10. 453 | 454 | 9. Acceptance Not Required for Having Copies. 455 | 456 | You are not required to accept this License in order to receive or 457 | run a copy of the Program. Ancillary propagation of a covered work 458 | occurring solely as a consequence of using peer-to-peer transmission 459 | to receive a copy likewise does not require acceptance. However, 460 | nothing other than this License grants you permission to propagate or 461 | modify any covered work. These actions infringe copyright if you do 462 | not accept this License. Therefore, by modifying or propagating a 463 | covered work, you indicate your acceptance of this License to do so. 464 | 465 | 10. Automatic Licensing of Downstream Recipients. 466 | 467 | Each time you convey a covered work, the recipient automatically 468 | receives a license from the original licensors, to run, modify and 469 | propagate that work, subject to this License. You are not responsible 470 | for enforcing compliance by third parties with this License. 471 | 472 | An "entity transaction" is a transaction transferring control of an 473 | organization, or substantially all assets of one, or subdividing an 474 | organization, or merging organizations. If propagation of a covered 475 | work results from an entity transaction, each party to that 476 | transaction who receives a copy of the work also receives whatever 477 | licenses to the work the party's predecessor in interest had or could 478 | give under the previous paragraph, plus a right to possession of the 479 | Corresponding Source of the work from the predecessor in interest, if 480 | the predecessor has it or can get it with reasonable efforts. 481 | 482 | You may not impose any further restrictions on the exercise of the 483 | rights granted or affirmed under this License. For example, you may 484 | not impose a license fee, royalty, or other charge for exercise of 485 | rights granted under this License, and you may not initiate litigation 486 | (including a cross-claim or counterclaim in a lawsuit) alleging that 487 | any patent claim is infringed by making, using, selling, offering for 488 | sale, or importing the Program or any portion of it. 489 | 490 | 11. Patents. 491 | 492 | A "contributor" is a copyright holder who authorizes use under this 493 | License of the Program or a work on which the Program is based. The 494 | work thus licensed is called the contributor's "contributor version". 495 | 496 | A contributor's "essential patent claims" are all patent claims 497 | owned or controlled by the contributor, whether already acquired or 498 | hereafter acquired, that would be infringed by some manner, permitted 499 | by this License, of making, using, or selling its contributor version, 500 | but do not include claims that would be infringed only as a 501 | consequence of further modification of the contributor version. For 502 | purposes of this definition, "control" includes the right to grant 503 | patent sublicenses in a manner consistent with the requirements of 504 | this License. 505 | 506 | Each contributor grants you a non-exclusive, worldwide, royalty-free 507 | patent license under the contributor's essential patent claims, to 508 | make, use, sell, offer for sale, import and otherwise run, modify and 509 | propagate the contents of its contributor version. 510 | 511 | In the following three paragraphs, a "patent license" is any express 512 | agreement or commitment, however denominated, not to enforce a patent 513 | (such as an express permission to practice a patent or covenant not to 514 | sue for patent infringement). To "grant" such a patent license to a 515 | party means to make such an agreement or commitment not to enforce a 516 | patent against the party. 517 | 518 | If you convey a covered work, knowingly relying on a patent license, 519 | and the Corresponding Source of the work is not available for anyone 520 | to copy, free of charge and under the terms of this License, through a 521 | publicly available network server or other readily accessible means, 522 | then you must either (1) cause the Corresponding Source to be so 523 | available, or (2) arrange to deprive yourself of the benefit of the 524 | patent license for this particular work, or (3) arrange, in a manner 525 | consistent with the requirements of this License, to extend the patent 526 | license to downstream recipients. "Knowingly relying" means you have 527 | actual knowledge that, but for the patent license, your conveying the 528 | covered work in a country, or your recipient's use of the covered work 529 | in a country, would infringe one or more identifiable patents in that 530 | country that you have reason to believe are valid. 531 | 532 | If, pursuant to or in connection with a single transaction or 533 | arrangement, you convey, or propagate by procuring conveyance of, a 534 | covered work, and grant a patent license to some of the parties 535 | receiving the covered work authorizing them to use, propagate, modify 536 | or convey a specific copy of the covered work, then the patent license 537 | you grant is automatically extended to all recipients of the covered 538 | work and works based on it. 539 | 540 | A patent license is "discriminatory" if it does not include within 541 | the scope of its coverage, prohibits the exercise of, or is 542 | conditioned on the non-exercise of one or more of the rights that are 543 | specifically granted under this License. You may not convey a covered 544 | work if you are a party to an arrangement with a third party that is 545 | in the business of distributing software, under which you make payment 546 | to the third party based on the extent of your activity of conveying 547 | the work, and under which the third party grants, to any of the 548 | parties who would receive the covered work from you, a discriminatory 549 | patent license (a) in connection with copies of the covered work 550 | conveyed by you (or copies made from those copies), or (b) primarily 551 | for and in connection with specific products or compilations that 552 | contain the covered work, unless you entered into that arrangement, 553 | or that patent license was granted, prior to 28 March 2007. 554 | 555 | Nothing in this License shall be construed as excluding or limiting 556 | any implied license or other defenses to infringement that may 557 | otherwise be available to you under applicable patent law. 558 | 559 | 12. No Surrender of Others' Freedom. 560 | 561 | If conditions are imposed on you (whether by court order, agreement or 562 | otherwise) that contradict the conditions of this License, they do not 563 | excuse you from the conditions of this License. If you cannot convey a 564 | covered work so as to satisfy simultaneously your obligations under this 565 | License and any other pertinent obligations, then as a consequence you may 566 | not convey it at all. For example, if you agree to terms that obligate you 567 | to collect a royalty for further conveying from those to whom you convey 568 | the Program, the only way you could satisfy both those terms and this 569 | License would be to refrain entirely from conveying the Program. 570 | 571 | 13. Use with the GNU Affero General Public License. 572 | 573 | Notwithstanding any other provision of this License, you have 574 | permission to link or combine any covered work with a work licensed 575 | under version 3 of the GNU Affero General Public License into a single 576 | combined work, and to convey the resulting work. The terms of this 577 | License will continue to apply to the part which is the covered work, 578 | but the special requirements of the GNU Affero General Public License, 579 | section 13, concerning interaction through a network will apply to the 580 | combination as such. 581 | 582 | 14. Revised Versions of this License. 583 | 584 | The Free Software Foundation may publish revised and/or new versions of 585 | the GNU General Public License from time to time. Such new versions will 586 | be similar in spirit to the present version, but may differ in detail to 587 | address new problems or concerns. 588 | 589 | Each version is given a distinguishing version number. If the 590 | Program specifies that a certain numbered version of the GNU General 591 | Public License "or any later version" applies to it, you have the 592 | option of following the terms and conditions either of that numbered 593 | version or of any later version published by the Free Software 594 | Foundation. If the Program does not specify a version number of the 595 | GNU General Public License, you may choose any version ever published 596 | by the Free Software Foundation. 597 | 598 | If the Program specifies that a proxy can decide which future 599 | versions of the GNU General Public License can be used, that proxy's 600 | public statement of acceptance of a version permanently authorizes you 601 | to choose that version for the Program. 602 | 603 | Later license versions may give you additional or different 604 | permissions. However, no additional obligations are imposed on any 605 | author or copyright holder as a result of your choosing to follow a 606 | later version. 607 | 608 | 15. Disclaimer of Warranty. 609 | 610 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 611 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 612 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 613 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 614 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 615 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 616 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 617 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 618 | 619 | 16. Limitation of Liability. 620 | 621 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 622 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 623 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 624 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 625 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 626 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 627 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 628 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 629 | SUCH DAMAGES. 630 | 631 | 17. Interpretation of Sections 15 and 16. 632 | 633 | If the disclaimer of warranty and limitation of liability provided 634 | above cannot be given local legal effect according to their terms, 635 | reviewing courts shall apply local law that most closely approximates 636 | an absolute waiver of all civil liability in connection with the 637 | Program, unless a warranty or assumption of liability accompanies a 638 | copy of the Program in return for a fee. 639 | 640 | END OF TERMS AND CONDITIONS 641 | 642 | How to Apply These Terms to Your New Programs 643 | 644 | If you develop a new program, and you want it to be of the greatest 645 | possible use to the public, the best way to achieve this is to make it 646 | free software which everyone can redistribute and change under these terms. 647 | 648 | To do so, attach the following notices to the program. It is safest 649 | to attach them to the start of each source file to most effectively 650 | state the exclusion of warranty; and each file should have at least 651 | the "copyright" line and a pointer to where the full notice is found. 652 | 653 | 654 | Copyright (C) 655 | 656 | This program is free software: you can redistribute it and/or modify 657 | it under the terms of the GNU General Public License as published by 658 | the Free Software Foundation, either version 3 of the License, or 659 | (at your option) any later version. 660 | 661 | This program is distributed in the hope that it will be useful, 662 | but WITHOUT ANY WARRANTY; without even the implied warranty of 663 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 664 | GNU General Public License for more details. 665 | 666 | You should have received a copy of the GNU General Public License 667 | along with this program. If not, see . 668 | 669 | Also add information on how to contact you by electronic and paper mail. 670 | 671 | If the program does terminal interaction, make it output a short 672 | notice like this when it starts in an interactive mode: 673 | 674 | Copyright (C) 675 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 676 | This is free software, and you are welcome to redistribute it 677 | under certain conditions; type `show c' for details. 678 | 679 | The hypothetical commands `show w' and `show c' should show the appropriate 680 | parts of the General Public License. Of course, your program's commands 681 | might be different; for a GUI interface, you would use an "about box". 682 | 683 | You should also get your employer (if you work as a programmer) or school, 684 | if any, to sign a "copyright disclaimer" for the program, if necessary. 685 | For more information on this, and how to apply and follow the GNU GPL, see 686 | . 687 | 688 | The GNU General Public License does not permit incorporating your program 689 | into proprietary programs. If your program is a subroutine library, you 690 | may consider it more useful to permit linking proprietary applications with 691 | the library. If this is what you want to do, use the GNU Lesser General 692 | Public License instead of this License. But first, please read 693 | . 694 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![](https://neurojson.org/wiki/upload/neurojson_banner_long.png) 2 | 3 | # EasyH5 Toolbox - An easy-to-use HDF5 data interface (loadh5 and saveh5) 4 | 5 | * Copyright (C) 2019,2022,2025 Qianqian Fang 6 | * License: GNU General Public License version 3 (GPL v3) or 3-clause BSD license, see LICENSE*.txt 7 | * Version: 0.9 (code name: Daseot - Korean 5) 8 | * URL: https://github.com/NeuroJSON/easyh5 9 | * Compatibility: MATLAB R2010b or newer and GNU Octave 5.0 or newer 10 | * Acknowledgement: This project is supported by US National Institute of Health (NIH) 11 | grant [U24-NS124027 (NeuroJSON)](https://reporter.nih.gov/project-details/10308329) 12 | 13 | ## Overview 14 | 15 | EasyH5 is a fully automated, fast, compact and portable MATLAB object to HDF5 16 | exporter/importer. It contains two easy-to-use functions - `loadh5.m` and 17 | `saveh5.m`. The `saveh5.m` can handle almost all MATLAB data types, including 18 | structs, struct arrays, cells, cell arrays, real and complex arrays, strings, 19 | and `containers.Map` objects. All other data classes (such as a table, digraph, 20 | etc) can also be stored/loaded seemlessly using an undocumented data serialization 21 | interface (MATLAB only). 22 | 23 | EasyH5 stores complex-valued arrays using a special compound data type in an 24 | HDF5 dataset. The real-part of the data are stored as `Real` and the imaginary 25 | part is stored as the `Imag` component. The `loadh5.m` automatically converts 26 | such data structure to a complex array. Starting from v0.8, EasyH5 also supports 27 | saving and loading sparse arrays using a compound dataset with 2 or 3 28 | specialized subfields: `SparseArray`, `Real`, and, in the case of a sparse 29 | complex array, `Imag`. The sparse array dimension is stored as an attribute 30 | named `SparseArraySize`, attached with the dataset. Using the `deflate` filter 31 | to save compressed arrays is supported in v0.8 and later. 32 | 33 | Because HDF5 does not directly support 1-D/N-D cell arrays or struct arrays, 34 | EasyH5 converts these data structures into data groups with names in the 35 | following format 36 | ``` 37 | ['/hdf5/path/.../varname',num2str(idx1d)] 38 | ``` 39 | where `varname` is the variable/field name to the cell/struct array object, 40 | and `idx1d` is the 1-D integer index of the cell/struct array. We also provide 41 | a function, `regrouph5.m` to automatically collapse these group/dataset names 42 | into 1-D cell/struct arrays after loading the data using `loadh5.m`. See examples 43 | below. 44 | 45 | ## Installation 46 | 47 | The EasyH5 toolbox can be installed using a single command 48 | ``` 49 | addpath('/path/to/easyh5'); 50 | ``` 51 | where the `/path/to/easyh5` should be replaced by the unzipped folder 52 | of the toolbox (i.e. the folder containing `loadh5.m/saveh5.m`). 53 | 54 | In v0.8 and newer releases, EasyH5 supports GNU Octave 5.x or later. 55 | In order to use EasyH5 with Octave, one must install a toolbox named oct-hdf5 56 | by running the following command in Octave once 57 | ``` 58 | pkg install https://github.com/NeuroJSON/oct-hdf5/archive/refs/tags/git20250413.zip 59 | pkg load oct-hdf5 60 | ``` 61 | 62 | EasyH5 supports MATLAB R2010b or newer. However, `saveh5` can not save empty 63 | arrays for MATLAB releases before 2015. 64 | 65 | 66 | ## Usage 67 | 68 | ### `saveh5` - Save a MATLAB struct (array) or cell (array) into an HDF5 file 69 | Save a MATLAB struct (array) or cell (array) into an HDF5 file. 70 | 71 | Example: 72 | ``` 73 | a=struct('a',rand(5),'c','string','b',true,'d',2+3i,'e',{'test',[],1:5}); 74 | saveh5(a,'test.h5'); 75 | saveh5(a(1),'test2.h5','rootname',''); 76 | saveh5(a(1),'test2.h5','compression','deflate','compressarraysize',1); 77 | saveh5('appending data to existing file','test.h5','rootname','/name','append',1); 78 | saveh5(a,'test.h5j','jdata',1); 79 | ``` 80 | ### `loadh5` - Load data in an HDF5 file to a MATLAB structure. 81 | Load data in an HDF5 file to a MATLAB structure. 82 | 83 | Example: 84 | ``` 85 | a={rand(2), struct('va',1,'vb','string'), 1+2i}; 86 | saveh5(a,'test.h5'); 87 | a2=loadh5('test.h5') 88 | a3=loadh5('test.h5','regroup',1) 89 | isequaln(a,a3.a) 90 | a4=loadh5('test.h5','/a1') 91 | ``` 92 | ### `regrouph5` - Processing an HDF5 based data and group indexed datasets into a cell array 93 | Processing a loadh5 restored data and merge "indexed datasets", whose 94 | names start with an ASCII string followed by a contiguous integer 95 | sequence number starting from 1, into a cell array. For example, 96 | datasets `{data.a1, data.a2, data.a3}` will be merged into a cell/struct 97 | array data.a with 3 elements. 98 | 99 | Example: 100 | ``` 101 | a=struct('a1',rand(5),'a2','string','a3',true,'d',2+3i,'e',{'test',[],1:5}); 102 | a(1).a1=0; a(2).a2='test'; 103 | data=regrouph5(a) 104 | saveh5(a,'test.h5'); 105 | rawdata=loadh5('test.h5') 106 | data=regrouph5(rawdata) 107 | ``` 108 | 109 | ## Known problems 110 | - EasyH5 currently does not support 2D cell and struct arrays 111 | - If a cell name ends with a number, such as `a10={...}`; `regrouph5` can not group the cell correctly 112 | - If a database/group name is longer than 63 characters, it may have the risk of being truncated 113 | - When saving a dynamic expression instead of a named variable, the data are stored under path `/data` 114 | 115 | ## Contribute to EasyH5 116 | 117 | Please submit your bug reports, feature requests and questions to the Github Issues page at 118 | 119 | https://github.com/NeuroJSON/easyh5/issues 120 | 121 | Please feel free to fork our software, making changes, and submit your revision back 122 | to us via "Pull Requests". EasyH5 is open-source and welcome to your contributions! 123 | 124 | -------------------------------------------------------------------------------- /decodevarname.m: -------------------------------------------------------------------------------- 1 | function newname = decodevarname(name, varargin) 2 | % 3 | % newname = decodevarname(name) 4 | % 5 | % Decode a hex-encoded variable name (from encodevarname) and restore 6 | % its original form 7 | % 8 | % This function is sensitive to the default charset 9 | % settings in MATLAB, please call feature('DefaultCharacterSet','utf8') 10 | % to set the encoding to UTF-8 before calling this function. 11 | % 12 | % author: Qianqian Fang (q.fang neu.edu) 13 | % 14 | % input: 15 | % name: a string output from encodevarname, which converts the leading non-ascii 16 | % letter into "x0xHH_" and non-ascii letters into "_0xHH_" 17 | % format, where hex key HH stores the ascii (or Unicode) value 18 | % of the character. 19 | % 20 | % output: 21 | % newname: the restored original string 22 | % 23 | % example: 24 | % decodevarname('x0x5F_a') % returns _a 25 | % decodevarname('a_') % returns a_ as it is a valid variable name 26 | % decodevarname('x0xE58F98__0xE9878F_') % returns '变量' 27 | % 28 | % this file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 29 | % 30 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 31 | % 32 | 33 | newname = name; 34 | isunpack = 1; 35 | if (nargin == 2 && ~isstruct(varargin{1})) 36 | isunpack = varargin{1}; 37 | elseif (nargin > 1) 38 | isunpack = jsonopt('UnpackHex', 1, varargin{:}); 39 | end 40 | 41 | if (isunpack) 42 | if (isempty(strfind(name, '0x')) || isempty(regexp(name, '0x([0-9a-fA-F]+)_', 'once'))) 43 | return 44 | end 45 | if (exist('native2unicode', 'builtin')) 46 | h2u = @hex2unicode; 47 | newname = regexprep(name, '(^x|_){1}0x([0-9a-fA-F]+)_', '${h2u($2)}'); 48 | else 49 | if (isunpack && strcmp(name, 'x0x0_')) 50 | newname = ''; 51 | return 52 | end 53 | pos = regexp(name, '(^x|_){1}0x([0-9a-fA-F]+)_', 'start'); 54 | pend = regexp(name, '(^x|_){1}0x([0-9a-fA-F]+)_', 'end'); 55 | if (isempty(pos)) 56 | return 57 | end 58 | str0 = name; 59 | pos0 = [0 pend(:)' length(name)]; 60 | newname = ''; 61 | for i = 1:length(pos) 62 | newname = [newname str0(pos0(i) + 1:pos(i) - 1) char(hex2dec(str0(pos(i) + 3:pend(i) - 1)))]; 63 | end 64 | if (pos(end) ~= length(name)) 65 | newname = [newname str0(pos0(end - 1) + 1:pos0(end))]; 66 | end 67 | end 68 | end 69 | 70 | % -------------------------------------------------------------------------- 71 | function str = hex2unicode(hexstr) 72 | val = hex2dec(hexstr); 73 | id = histc(val, [0 2^8 2^16 2^32 2^64]); 74 | type = {'uint8', 'uint16', 'uint32', 'uint64'}; 75 | bytes = typecast(cast(val, type{id ~= 0}), 'uint8'); 76 | str = native2unicode(fliplr(bytes(:, 1:find(bytes, 1, 'last')))); 77 | -------------------------------------------------------------------------------- /encodevarname.m: -------------------------------------------------------------------------------- 1 | function str = encodevarname(str, varargin) 2 | % 3 | % newname = encodevarname(name) 4 | % 5 | % Encode an invalid variable name using a hex-format for bi-directional 6 | % conversions. 7 | 8 | % This function is sensitive to the default charset 9 | % settings in MATLAB, please call feature('DefaultCharacterSet','utf8') 10 | % to set the encoding to UTF-8 before calling this function. 11 | % 12 | % author: Qianqian Fang (q.fang neu.edu) 13 | % 14 | % input: 15 | % name: a string, can be either a valid or invalid variable name 16 | % 17 | % output: 18 | % newname: a valid variable name by converting the leading non-ascii 19 | % letter into "x0xHH_" and non-ascii letters into "_0xHH_" 20 | % format, where HH is the ascii (or Unicode) value of the 21 | % character. 22 | % 23 | % if the encoded variable name CAN NOT be longer than 63, i.e. 24 | % the maximum variable name specified by namelengthmax, and 25 | % one uses the output of this function as a struct or variable 26 | % name, the name will be truncated at 63. Please consider using 27 | % the name as a containers.Map key, which does not have such 28 | % limit. 29 | % 30 | % example: 31 | % encodevarname('_a') % returns x0x5F_a 32 | % encodevarname('a_') % returns a_ as it is a valid variable name 33 | % encodevarname('变量') % returns 'x0xE58F98__0xE9878F_' 34 | % 35 | % this file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 36 | % 37 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 38 | % 39 | 40 | if (~isvarname(str(1))) 41 | if (exist('unicode2native', 'builtin')) 42 | str = sprintf('x0x%s_%s', sprintf('%X', unicode2native(str(1))), str(2:end)); 43 | else 44 | str = sprintf('x0x%X_%s', char(str(1)) + 0, str(2:end)); 45 | end 46 | end 47 | if (isvarname(str)) 48 | return 49 | end 50 | if (exist('unicode2native', 'builtin')) 51 | str = regexprep(str, '([^0-9A-Za-z_])', '_0x${sprintf(''%X'',unicode2native($1))}_'); 52 | else 53 | cpos = find(~ismember(str, ['0':'9', 'A':'Z', 'a':'z', '_'])); 54 | % cpos=regexp(str,'[^0-9A-Za-z_]'); 55 | if (isempty(cpos)) 56 | return 57 | end 58 | str0 = str; 59 | pos0 = [0 cpos(:)' length(str)]; 60 | str = ''; 61 | for i = 1:length(cpos) 62 | str = [str str0(pos0(i) + 1:cpos(i) - 1) sprintf('_0x%X_', str0(cpos(i)) + 0)]; 63 | end 64 | if (cpos(end) ~= length(str)) 65 | str = [str str0(pos0(end - 1) + 1:pos0(end))]; 66 | end 67 | end 68 | -------------------------------------------------------------------------------- /examples/demo_eazyh5_basic.m: -------------------------------------------------------------------------------- 1 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2 | % Demonstration of Basic Utilities of EasyH5 3 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 4 | 5 | rngstate = rand ('state'); 6 | randseed = hex2dec('623F9A9E'); 7 | clear data2hdf h52data opt; 8 | 9 | opt.releaseid = 0; 10 | opt.isoctave = 0; 11 | vers = ver('MATLAB'); 12 | if (~isempty(vers)) 13 | opt.releaseid = datenum(vers(1).Date); 14 | opt.skipempty = (opt.releaseid < datenum('1-Jan-2015')); 15 | else 16 | opt.isoctave = 1; 17 | opt.skipempty = 0; 18 | end 19 | 20 | testequal = @isequal; 21 | if (exist('isequaln', 'builtin')) 22 | testequal = @isequaln; 23 | end 24 | 25 | fprintf(1, '\n%%=================================================\n'); 26 | fprintf(1, '%% a simple scalar value \n'); 27 | fprintf(1, '%%=================================================\n\n'); 28 | 29 | data2hdf = pi; 30 | saveh5(data2hdf, 'test.h5'); 31 | h52data = loadh5('test.h5'); 32 | testequal(data2hdf, h52data.data2hdf); 33 | 34 | fprintf(1, '\n%%=================================================\n'); 35 | fprintf(1, '%% an empty array \n'); 36 | fprintf(1, '%%=================================================\n\n'); 37 | 38 | data2hdf = []; 39 | saveh5(data2hdf, 'test.h5'); 40 | h52data = loadh5('test.h5'); 41 | if (~opt.skipempty) 42 | testequal(data2hdf, h52data.data2hdf); 43 | end 44 | 45 | fprintf(1, '\n%%=================================================\n'); 46 | fprintf(1, '%% an ampty string \n'); 47 | fprintf(1, '%%=================================================\n\n'); 48 | 49 | data2hdf = ''; 50 | saveh5(data2hdf, 'test.h5'); 51 | h52data = loadh5('test.h5'); 52 | if (~opt.skipempty) 53 | testequal(data2hdf, h52data.data2hdf); 54 | end 55 | 56 | fprintf(1, '\n%%=================================================\n'); 57 | fprintf(1, '%% a simple row vector \n'); 58 | fprintf(1, '%%=================================================\n\n'); 59 | 60 | data2hdf = 1:3; 61 | saveh5(data2hdf, 'test.h5'); 62 | h52data = loadh5('test.h5'); 63 | testequal(data2hdf, h52data.data2hdf); 64 | 65 | fprintf(1, '\n%%=================================================\n'); 66 | fprintf(1, '%% a simple column vector \n'); 67 | fprintf(1, '%%=================================================\n\n'); 68 | 69 | data2hdf = (1:3)'; 70 | saveh5(data2hdf, 'test.h5'); 71 | h52data = loadh5('test.h5'); 72 | testequal(data2hdf, h52data.data2hdf); 73 | 74 | fprintf(1, '\n%%=================================================\n'); 75 | fprintf(1, '%% a string array \n'); 76 | fprintf(1, '%%=================================================\n\n'); 77 | 78 | data2hdf = ['AC'; 'EG']; 79 | saveh5(data2hdf, 'test.h5'); 80 | h52data = loadh5('test.h5'); 81 | testequal(data2hdf, h52data.data2hdf); 82 | 83 | fprintf(1, '\n%%=================================================\n'); 84 | fprintf(1, '%% a string with escape symbols \n'); 85 | fprintf(1, '%%=================================================\n\n'); 86 | 87 | data2hdf = sprintf('AB\tCD\none"two'); 88 | saveh5(data2hdf, 'test.h5'); 89 | h52data = loadh5('test.h5'); 90 | testequal(data2hdf, h52data.data2hdf); 91 | 92 | fprintf(1, '\n%%=================================================\n'); 93 | fprintf(1, '%% a mix-typed cell \n'); 94 | fprintf(1, '%%=================================================\n\n'); 95 | 96 | data2hdf = {'a', true, [2; 3]}; 97 | saveh5(data2hdf, 'test.h5'); 98 | h52data = loadh5('test.h5', 'regroup', 1); 99 | testequal(data2hdf, h52data.data2hdf); 100 | 101 | fprintf(1, '\n%%=================================================\n'); 102 | fprintf(1, '%% a 3-D array in nested array form\n'); 103 | fprintf(1, '%%=================================================\n\n'); 104 | 105 | data2hdf = reshape(1:(2 * 4 * 6), [2, 4, 6]); 106 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 107 | h52data = loadh5('test.h5'); 108 | testequal(data2hdf, h52data.data2hdf); 109 | 110 | fprintf(1, '\n%%=================================================\n'); 111 | fprintf(1, '%% a 4-D array in annotated array form\n'); 112 | fprintf(1, '%%=================================================\n\n'); 113 | 114 | data2hdf = reshape(1:(2 * 4 * 3 * 2), [2, 4, 3, 2]); 115 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 116 | h52data = loadh5('test.h5'); 117 | testequal(data2hdf, h52data.data2hdf); 118 | 119 | fprintf(1, '\n%%=================================================\n'); 120 | fprintf(1, '%% a 3-D array in annotated array form (JSONLab 1.9 or earlier)\n'); 121 | fprintf(1, '%%=================================================\n\n'); 122 | 123 | data2hdf = reshape(1:(2 * 4 * 6), [2, 4, 6]); 124 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 125 | h52data = loadh5('test.h5'); 126 | testequal(data2hdf, h52data.data2hdf); 127 | 128 | if (~opt.isoctave) 129 | 130 | fprintf(1, '\n%%=================================================\n'); 131 | fprintf(1, '%% a complex number\n'); 132 | fprintf(1, '%%=================================================\n\n'); 133 | 134 | data2hdf = 1 + 2i; 135 | saveh5(data2hdf, 'test.h5'); 136 | h52data = loadh5('test.h5'); 137 | testequal(data2hdf, h52data.data2hdf); 138 | 139 | fprintf(1, '\n%%=================================================\n'); 140 | fprintf(1, '%% a complex matrix\n'); 141 | fprintf(1, '%%=================================================\n\n'); 142 | 143 | data2hdf = magic(6); 144 | data2hdf = data2hdf(:, 1:3) + data2hdf(:, 4:6) * 1i; 145 | saveh5(data2hdf, 'test.h5'); 146 | h52data = loadh5('test.h5'); 147 | testequal(data2hdf, h52data.data2hdf); 148 | 149 | end 150 | 151 | fprintf(1, '\n%%=================================================\n'); 152 | fprintf(1, '%% MATLAB special constants\n'); 153 | fprintf(1, '%%=================================================\n\n'); 154 | 155 | data2hdf = [NaN Inf -Inf]; 156 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 157 | h52data = loadh5('test.h5'); 158 | testequal(data2hdf, h52data.data2hdf); 159 | 160 | fprintf(1, '\n%%=================================================\n'); 161 | fprintf(1, '%% a real sparse matrix\n'); 162 | fprintf(1, '%%=================================================\n\n'); 163 | 164 | if (~opt.isoctave) 165 | 166 | data2hdf = sprand(10, 10, 0.1); 167 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 168 | h52data = loadh5('test.h5'); 169 | testequal(data2hdf, h52data.data2hdf); 170 | 171 | fprintf(1, '\n%%=================================================\n'); 172 | fprintf(1, '%% a complex sparse matrix\n'); 173 | fprintf(1, '%%=================================================\n\n'); 174 | 175 | data2hdf = data2hdf - data2hdf * 1i; 176 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 177 | h52data = loadh5('test.h5'); 178 | testequal(data2hdf, h52data.data2hdf); 179 | 180 | fprintf(1, '\n%%=================================================\n'); 181 | fprintf(1, '%% an all-zero sparse matrix\n'); 182 | fprintf(1, '%%=================================================\n\n'); 183 | 184 | data2hdf = sparse(2, 3); 185 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 186 | h52data = loadh5('test.h5'); 187 | if (~opt.skipempty) 188 | testequal(data2hdf, h52data.data2hdf); 189 | end 190 | 191 | fprintf(1, '\n%%=================================================\n'); 192 | fprintf(1, '%% an empty sparse matrix\n'); 193 | fprintf(1, '%%=================================================\n\n'); 194 | 195 | data2hdf = sparse([]); 196 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 197 | h52data = loadh5('test.h5'); 198 | if (~opt.skipempty) 199 | testequal(data2hdf, h52data.data2hdf); 200 | end 201 | 202 | end 203 | 204 | fprintf(1, '\n%%=================================================\n'); 205 | fprintf(1, '%% an empty 0-by-0 real matrix\n'); 206 | fprintf(1, '%%=================================================\n\n'); 207 | 208 | data2hdf = []; 209 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 210 | h52data = loadh5('test.h5'); 211 | if (~opt.skipempty) 212 | testequal(data2hdf, h52data.data2hdf); 213 | end 214 | 215 | fprintf(1, '\n%%=================================================\n'); 216 | fprintf(1, '%% an empty 0-by-3 real matrix\n'); 217 | fprintf(1, '%%=================================================\n\n'); 218 | 219 | data2hdf = zeros(0, 3); 220 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 221 | h52data = loadh5('test.h5'); 222 | if (~opt.skipempty) 223 | testequal(data2hdf, h52data.data2hdf); 224 | end 225 | 226 | if (~opt.isoctave) 227 | 228 | fprintf(1, '\n%%=================================================\n'); 229 | fprintf(1, '%% a sparse real column vector\n'); 230 | fprintf(1, '%%=================================================\n\n'); 231 | 232 | data2hdf = sparse([0, 3, 0, 1, 4]'); 233 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 234 | h52data = loadh5('test.h5'); 235 | testequal(data2hdf, h52data.data2hdf); 236 | 237 | fprintf(1, '\n%%=================================================\n'); 238 | fprintf(1, '%% a sparse complex column vector\n'); 239 | fprintf(1, '%%=================================================\n\n'); 240 | 241 | data2hdf = data2hdf - 1i * data2hdf; 242 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 243 | h52data = loadh5('test.h5'); 244 | testequal(data2hdf, h52data.data2hdf); 245 | 246 | fprintf(1, '\n%%=================================================\n'); 247 | fprintf(1, '%% a sparse real row vector\n'); 248 | fprintf(1, '%%=================================================\n\n'); 249 | 250 | data2hdf = sparse([0, 3, 0, 1, 4]); 251 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 252 | h52data = loadh5('test.h5'); 253 | testequal(data2hdf, h52data.data2hdf); 254 | 255 | fprintf(1, '\n%%=================================================\n'); 256 | fprintf(1, '%% a sparse complex row vector\n'); 257 | fprintf(1, '%%=================================================\n\n'); 258 | 259 | data2hdf = data2hdf - 1i * data2hdf; 260 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 261 | h52data = loadh5('test.h5'); 262 | testequal(data2hdf, h52data.data2hdf); 263 | 264 | end 265 | 266 | fprintf(1, '\n%%=================================================\n'); 267 | fprintf(1, '%% a structure\n'); 268 | fprintf(1, '%%=================================================\n\n'); 269 | 270 | data2hdf = struct('name', 'Think Different', 'year', 1997, 'magic', magic(3), ... 271 | 'misfits', [Inf, NaN], 'embedded', struct('left', true, 'right', false)); 272 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 273 | h52data = loadh5('test.h5', 'regroup', 1); 274 | testequal(data2hdf, h52data.data2hdf); 275 | 276 | fprintf(1, '\n%%=================================================\n'); 277 | fprintf(1, '%% a structure array\n'); 278 | fprintf(1, '%%=================================================\n\n'); 279 | 280 | data2hdf = struct('name', 'Nexus Prime', 'rank', 9); 281 | data2hdf(2) = struct('name', 'Sentinel Prime', 'rank', 9); 282 | data2hdf(3) = struct('name', 'Optimus Prime', 'rank', 9); 283 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 284 | h52data = loadh5('test.h5', 'regroup', 1); 285 | testequal(data2hdf, h52data.data2hdf); 286 | 287 | fprintf(1, '\n%%=================================================\n'); 288 | fprintf(1, '%% a cell array\n'); 289 | fprintf(1, '%%=================================================\n\n'); 290 | 291 | data2hdf = cell(3, 1); 292 | data2hdf{1} = struct('buzz', 1.1, 'rex', 1.2, 'bo', 1.3, 'hamm', 2.0, 'slink', 2.1, 'potato', 2.2, ... 293 | 'woody', 3.0, 'sarge', 3.1, 'etch', 4.0, 'lenny', 5.0, 'squeeze', 6.0, 'wheezy', 7.0); 294 | data2hdf{2} = struct('Ubuntu', ['Kubuntu'; 'Xubuntu'; 'Lubuntu']); 295 | data2hdf{3} = [10.04, 10.10, 11.04, 11.10]; 296 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 297 | h52data = loadh5('test.h5', 'regroup', 1); 298 | testequal(data2hdf, h52data.data2hdf); 299 | 300 | if (~opt.isoctave) 301 | 302 | fprintf(1, '\n%%=================================================\n'); 303 | fprintf(1, '%% a function handle\n'); 304 | fprintf(1, '%%=================================================\n\n'); 305 | 306 | data2hdf = @(x) x + 1; 307 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 308 | h52data = loadh5('test.h5'); 309 | testequal(data2hdf, h52data.data2hdf); 310 | 311 | end 312 | 313 | fprintf(1, '\n%%=================================================\n'); 314 | fprintf(1, '%% a 2D cell array\n'); 315 | fprintf(1, '%%=================================================\n\n'); 316 | 317 | data2hdf = {{1, {2, 3}}, {4, 5}, {6}; {7}, {8, 9}, {10}}; 318 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 319 | h52data = loadh5('test.h5', 'regroup', 1); % only saveh5 works for cell arrays, loadh5 has issues 320 | % testequal(data2hdf,h52data.data2hdf) 321 | 322 | fprintf(1, '\n%%=================================================\n'); 323 | fprintf(1, '%% a 2D struct array\n'); 324 | fprintf(1, '%%=================================================\n\n'); 325 | 326 | data2hdf = repmat(struct('idx', 0, 'data', 'structs'), [2, 3]); 327 | for i = 1:6 328 | data2hdf(i).idx = i; 329 | end 330 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 331 | h52data = loadh5('test.h5', 'regroup', 1); 332 | testequal(data2hdf, h52data.data2hdf); 333 | 334 | if (exist('datetime')) 335 | fprintf(1, '\n%%=================================================\n'); 336 | fprintf(1, '%% datetime object \n'); 337 | fprintf(1, '%%=================================================\n\n'); 338 | 339 | data2hdf = datetime({'8 April 2015', '9 May 2015'}, 'InputFormat', 'd MMMM yyyy'); 340 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 341 | h52data = loadh5('test.h5'); 342 | testequal(data2hdf, h52data.data2hdf); 343 | end 344 | 345 | if (exist('containers.Map')) 346 | fprintf(1, '\n%%=================================================\n'); 347 | fprintf(1, '%% a container.Maps object \n'); 348 | fprintf(1, '%%=================================================\n\n'); 349 | 350 | data2hdf = containers.Map({'Andy', 'William', 'Om'}, [21, 21, 22]); 351 | saveh5(data2hdf, 'test.h5'); 352 | h52data = loadh5('test.h5'); 353 | testequal(data2hdf, h52data.data2hdf); 354 | end 355 | 356 | if (exist('istable')) 357 | fprintf(1, '\n%%=================================================\n'); 358 | fprintf(1, '%% a table object \n'); 359 | fprintf(1, '%%=================================================\n\n'); 360 | 361 | Names = {'Andy', 'William', 'Om'}'; 362 | Age = [21, 21, 22]'; 363 | data2hdf = table(Names, Age); 364 | saveh5(data2hdf, 'test.h5'); % nestarray for 4-D or above is not working 365 | h52data = loadh5('test.h5'); 366 | testequal(data2hdf, h52data.data2hdf); 367 | end 368 | 369 | try 370 | val = zlibencode('test'); 371 | fprintf(1, '\n%%=================================================\n'); 372 | fprintf(1, '%% a 2-D array in compressed array format\n'); 373 | fprintf(1, '%%=================================================\n\n'); 374 | 375 | data2hdf = eye(10); 376 | data2hdf(20, 1) = 1; 377 | saveh5(data2hdf, 'test.h5', 'compression', 'deflate'); % nestarray for 4-D or above is not working 378 | h52data = loadh5('test.h5'); 379 | testequal(data2hdf, h52data.data2hdf); 380 | catch 381 | end 382 | 383 | rand ('state', rngstate); 384 | -------------------------------------------------------------------------------- /jdatadecode.m: -------------------------------------------------------------------------------- 1 | function newdata = jdatadecode(data, varargin) 2 | % 3 | % newdata=jdatadecode(data,opt,...) 4 | % 5 | % Convert all JData object (in the form of a struct array) into an array 6 | % (accepts JData objects loaded from either loadjson/loadubjson or 7 | % jsondecode for MATLAB R2016b or later) 8 | % 9 | % This function implements the JData Specification Draft 3 (Jun. 2020) 10 | % see https://github.com/NeuroJSON/jdata for details 11 | % 12 | % authors:Qianqian Fang (q.fang neu.edu) 13 | % 14 | % input: 15 | % data: a struct array. If data contains JData keywords in the first 16 | % level children, these fields are parsed and regrouped into a 17 | % data object (arrays, trees, graphs etc) based on JData 18 | % specification. The JData keywords are 19 | % "_ArrayType_", "_ArraySize_", "_ArrayData_" 20 | % "_ArrayIsSparse_", "_ArrayIsComplex_", 21 | % "_ArrayZipType_", "_ArrayZipSize", "_ArrayZipData_" 22 | % opt: (optional) a list of 'Param',value pairs for additional options 23 | % The supported options include 24 | % Recursive: [1|0] if set to 1, will apply the conversion to 25 | % every child; 0 to disable 26 | % Base64: [0|1] if set to 1, _ArrayZipData_ is assumed to 27 | % be encoded with base64 format and need to be 28 | % decoded first. This is needed for JSON but not 29 | % UBJSON data 30 | % Prefix: ['x0x5F'|'x'] for JData files loaded via loadjson/loadubjson, the 31 | % default JData keyword prefix is 'x0x5F'; if the 32 | % json file is loaded using matlab2018's 33 | % jsondecode(), the prefix is 'x'; this function 34 | % attempts to automatically determine the prefix; 35 | % for octave, the default value is an empty string ''. 36 | % FullArrayShape: [0|1] if set to 1, converting _ArrayShape_ 37 | % objects to full matrices, otherwise, stay sparse 38 | % MaxLinkLevel: [0|int] When expanding _DataLink_ pointers, 39 | % this sets the maximum level of recursion 40 | % FormatVersion: [2|float]: set the JSONLab output version; 41 | % since v2.0, JSONLab uses JData specification Draft 1 42 | % for output format, it is incompatible with all 43 | % previous releases; if old output is desired, 44 | % please set FormatVersion to 1 45 | % 46 | % output: 47 | % newdata: the converted data if the input data does contain a JData 48 | % structure; otherwise, the same as the input. 49 | % 50 | % examples: 51 | % obj={[],{'test'},true,struct('sparse',sparse(2,3),'magic',uint8(magic(5)))} 52 | % jdata=jdatadecode(jdataencode(obj)) 53 | % isequaln(obj,jdata) 54 | % 55 | % license: 56 | % BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 57 | % 58 | % -- this function is part of JSONLab toolbox (https://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) 59 | % 60 | 61 | newdata = data; 62 | opt = struct; 63 | if (nargin == 2) 64 | opt = varargin{1}; 65 | elseif (nargin > 2) 66 | opt = varargin2struct(varargin{:}); 67 | end 68 | opt.fullarrayshape = jsonopt('FullArrayShape', 0, opt); 69 | opt.maxlinklevel = jsonopt('MaxLinkLevel', 0, opt); 70 | 71 | %% process non-structure inputs 72 | if (~isstruct(data)) 73 | if (iscell(data)) 74 | newdata = cellfun(@(x) jdatadecode(x, opt), data, 'UniformOutput', false); 75 | elseif (isa(data, 'containers.Map')) 76 | newdata = containers.Map('KeyType', data.KeyType, 'ValueType', 'any'); 77 | names = data.keys; 78 | for i = 1:length(names) 79 | newdata(names{i}) = jdatadecode(data(names{i}), opt); 80 | end 81 | end 82 | return 83 | end 84 | 85 | %% assume the input is a struct below 86 | fn = fieldnames(data); 87 | len = length(data); 88 | needbase64 = jsonopt('Base64', 0, opt); 89 | format = jsonopt('FormatVersion', 2, opt); 90 | if (isoctavemesh) 91 | prefix = jsonopt('Prefix', '', opt); 92 | else 93 | prefix = jsonopt('Prefix', 'x0x5F', opt); 94 | end 95 | if (~isfield(data, N_('_ArrayType_')) && isfield(data, 'x_ArrayType_')) 96 | prefix = 'x'; 97 | opt.prefix = 'x'; 98 | end 99 | 100 | %% recursively process subfields 101 | if (jsonopt('Recursive', 1, opt) == 1) 102 | for i = 1:length(fn) % depth-first 103 | for j = 1:len 104 | if (isstruct(data(j).(fn{i})) || isa(data(j).(fn{i}), 'containers.Map')) 105 | newdata(j).(fn{i}) = jdatadecode(data(j).(fn{i}), opt); 106 | elseif (iscell(data(j).(fn{i}))) 107 | newdata(j).(fn{i}) = cellfun(@(x) jdatadecode(x, opt), newdata(j).(fn{i}), 'UniformOutput', false); 108 | end 109 | end 110 | end 111 | end 112 | 113 | %% handle array data 114 | if (isfield(data, N_('_ArrayType_')) && (isfield(data, N_('_ArrayData_')) || (isfield(data, N_('_ArrayZipData_')) && ~isstruct(data.(N_('_ArrayZipData_')))))) 115 | newdata = cell(len, 1); 116 | for j = 1:len 117 | if (isfield(data, N_('_ArrayZipSize_')) && isfield(data, N_('_ArrayZipData_'))) 118 | zipmethod = 'zip'; 119 | if (isstruct(data(j).(N_('_ArrayZipSize_')))) 120 | data(j).(N_('_ArrayZipSize_')) = jdatadecode(data(j).(N_('_ArrayZipSize_')), opt); 121 | end 122 | dims = data(j).(N_('_ArrayZipSize_'))(:)'; 123 | if (length(dims) == 1) 124 | dims = [1 dims]; 125 | end 126 | if (isfield(data, N_('_ArrayZipType_'))) 127 | zipmethod = data(j).(N_('_ArrayZipType_')); 128 | end 129 | if (ismember(zipmethod, {'zlib', 'gzip', 'lzma', 'lzip', 'lz4', 'lz4hc', 'base64'}) || ~isempty(regexp(zipmethod, '^blosc2', 'once'))) 130 | decodeparam = {}; 131 | if (~isempty(regexp(zipmethod, '^blosc2', 'once'))) 132 | decompfun = @blosc2decode; 133 | decodeparam = {zipmethod}; 134 | else 135 | decompfun = str2func([zipmethod 'decode']); 136 | end 137 | arraytype = data(j).(N_('_ArrayType_')); 138 | chartype = 0; 139 | if (strcmp(arraytype, 'char') || strcmp(arraytype, 'logical')) 140 | chartype = 1; 141 | arraytype = 'uint8'; 142 | end 143 | if (needbase64 && strcmp(zipmethod, 'base64') == 0) 144 | ndata = reshape(typecast(decompfun(base64decode(data(j).(N_('_ArrayZipData_'))), decodeparam{:}), arraytype), dims); 145 | else 146 | ndata = reshape(typecast(decompfun(data(j).(N_('_ArrayZipData_')), decodeparam{:}), arraytype), dims); 147 | end 148 | if (chartype) 149 | ndata = char(ndata); 150 | end 151 | else 152 | error('compression method is not supported'); 153 | end 154 | else 155 | if (isstruct(data(j).(N_('_ArrayData_')))) 156 | data(j).(N_('_ArrayData_')) = jdatadecode(data(j).(N_('_ArrayData_')), opt); 157 | end 158 | if (isstruct(data(j).(N_('_ArrayData_'))) && isfield(data(j).(N_('_ArrayData_')), N_('_ArrayType_'))) 159 | data(j).(N_('_ArrayData_')) = jdatadecode(data(j).(N_('_ArrayData_')), varargin{:}); 160 | end 161 | if (iscell(data(j).(N_('_ArrayData_')))) 162 | data(j).(N_('_ArrayData_')) = cell2mat(cellfun(@(x) double(x(:)), data(j).(N_('_ArrayData_')), 'uniformoutput', 0)).'; 163 | end 164 | ndata = cast(data(j).(N_('_ArrayData_')), char(data(j).(N_('_ArrayType_')))); 165 | end 166 | if (isfield(data, N_('_ArrayZipSize_'))) 167 | if (isstruct(data(j).(N_('_ArrayZipSize_')))) 168 | data(j).(N_('_ArrayZipSize_')) = jdatadecode(data(j).(N_('_ArrayZipSize_')), opt); 169 | end 170 | dims = data(j).(N_('_ArrayZipSize_'))(:)'; 171 | if (iscell(dims)) 172 | dims = cell2mat(dims); 173 | end 174 | if (length(dims) == 1) 175 | dims = [1 dims]; 176 | end 177 | ndata = reshape(ndata(:), fliplr(dims)); 178 | ndata = permute(ndata, ndims(ndata):-1:1); 179 | end 180 | iscpx = 0; 181 | if (isfield(data, N_('_ArrayIsComplex_')) && isstruct(data(j).(N_('_ArrayIsComplex_')))) 182 | data(j).(N_('_ArrayIsComplex_')) = jdatadecode(data(j).(N_('_ArrayIsComplex_')), opt); 183 | end 184 | if (isfield(data, N_('_ArrayIsComplex_')) && data(j).(N_('_ArrayIsComplex_'))) 185 | iscpx = 1; 186 | end 187 | iscol = 0; 188 | if (isfield(data, N_('_ArrayOrder_'))) 189 | arrayorder = data(j).(N_('_ArrayOrder_')); 190 | if (~isempty(arrayorder) && (arrayorder(1) == 'c' || arrayorder(1) == 'C')) 191 | iscol = 1; 192 | end 193 | end 194 | if (isfield(data, N_('_ArrayIsSparse_')) && isstruct(data(j).(N_('_ArrayIsSparse_')))) 195 | data(j).(N_('_ArrayIsSparse_')) = jdatadecode(data(j).(N_('_ArrayIsSparse_')), opt); 196 | end 197 | if (isfield(data, N_('_ArrayIsSparse_')) && data(j).(N_('_ArrayIsSparse_'))) 198 | if (isfield(data, N_('_ArraySize_'))) 199 | if (isstruct(data(j).(N_('_ArraySize_')))) 200 | data(j).(N_('_ArraySize_')) = jdatadecode(data(j).(N_('_ArraySize_')), opt); 201 | end 202 | dim = data(j).(N_('_ArraySize_'))(:)'; 203 | if (iscell(dim)) 204 | dim = cell2mat(dim); 205 | end 206 | dim = double(dim); 207 | if (length(dim) == 1) 208 | dim = [1 dim]; 209 | end 210 | if (iscpx) 211 | ndata(end - 1, :) = complex(ndata(end - 1, :), ndata(end, :)); 212 | end 213 | if isempty(ndata) 214 | % All-zeros sparse 215 | ndata = sparse(dim(1), prod(dim(2:end))); 216 | elseif dim(1) == 1 217 | % Sparse row vector 218 | ndata = sparse(1, ndata(1, :), ndata(2, :), dim(1), prod(dim(2:end))); 219 | elseif dim(2) == 1 220 | % Sparse column vector 221 | ndata = sparse(ndata(1, :), 1, ndata(2, :), dim(1), prod(dim(2:end))); 222 | else 223 | % Generic sparse array. 224 | ndata = sparse(ndata(1, :), ndata(2, :), ndata(3, :), dim(1), prod(dim(2:end))); 225 | end 226 | else 227 | if (iscpx && size(ndata, 2) == 4) 228 | ndata(3, :) = complex(ndata(3, :), ndata(4, :)); 229 | end 230 | ndata = sparse(ndata(1, :), ndata(2, :), ndata(3, :)); 231 | end 232 | elseif (isfield(data, N_('_ArrayShape_'))) 233 | if (isstruct(data(j).(N_('_ArrayShape_')))) 234 | data(j).(N_('_ArrayShape_')) = jdatadecode(data(j).(N_('_ArrayShape_')), opt); 235 | end 236 | if (iscpx) 237 | if (size(ndata, 1) == 2) 238 | dim = size(ndata); 239 | dim(end + 1) = 1; 240 | arraydata = reshape(complex(ndata(1, :), ndata(2, :)), dim(2:end)); 241 | else 242 | error('The first dimension must be 2 for complex-valued arrays'); 243 | end 244 | else 245 | arraydata = data.(N_('_ArrayData_')); 246 | end 247 | shapeid = data.(N_('_ArrayShape_')); 248 | if (isfield(data, N_('_ArrayZipSize_'))) 249 | datasize = data.(N_('_ArrayZipSize_')); 250 | if (iscell(datasize)) 251 | datasize = cell2mat(datasize); 252 | end 253 | datasize = double(datasize); 254 | if (iscpx) 255 | datasize = datasize(2:end); 256 | end 257 | else 258 | datasize = size(arraydata); 259 | end 260 | if (isstruct(data(j).(N_('_ArraySize_')))) 261 | data(j).(N_('_ArraySize_')) = jdatadecode(data(j).(N_('_ArraySize_')), opt); 262 | end 263 | arraysize = data.(N_('_ArraySize_')); 264 | 265 | if (iscell(arraysize)) 266 | arraysize = cell2mat(arraysize); 267 | end 268 | arraysize = double(arraysize); 269 | if (ischar(shapeid)) 270 | shapeid = {shapeid}; 271 | end 272 | arraydata = double(arraydata).'; 273 | if (strcmpi(shapeid{1}, 'diag')) 274 | ndata = spdiags(arraydata(:), 0, arraysize(1), arraysize(2)); 275 | elseif (strcmpi(shapeid{1}, 'upper') || strcmpi(shapeid{1}, 'uppersymm')) 276 | ndata = zeros(arraysize); 277 | ndata(triu(true(size(ndata)))') = arraydata(:); 278 | if (strcmpi(shapeid{1}, 'uppersymm')) 279 | ndata(triu(true(size(ndata)))) = arraydata(:); 280 | end 281 | ndata = ndata.'; 282 | elseif (strcmpi(shapeid{1}, 'lower') || strcmpi(shapeid{1}, 'lowersymm')) 283 | ndata = zeros(arraysize); 284 | ndata(tril(true(size(ndata)))') = arraydata(:); 285 | if (strcmpi(shapeid{1}, 'lowersymm')) 286 | ndata(tril(true(size(ndata)))) = arraydata(:); 287 | end 288 | ndata = ndata.'; 289 | elseif (strcmpi(shapeid{1}, 'upperband') || strcmpi(shapeid{1}, 'uppersymmband')) 290 | if (length(shapeid) > 1 && isvector(arraydata)) 291 | datasize = double([shapeid{2} + 1, prod(datasize) / (shapeid{2} + 1)]); 292 | end 293 | ndata = spdiags(reshape(arraydata, min(arraysize), datasize(1)), -datasize(1) + 1:0, arraysize(2), arraysize(1)).'; 294 | if (strcmpi(shapeid{1}, 'uppersymmband')) 295 | diagonal = diag(ndata); 296 | ndata = ndata + ndata.'; 297 | ndata(1:arraysize(1) + 1:end) = diagonal; 298 | end 299 | elseif (strcmpi(shapeid{1}, 'lowerband') || strcmpi(shapeid{1}, 'lowersymmband')) 300 | if (length(shapeid) > 1 && isvector(arraydata)) 301 | datasize = double([shapeid{2} + 1, prod(datasize) / (shapeid{2} + 1)]); 302 | end 303 | ndata = spdiags(reshape(arraydata, min(arraysize), datasize(1)), 0:datasize(1) - 1, arraysize(2), arraysize(1)).'; 304 | if (strcmpi(shapeid{1}, 'lowersymmband')) 305 | diagonal = diag(ndata); 306 | ndata = ndata + ndata.'; 307 | ndata(1:arraysize(1) + 1:end) = diagonal; 308 | end 309 | elseif (strcmpi(shapeid{1}, 'band')) 310 | if (length(shapeid) > 1 && isvector(arraydata)) 311 | datasize = double([shapeid{2} + shapeid{3} + 1, prod(datasize) / (shapeid{2} + shapeid{3} + 1)]); 312 | end 313 | ndata = spdiags(reshape(arraydata, min(arraysize), datasize(1)), double(shapeid{2}):-1:-double(shapeid{3}), arraysize(1), arraysize(2)); 314 | elseif (strcmpi(shapeid{1}, 'toeplitz')) 315 | arraydata = reshape(arraydata, flipud(datasize(:))'); 316 | ndata = toeplitz(arraydata(1:arraysize(1), 2), arraydata(1:arraysize(2), 1)); 317 | end 318 | if (opt.fullarrayshape && issparse(ndata)) 319 | ndata = cast(full(ndata), data(j).(N_('_ArrayType_'))); 320 | end 321 | elseif (isfield(data, N_('_ArraySize_'))) 322 | if (isstruct(data(j).(N_('_ArraySize_')))) 323 | data(j).(N_('_ArraySize_')) = jdatadecode(data(j).(N_('_ArraySize_')), opt); 324 | end 325 | if (iscpx) 326 | ndata = complex(ndata(1, :), ndata(2, :)); 327 | end 328 | if (format > 1.9 && iscol == 0) 329 | data(j).(N_('_ArraySize_')) = data(j).(N_('_ArraySize_'))(end:-1:1); 330 | end 331 | dims = data(j).(N_('_ArraySize_'))(:)'; 332 | if (iscell(dims)) 333 | dims = cell2mat(dims); 334 | end 335 | if (length(dims) == 1) 336 | dims = [1 dims]; 337 | end 338 | ndata = reshape(ndata(:), dims(:)'); 339 | if (format > 1.9 && iscol == 0) 340 | ndata = permute(ndata, ndims(ndata):-1:1); 341 | end 342 | end 343 | newdata{j} = ndata; 344 | end 345 | if (len == 1) 346 | newdata = newdata{1}; 347 | end 348 | end 349 | 350 | %% handle table data 351 | if (isfield(data, N_('_TableRecords_'))) 352 | newdata = cell(len, 1); 353 | for j = 1:len 354 | ndata = data(j).(N_('_TableRecords_')); 355 | if (iscell(ndata)) 356 | if (iscell(ndata{1})) 357 | rownum = length(ndata); 358 | colnum = length(ndata{1}); 359 | nd = cell(rownum, colnum); 360 | for i1 = 1:rownum 361 | for i2 = 1:colnum 362 | nd{i1, i2} = ndata{i1}{i2}; 363 | end 364 | end 365 | newdata{j} = cell2table(nd); 366 | else 367 | newdata{j} = cell2table(ndata); 368 | end 369 | else 370 | newdata{j} = array2table(ndata); 371 | end 372 | if (isfield(data(j), N_('_TableRows_')) && ~isempty(data(j).(N_('_TableRows_')))) 373 | newdata{j}.Properties.RowNames = data(j).(N_('_TableRows_'))(:); 374 | end 375 | if (isfield(data(j), N_('_TableCols_')) && ~isempty(data(j).(N_('_TableCols_')))) 376 | newdata{j}.Properties.VariableNames = data(j).(N_('_TableCols_')); 377 | end 378 | end 379 | if (len == 1) 380 | newdata = newdata{1}; 381 | end 382 | end 383 | 384 | %% handle map data 385 | if (isfield(data, N_('_MapData_'))) 386 | newdata = cell(len, 1); 387 | for j = 1:len 388 | key = cell(1, length(data(j).(N_('_MapData_')))); 389 | val = cell(size(key)); 390 | for k = 1:length(data(j).(N_('_MapData_'))) 391 | key{k} = data(j).(N_('_MapData_')){k}{1}; 392 | val{k} = jdatadecode(data(j).(N_('_MapData_')){k}{2}, opt); 393 | end 394 | ndata = containers.Map(key, val); 395 | newdata{j} = ndata; 396 | end 397 | if (len == 1) 398 | newdata = newdata{1}; 399 | end 400 | end 401 | 402 | %% handle graph data 403 | if (isfield(data, N_('_GraphNodes_')) && exist('graph', 'file') && exist('digraph', 'file')) 404 | newdata = cell(len, 1); 405 | isdirected = 1; 406 | for j = 1:len 407 | nodedata = data(j).(N_('_GraphNodes_')); 408 | if (isstruct(nodedata)) 409 | nodetable = struct2table(nodedata); 410 | elseif (isa(nodedata, 'containers.Map')) 411 | nodetable = [keys(nodedata); values(nodedata)]; 412 | if (strcmp(nodedata.KeyType, 'char')) 413 | nodetable = table(nodetable(1, :)', nodetable(2, :)', 'VariableNames', {'Name', 'Data'}); 414 | else 415 | nodetable = table(nodetable(2, :)', 'VariableNames', {'Data'}); 416 | end 417 | else 418 | nodetable = table; 419 | end 420 | 421 | if (isfield(data, N_('_GraphEdges_'))) 422 | edgedata = data(j).(N_('_GraphEdges_')); 423 | elseif (isfield(data, N_('_GraphEdges0_'))) 424 | edgedata = data(j).(N_('_GraphEdges0_')); 425 | isdirected = 0; 426 | elseif (isfield(data, N_('_GraphMatrix_'))) 427 | edgedata = jdatadecode(data(j).(N_('_GraphMatrix_')), varargin{:}); 428 | end 429 | 430 | if (exist('edgedata', 'var')) 431 | if (iscell(edgedata)) 432 | endnodes = edgedata(:, 1:2); 433 | endnodes = reshape([endnodes{:}], size(edgedata, 1), 2); 434 | weight = cell2mat(edgedata(:, 3:end)); 435 | edgetable = table(endnodes, [weight.Weight]', 'VariableNames', {'EndNodes', 'Weight'}); 436 | 437 | if (isdirected) 438 | newdata{j} = digraph(edgetable, nodetable); 439 | else 440 | newdata{j} = graph(edgetable, nodetable); 441 | end 442 | elseif (ndims(edgedata) == 2 && isstruct(nodetable)) 443 | newdata{j} = digraph(edgedata, fieldnames(nodetable)); 444 | end 445 | end 446 | end 447 | if (len == 1) 448 | newdata = newdata{1}; 449 | end 450 | end 451 | 452 | %% handle bytestream and arbitrary matlab objects 453 | if (isfield(data, N_('_ByteStream_')) && isfield(data, N_('_DataInfo_'))) 454 | newdata = cell(len, 1); 455 | for j = 1:len 456 | if (isfield(data(j).(N_('_DataInfo_')), 'MATLABObjectClass')) 457 | if (needbase64) 458 | newdata{j} = getArrayFromByteStream(base64decode(data(j).(N_('_ByteStream_')))); 459 | else 460 | newdata{j} = getArrayFromByteStream(data(j).(N_('_ByteStream_'))); 461 | end 462 | end 463 | end 464 | if (len == 1) 465 | newdata = newdata{1}; 466 | end 467 | end 468 | 469 | %% handle data link 470 | if (opt.maxlinklevel > 0 && isfield(data, N_('_DataLink_'))) 471 | if (ischar(data.(N_('_DataLink_')))) 472 | datalink = data.(N_('_DataLink_')); 473 | if (regexp(datalink, '\:\$')) 474 | ref = regexp(datalink, '^(?[a-zA-Z]+://)*(?.+)(?\:)()*(?(?<=:)\$\d*\.*.*)*', 'names'); 475 | else 476 | ref = regexp(datalink, '^(?[a-zA-Z]+://)*(?.+)(?\:)*(?(?<=:)\$\d*\..*)*', 'names'); 477 | end 478 | if (~isempty(ref.path)) 479 | uripath = [ref.proto ref.path]; 480 | [newdata, fname] = jdlink(uripath); 481 | if (exist(fname, 'file')) 482 | opt.maxlinklevel = opt.maxlinklevel - 1; 483 | if (~isempty(ref.jsonpath)) 484 | newdata = jsonpath(newdata, ref.jsonpath); 485 | end 486 | end 487 | end 488 | end 489 | end 490 | 491 | %% subfunctions 492 | function escaped = N_(str) 493 | escaped = [prefix str]; 494 | end 495 | 496 | end 497 | -------------------------------------------------------------------------------- /jdataencode.m: -------------------------------------------------------------------------------- 1 | function jdata = jdataencode(data, varargin) 2 | % 3 | % jdata=jdataencode(data) 4 | % or 5 | % jdata=jdataencode(data, options) 6 | % jdata=jdataencode(data, 'Param1',value1, 'Param2',value2,...) 7 | % 8 | % Annotate a MATLAB struct or cell array into a JData-compliant data 9 | % structure as defined in the JData spec: https://github.com/NeuroJSON/jdata. 10 | % This encoded form servers as an intermediate format that allows unambiguous 11 | % storage, exchange of complex data structures and easy-to-serialize by 12 | % json encoders such as savejson and jsonencode (MATLAB R2016b or newer) 13 | % 14 | % This function implements the JData Specification Draft 3 (Jun. 2020) 15 | % see https://github.com/NeuroJSON/jdata for details 16 | % 17 | % author: Qianqian Fang (q.fang neu.edu) 18 | % 19 | % input: 20 | % data: a structure (array) or cell (array) to be encoded. 21 | % options: (optional) a struct or Param/value pairs for user 22 | % specified options (first in [.|.] is the default) 23 | % AnnotateArray: [0|1] - if set to 1, convert all 1D/2D matrices 24 | % to the annotated JData array format to preserve data types; 25 | % N-D (N>2), complex and sparse arrays are encoded using the 26 | % annotated format by default. Please set this option to 1 if 27 | % you intend to use MATLAB's jsonencode to convert to JSON. 28 | % Base64: [0|1] if set to 1, _ArrayZipData_ is assumed to 29 | % be encoded with base64 format and need to be 30 | % decoded first. This is needed for JSON but not 31 | % UBJSON data 32 | % Prefix: ['x0x5F'|'x'] for JData files loaded via loadjson/loadubjson, the 33 | % default JData keyword prefix is 'x0x5F'; if the 34 | % json file is loaded using matlab2018's 35 | % jsondecode(), the prefix is 'x'; this function 36 | % attempts to automatically determine the prefix; 37 | % for octave, the default value is an empty string ''. 38 | % UseArrayZipSize: [1|0] if set to 1, _ArrayZipSize_ will be added to 39 | % store the "pre-processed" data dimensions, i.e. 40 | % the original data stored in _ArrayData_, and then flaten 41 | % _ArrayData_ into a row vector using row-major 42 | % order; if set to 0, a 2D _ArrayData_ will be used 43 | % UseArrayShape: [0|1] if set to 1, a matrix will be tested by 44 | % to determine if it is diagonal, triangular, banded or 45 | % toeplitz, and use _ArrayShape_ to encode the matrix 46 | % MapAsStruct: [0|1] if set to 1, convert containers.Map into 47 | % struct; otherwise, keep it as map 48 | % Compression: ['zlib'|'gzip','lzma','lz4','lz4hc'] - use zlib method 49 | % to compress data array 50 | % CompressArraySize: [100|int]: only to compress an array if the 51 | % total element count is larger than this number. 52 | % FormatVersion [2|float]: set the JSONLab output version; since 53 | % v2.0, JSONLab uses JData specification Draft 1 54 | % for output format, it is incompatible with all 55 | % previous releases; if old output is desired, 56 | % please set FormatVersion to 1.9 or earlier. 57 | % 58 | % example: 59 | % jd=jdataencode(struct('a',rand(5)+1i*rand(5),'b',[],'c',sparse(5,5))) 60 | % 61 | % encodedmat=jdataencode(single(magic(5)),'annotatearray',1,'prefix','x') 62 | % jdatadecode(jsondecode(jsonencode(encodedmat))) % serialize by jsonencode 63 | % jdatadecode(loadjson(savejson('',encodedmat))) % serialize by savejson 64 | % 65 | % encodedtoeplitz=jdataencode(uint8(toeplitz([1,2,3,4],[1,5,6])),'usearrayshape',1,'prefix','x') 66 | % jdatadecode(jsondecode(jsonencode(encodedtoeplitz))) % serialize by jsonencode 67 | % jdatadecode(loadjson(savejson('',encodedtoeplitz))) % serialize by savejson 68 | % 69 | % license: 70 | % BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 71 | % 72 | % -- this function is part of JSONLab toolbox (https://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) 73 | % 74 | 75 | if (nargin == 0) 76 | help jdataencode; 77 | return 78 | end 79 | 80 | opt = varargin2struct(varargin{:}); 81 | if (isoctavemesh) 82 | opt.prefix = jsonopt('Prefix', '', opt); 83 | else 84 | opt.prefix = jsonopt('Prefix', sprintf('x0x%X', '_' + 0), opt); 85 | end 86 | opt.compression = jsonopt('Compression', '', opt); 87 | opt.nestarray = jsonopt('NestArray', 0, opt); 88 | opt.formatversion = jsonopt('FormatVersion', 2, opt); 89 | opt.compressarraysize = jsonopt('CompressArraySize', 100, opt); 90 | opt.base64 = jsonopt('Base64', 0, opt); 91 | opt.mapasstruct = jsonopt('MapAsStruct', 0, opt); 92 | opt.usearrayzipsize = jsonopt('UseArrayZipSize', 1, opt); 93 | opt.messagepack = jsonopt('MessagePack', 0, opt); 94 | opt.usearrayshape = jsonopt('UseArrayShape', 0, opt) && exist('bandwidth'); 95 | opt.annotatearray = jsonopt('AnnotateArray', 0, opt); 96 | 97 | jdata = obj2jd(data, opt); 98 | 99 | %% ------------------------------------------------------------------------- 100 | function newitem = obj2jd(item, varargin) 101 | 102 | if (iscell(item)) 103 | newitem = cell2jd(item, varargin{:}); 104 | elseif (isa(item, 'jdict')) 105 | newitem = obj2jd(item(), varargin{:}); 106 | elseif (isstruct(item)) 107 | newitem = struct2jd(item, varargin{:}); 108 | elseif (isnumeric(item) || islogical(item) || isa(item, 'timeseries')) 109 | newitem = mat2jd(item, varargin{:}); 110 | elseif (ischar(item) || isa(item, 'string')) 111 | newitem = mat2jd(item, varargin{:}); 112 | elseif (isa(item, 'containers.Map') || isa(item, 'dictionary')) 113 | newitem = map2jd(item, varargin{:}); 114 | elseif (isa(item, 'categorical')) 115 | newitem = cell2jd(cellstr(item), varargin{:}); 116 | elseif (isa(item, 'function_handle')) 117 | newitem = struct2jd(functions(item), varargin{:}); 118 | elseif (isa(item, 'table')) 119 | newitem = table2jd(item, varargin{:}); 120 | elseif (isa(item, 'digraph') || isa(item, 'graph')) 121 | newitem = graph2jd(item, varargin{:}); 122 | elseif (isobject(item)) 123 | newitem = matlabobject2jd(item, varargin{:}); 124 | else 125 | newitem = item; 126 | end 127 | 128 | %% ------------------------------------------------------------------------- 129 | function newitem = cell2jd(item, varargin) 130 | 131 | newitem = cellfun(@(x) obj2jd(x, varargin{:}), item, 'UniformOutput', false); 132 | 133 | %% ------------------------------------------------------------------------- 134 | function newitem = struct2jd(item, varargin) 135 | 136 | num = numel(item); 137 | if (num > 1) % struct array 138 | newitem = obj2jd(num2cell(item), varargin{:}); 139 | try 140 | newitem = cell2mat(newitem); 141 | catch 142 | end 143 | elseif (num == 1) % a single struct 144 | names = fieldnames(item); 145 | newitem = struct; 146 | for i = 1:length(names) 147 | newitem.(names{i}) = obj2jd(item.(names{i}), varargin{:}); 148 | end 149 | else 150 | newitem = item; 151 | end 152 | 153 | %% ------------------------------------------------------------------------- 154 | function newitem = map2jd(item, varargin) 155 | 156 | names = item.keys; 157 | if (varargin{1}.mapasstruct) % convert a map to struct 158 | newitem = struct; 159 | if (~strcmp(item.KeyType, 'char')) 160 | data = num2cell(reshape([names, item.values], length(names), 2), 2); 161 | for i = 1:length(names) 162 | data{i}{2} = obj2jd(data{i}{2}, varargin{:}); 163 | end 164 | newitem.(N_('_MapData_', varargin{:})) = data; 165 | else 166 | for i = 1:length(names) 167 | newitem.(N_(names{i}, varargin{:})) = obj2jd(item(names{i}), varargin{:}); 168 | end 169 | end 170 | else % keep as a map and only encode its values 171 | if (isa(item, 'dictionary')) 172 | newitem = dictionary(); 173 | elseif (strcmp(item.KeyType, 'char')) 174 | newitem = containers.Map(); 175 | else 176 | newitem = containers.Map('KeyType', item.KeyType, 'ValueType', 'any'); 177 | end 178 | if (isa(item, 'dictionary')) 179 | for i = 1:length(names) 180 | newitem(names(i)) = obj2jd(item(names(i)), varargin{:}); 181 | end 182 | else 183 | for i = 1:length(names) 184 | newitem(names{i}) = obj2jd(item(names{i}), varargin{:}); 185 | end 186 | end 187 | end 188 | 189 | %% ------------------------------------------------------------------------- 190 | function newitem = mat2jd(item, varargin) 191 | 192 | N = @(x) N_(x, varargin{:}); 193 | newitem = struct(N('_ArrayType_'), class(item), N('_ArraySize_'), size(item)); 194 | 195 | zipmethod = varargin{1}.compression; 196 | minsize = varargin{1}.compressarraysize; 197 | 198 | if (isa(item, 'timeseries')) 199 | if (item.TimeInfo.isUniform && item.TimeInfo.Increment == 1) 200 | if (ndims(item.Data) == 3 && size(item.Data, 1) == 1 && size(item.Data, 2) == 1) 201 | item = permute(item.Data, [2 3 1]); 202 | else 203 | item = squeeze(item.Data); 204 | end 205 | else 206 | item = [item.Time squeeze(item.Data)]; 207 | end 208 | end 209 | 210 | % 2d numerical (real/complex/sparse) arrays with _ArrayShape_ encoding enabled 211 | if (varargin{1}.usearrayshape && ndims(item) == 2 && ~isvector(item)) 212 | encoded = 1; 213 | if (~isreal(item)) 214 | newitem.(N('_ArrayIsComplex_')) = true; 215 | end 216 | symmtag = ''; 217 | if (isreal(item) && issymmetric(double(item))) 218 | symmtag = 'symm'; 219 | item = tril(item); 220 | elseif (~isreal(item) && ishermitian(double(item))) 221 | symmtag = 'herm'; 222 | item = tril(item); 223 | end 224 | [lband, uband] = bandwidth(double(item)); 225 | newitem.(N('_ArrayZipSize_')) = [lband + uband + 1, min(size(item, 1), size(item, 2))]; 226 | if (lband + uband == 0) % isdiag 227 | newitem.(N('_ArrayShape_')) = 'diag'; 228 | newitem.(N('_ArrayData_')) = diag(item).'; 229 | elseif (uband == 0 && lband == size(item, 1) - 1) % lower triangular 230 | newitem.(N('_ArrayShape_')) = ['lower' symmtag]; 231 | item = item.'; 232 | newitem.(N('_ArrayData_')) = item(triu(true(size(item)))).'; 233 | elseif (lband == 0 && uband == size(item, 2) - 1) % upper triangular 234 | newitem.(N('_ArrayShape_')) = 'upper'; 235 | item = item.'; 236 | newitem.(N('_ArrayData_')) = item(tril(true(size(item)))).'; 237 | elseif (lband == 0) % upper band 238 | newitem.(N('_ArrayShape_')) = {'upperband', uband}; 239 | newitem.(N('_ArrayData_')) = spdiags(item.', -uband:lband).'; 240 | elseif (uband == 0) % lower band 241 | newitem.(N('_ArrayShape_')) = {sprintf('lower%sband', symmtag), lband}; 242 | newitem.(N('_ArrayData_')) = spdiags(item.', -uband:lband).'; 243 | elseif (uband < size(item, 2) - 1 || lband < size(item, 1) - 1) % band 244 | newitem.(N('_ArrayShape_')) = {'band', uband, lband}; 245 | newitem.(N('_ArrayData_')) = spdiags(item.', -uband:lband).'; 246 | elseif (all(toeplitz(item(:, 1), item(1, :)) == item)) % Toeplitz matrix 247 | newitem.(N('_ArrayShape_')) = 'toeplitz'; 248 | newitem.(N('_ArrayZipSize_')) = [2, max(size(item))]; 249 | newitem.(N('_ArrayData_')) = zeros(2, max(size(item))); 250 | newitem.(N('_ArrayData_'))(1, 1:size(item, 2)) = item(1, :); 251 | newitem.(N('_ArrayData_'))(2, 1:size(item, 1)) = item(:, 1).'; 252 | else % full matrix 253 | newitem = rmfield(newitem, N('_ArrayZipSize_')); 254 | encoded = 0; 255 | end 256 | 257 | % serialize complex data at last 258 | if (encoded && isstruct(newitem) && ~isreal(newitem.(N('_ArrayData_')))) 259 | item = squeeze(zeros([2, size(newitem.(N('_ArrayData_')))])); 260 | item(1, :) = real(newitem.(N('_ArrayData_'))(:)); 261 | item(2, :) = imag(newitem.(N('_ArrayData_'))(:)); 262 | newitem.(N('_ArrayZipSize_')) = size(item); 263 | newitem.(N('_ArrayData_')) = item; 264 | end 265 | 266 | % wrap _ArrayData_ into a single row vector, and store preprocessed 267 | % size to _ArrayZipSize_ (force varargin{1}.usearrayzipsize=true) 268 | if (encoded) 269 | if (isstruct(newitem) && ~isvector(newitem.(N('_ArrayData_')))) 270 | item = newitem.(N('_ArrayData_')); 271 | item = permute(item, ndims(item):-1:1); 272 | newitem.(N('_ArrayData_')) = item(:).'; 273 | else 274 | newitem = rmfield(newitem, N('_ArrayZipSize_')); 275 | end 276 | newitem.(N('_ArrayData_')) = full(newitem.(N('_ArrayData_'))); 277 | return 278 | end 279 | end 280 | 281 | % no encoding for char arrays or non-sparse real vectors 282 | if (isempty(item) || isa(item, 'string') || ischar(item) || varargin{1}.nestarray || ... 283 | ((isvector(item) || ndims(item) == 2) && isreal(item) && ~issparse(item) && ... 284 | ~varargin{1}.annotatearray)) 285 | newitem = item; 286 | return 287 | end 288 | 289 | if (isa(item, 'logical')) 290 | item = uint8(item); 291 | end 292 | 293 | if (isreal(item)) 294 | if (issparse(item)) 295 | fulldata = full(item(item ~= 0)); 296 | newitem.(N('_ArrayIsSparse_')) = true; 297 | newitem.(N('_ArrayZipSize_')) = [2 + (~isvector(item)), length(fulldata)]; 298 | if (isvector(item)) 299 | newitem.(N('_ArrayData_')) = [find(item(:))', fulldata(:)']; 300 | else 301 | [ix, iy] = find(item); 302 | newitem.(N('_ArrayData_')) = [ix(:)', iy(:)', fulldata(:)']; 303 | end 304 | else 305 | if (varargin{1}.formatversion > 1.9) 306 | item = permute(item, ndims(item):-1:1); 307 | end 308 | newitem.(N('_ArrayData_')) = item(:)'; 309 | end 310 | else 311 | newitem.(N('_ArrayIsComplex_')) = true; 312 | if (issparse(item)) 313 | fulldata = full(item(item ~= 0)); 314 | newitem.(N('_ArrayIsSparse_')) = true; 315 | newitem.(N('_ArrayZipSize_')) = [3 + (~isvector(item)), length(fulldata)]; 316 | if (isvector(item)) 317 | newitem.(N('_ArrayData_')) = [find(item(:))', real(fulldata(:))', imag(fulldata(:))']; 318 | else 319 | [ix, iy] = find(item); 320 | newitem.(N('_ArrayData_')) = [ix(:)', iy(:)', real(fulldata(:))', imag(fulldata(:))']; 321 | end 322 | else 323 | if (varargin{1}.formatversion > 1.9) 324 | item = permute(item, ndims(item):-1:1); 325 | end 326 | newitem.(N('_ArrayZipSize_')) = [2, numel(item)]; 327 | newitem.(N('_ArrayData_')) = [real(item(:))', imag(item(:))']; 328 | end 329 | end 330 | 331 | if (varargin{1}.usearrayzipsize == 0 && isfield(newitem, N('_ArrayZipSize_'))) 332 | data = newitem.(N('_ArrayData_')); 333 | data = reshape(data, fliplr(newitem.(N('_ArrayZipSize_')))); 334 | newitem.(N('_ArrayData_')) = permute(data, ndims(data):-1:1); 335 | newitem = rmfield(newitem, N('_ArrayZipSize_')); 336 | end 337 | 338 | if (~isempty(zipmethod) && numel(item) > minsize) 339 | encodeparam = {}; 340 | if (~isempty(regexp(zipmethod, '^blosc2', 'once'))) 341 | compfun = @blosc2encode; 342 | encodeparam = {zipmethod, 'nthread', jsonopt('nthread', 1, varargin{1}), ... 343 | 'shuffle', jsonopt('shuffle', 1, varargin{1}), ... 344 | 'typesize', jsonopt('typesize', length(typecast(item(1), 'uint8')), varargin{1})}; 345 | else 346 | compfun = str2func([zipmethod 'encode']); 347 | end 348 | newitem.(N('_ArrayZipType_')) = lower(zipmethod); 349 | if (~isfield(newitem, N('_ArrayZipSize_'))) 350 | newitem.(N('_ArrayZipSize_')) = size(newitem.(N('_ArrayData_'))); 351 | end 352 | newitem.(N('_ArrayZipData_')) = compfun(typecast(newitem.(N('_ArrayData_'))(:).', 'uint8'), encodeparam{:}); 353 | newitem = rmfield(newitem, N('_ArrayData_')); 354 | if (varargin{1}.base64) 355 | newitem.(N('_ArrayZipData_')) = char(base64encode(newitem.(N('_ArrayZipData_')))); 356 | end 357 | end 358 | 359 | if (isfield(newitem, N('_ArrayData_')) && isempty(newitem.(N('_ArrayData_')))) 360 | newitem.(N('_ArrayData_')) = []; 361 | end 362 | 363 | %% ------------------------------------------------------------------------- 364 | function newitem = table2jd(item, varargin) 365 | 366 | newitem = struct; 367 | newitem.(N_('_TableCols_', varargin{:})) = item.Properties.VariableNames; 368 | newitem.(N_('_TableRows_', varargin{:})) = item.Properties.RowNames'; 369 | newitem.(N_('_TableRecords_', varargin{:})) = table2cell(item); 370 | 371 | %% ------------------------------------------------------------------------- 372 | function newitem = graph2jd(item, varargin) 373 | 374 | newitem = struct; 375 | nodedata = table2struct(item.Nodes); 376 | if (isfield(nodedata, 'Name')) 377 | nodedata = rmfield(nodedata, 'Name'); 378 | newitem.(N_('_GraphNodes_', varargin{:})) = containers.Map(item.Nodes.Name, num2cell(nodedata), 'UniformValues', false); 379 | else 380 | newitem.(N_('_GraphNodes_', varargin{:})) = containers.Map(1:max(item.Edges.EndNodes(:)), num2cell(nodedata), 'UniformValues', false); 381 | end 382 | edgenodes = num2cell(item.Edges.EndNodes); 383 | edgedata = table2struct(item.Edges); 384 | if (isfield(edgedata, 'EndNodes')) 385 | edgedata = rmfield(edgedata, 'EndNodes'); 386 | end 387 | edgenodes(:, 3) = num2cell(edgedata); 388 | if (isa(item, 'graph')) 389 | if (strcmp(varargin{1}.prefix, 'x')) 390 | newitem.(genvarname('_GraphEdges0_')) = edgenodes; 391 | else 392 | newitem.(encodevarname('_GraphEdges0_')) = edgenodes; 393 | end 394 | else 395 | newitem.(N_('_GraphEdges_', varargin{:})) = edgenodes; 396 | end 397 | 398 | %% ------------------------------------------------------------------------- 399 | function newitem = matlabobject2jd(item, varargin) 400 | try 401 | if numel(item) == 0 % empty object 402 | newitem = struct(); 403 | elseif numel(item) == 1 % 404 | newitem = char(item); 405 | else 406 | propertynames = properties(item); 407 | for p = 1:numel(propertynames) 408 | for o = numel(item):-1:1 % array of objects 409 | newitem(o).(propertynames{p}) = item(o).(propertynames{p}); 410 | end 411 | end 412 | end 413 | catch 414 | newitem = any2jd(item, varargin{:}); 415 | end 416 | 417 | %% ------------------------------------------------------------------------- 418 | function newitem = any2jd(item, varargin) 419 | try 420 | N = @(x) N_(x, varargin{:}); 421 | newitem.(N('_DataInfo_')) = struct('MATLABObjectClass', class(item), 'MATLABObjectSize', size(item)); 422 | newitem.(N('_ByteStream_')) = getByteStreamFromArray(item); % use undocumented matlab function 423 | if (varargin{1}.base64) 424 | newitem.(N('_ByteStream_')) = char(base64encode(newitem.(N('_ByteStream_')))); 425 | end 426 | catch 427 | error('any2jd: failed to convert object of type %s', class(item)); 428 | end 429 | 430 | %% ------------------------------------------------------------------------- 431 | function newname = N_(name, varargin) 432 | 433 | newname = [varargin{1}.prefix name]; 434 | -------------------------------------------------------------------------------- /jsonopt.m: -------------------------------------------------------------------------------- 1 | function val = jsonopt(key, default, varargin) 2 | % 3 | % val=jsonopt(key,default,optstruct) 4 | % 5 | % setting options based on a struct. The struct can be produced 6 | % by varargin2struct from a list of 'param','value' pairs 7 | % 8 | % authors:Qianqian Fang (q.fang neu.edu) 9 | % 10 | % input: 11 | % key: a string with which one look up a value from a struct 12 | % default: if the key does not exist, return default 13 | % optstruct: a struct where each sub-field is a key 14 | % 15 | % output: 16 | % val: if key exists, val=optstruct.key; otherwise val=default 17 | % 18 | % license: 19 | % BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 20 | % 21 | % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) 22 | % 23 | 24 | val = default; 25 | if (nargin <= 2) 26 | return 27 | end 28 | key0 = lower(key); 29 | opt = varargin{1}; 30 | if (isstruct(opt)) 31 | if (isfield(opt, key0)) 32 | val = opt.(key0); 33 | elseif (isfield(opt, key)) 34 | val = opt.(key); 35 | end 36 | end 37 | -------------------------------------------------------------------------------- /loadh5.m: -------------------------------------------------------------------------------- 1 | function varargout = loadh5(filename, varargin) 2 | % 3 | % [data, meta] = loadh5(filename) 4 | % [data, meta] = loadh5(root_id) 5 | % [data, meta] = loadh5(filename, rootpath) 6 | % [data, meta] = loadh5(filename, rootpath, options) 7 | % [data, meta] = loadh5(filename, options) 8 | % [data, meta] = loadh5(filename, 'Param1',value1, 'Param2',value2,...) 9 | % 10 | % Load data in an HDF5 file to a MATLAB structure. 11 | % 12 | % author: Qianqian Fang (q.fang neu.edu) 13 | % 14 | % input 15 | % filename 16 | % Name of the file to load data from 17 | % root_id: an HDF5 handle (of type 'H5ML.id' in MATLAB) 18 | % rootpath : (optional) 19 | % Root path to read part of the HDF5 file to load 20 | % options: (optional) a struct or Param/value pairs for user specified options 21 | % Order: 'creation' - creation order (default), or 'alphabet' - alphabetic 22 | % Regroup: [0|1]: if 1, call regrouph5() to combine indexed 23 | % groups into a cell array 24 | % PackHex: [1|0]: convert invalid characters in the group/dataset 25 | % names to 0x[hex code] by calling encodevarname.m; 26 | % if set to 0, call getvarname 27 | % ComplexFormat: {'realKey','imagKey'}: use 'realKey' and 'imagKey' 28 | % as possible keywords for the real and the imaginary part 29 | % of a complex array, respectively (sparse arrays not supported); 30 | % a common list of keypairs is used even without this option 31 | % Transpose: [1|0] - if set to 1 (default), the row-majored HDF5 32 | % datasets are transposed (to column-major) so that the 33 | % output MATLAB array has the same dimensions as in the 34 | % HDF5 dataset header. 35 | % 36 | % output 37 | % data: a structure (array) or cell (array) 38 | % meta: optional output to store the attributes stored in the file 39 | % 40 | % example: 41 | % a={rand(2), struct('va',1,'vb','string'), 1+2i}; 42 | % saveh5(a,'test.h5'); 43 | % a2=loadh5('test.h5') 44 | % a3=loadh5('test.h5','regroup',1) 45 | % isequaln(a,a3.a) 46 | % a4=loadh5('test.h5','/a1') 47 | % 48 | % This function was adapted from h5load.m by Pauli Virtanen 49 | % This file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 50 | % 51 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 52 | % 53 | 54 | path = ''; 55 | opt = struct; 56 | if (bitand(length(varargin), 1) == 0) 57 | opt = varargin2struct(varargin{:}); 58 | elseif (length(varargin) >= 3) 59 | path = varargin{1}; 60 | opt = varargin2struct(varargin{2:end}); 61 | elseif (length(varargin) == 1) 62 | path = varargin{1}; 63 | end 64 | 65 | opt.dotranspose = jsonopt('Transpose', 1, opt); 66 | opt.stringarray = jsonopt('StringArray', 0, opt); 67 | opt.rootpath = path; 68 | 69 | if (exist('OCTAVE_VERSION', 'builtin') ~= 0 && exist('h5info') == 0) 70 | try 71 | try 72 | pkg load oct-hdf5; 73 | catch 74 | [varargout{1:nargout}] = load(filename, '-hdf5'); 75 | if (opt.dotranspose) 76 | varargout{1} = transposemat(varargout{1}); 77 | end 78 | return 79 | end 80 | catch 81 | error(['To use EasyH5 in Octave, one must install oct-hdf5 first using\n\t' ... 82 | 'pkg install https://github.com/NeuroJSON/oct-hdf5/archive/refs/tags/git20250413.zip\n%s'], ''); 83 | end 84 | end 85 | 86 | if (isa(filename, 'H5ML.id')) 87 | loc = filename; 88 | else 89 | try 90 | loc = H5F.open(filename, 'H5F_ACC_RDONLY', 'H5P_DEFAULT'); 91 | catch ME 92 | error('fail to open file'); 93 | end 94 | end 95 | 96 | if (~(isfield(opt, 'complexformat') && iscellstr(opt.complexformat) && numel(opt.complexformat) == 2)) 97 | opt.complexformat = {'Real', 'Imag'}; 98 | end 99 | 100 | opt.releaseid = 0; 101 | vers = ver('MATLAB'); 102 | if (~isempty(vers)) 103 | opt.releaseid = datenum(vers(1).Date); 104 | end 105 | 106 | if ((isfield(opt, 'order') && strcmpi(opt.order, 'alphabet')) || opt.releaseid < datenum('1-Jan-2015')) 107 | opt.order = 'H5_INDEX_NAME'; 108 | else 109 | opt.order = 'H5_INDEX_CRT_ORDER'; 110 | end 111 | 112 | try 113 | if (nargin > 1 && ~isempty(path)) 114 | try 115 | rootgid = H5G.open(loc, path, 0); 116 | [varargout{1:nargout}] = load_one(rootgid, opt); 117 | H5G.close(rootgid); 118 | catch 119 | [gname, dname] = fileparts(path); 120 | rootgid = H5G.open(loc, gname, 0); 121 | [status, res] = group_iterate(rootgid, dname, struct('data', struct, 'meta', struct, 'opt', opt)); 122 | if (nargout > 0) 123 | varargout{1} = res.data; 124 | elseif (nargout > 1) 125 | varargout{2} = res.meta; 126 | end 127 | H5G.close(rootgid); 128 | end 129 | else 130 | [varargout{1:nargout}] = load_one(loc, opt); 131 | end 132 | H5F.close(loc); 133 | catch ME 134 | H5F.close(loc); 135 | rethrow(ME); 136 | end 137 | 138 | if (jsonopt('Regroup', 0, opt)) 139 | if (nargout >= 1) 140 | varargout{1} = regrouph5(varargout{1}); 141 | elseif (nargout >= 2) 142 | varargout{2} = regrouph5(varargout{2}); 143 | end 144 | end 145 | 146 | if (isfield(opt, 'jdata') && opt.jdata && nargout >= 1) 147 | varargout{1} = jdatadecode(varargout{1}, 'Base64', 0, opt); 148 | end 149 | 150 | % -------------------------------------------------------------------------- 151 | function [data, meta] = load_one(loc, opt) 152 | 153 | data = struct(); 154 | meta = struct(); 155 | inputdata = struct('data', data, 'meta', meta, 'opt', opt); 156 | 157 | % Load groups and datasets 158 | try 159 | [status, count, inputdata] = H5L.iterate(loc, opt.order, 'H5_ITER_INC', 0, @group_iterate, inputdata); 160 | catch ME 161 | if (strcmp(opt.order, 'H5_INDEX_CRT_ORDER')) 162 | [status, count, inputdata] = H5L.iterate(loc, 'H5_INDEX_NAME', 'H5_ITER_INC', 0, @group_iterate, inputdata); 163 | else 164 | rethrow(ME); 165 | end 166 | end 167 | 168 | data = inputdata.data; 169 | meta = inputdata.meta; 170 | 171 | % -------------------------------------------------------------------------- 172 | function [status, res] = group_iterate(group_id, objname, inputdata) 173 | status = 0; 174 | attr = struct(); 175 | 176 | encodename = jsonopt('PackHex', 1, inputdata.opt); 177 | 178 | try 179 | data = inputdata.data; 180 | meta = inputdata.meta; 181 | 182 | % objtype index 183 | if (exist('OCTAVE_VERSION', 'builtin') ~= 0) 184 | info = H5O.get_info_by_name(group_id, objname, 0); 185 | else 186 | info = H5G.get_objinfo(group_id, objname, 0); 187 | end 188 | 189 | objtype = info.type; 190 | objtype = objtype + 1; 191 | 192 | if objtype == 1 193 | % Group 194 | name = regexprep(objname, '.*/', ''); 195 | 196 | group_loc = H5G.open(group_id, name, 0); 197 | try 198 | [sub_data, sub_meta] = load_one(group_loc, inputdata.opt); 199 | H5G.close(group_loc); 200 | catch ME 201 | H5G.close(group_loc); 202 | rethrow(ME); 203 | end 204 | if (encodename) 205 | name = encodevarname(name); 206 | else 207 | name = genvarname(name); 208 | end 209 | data.(name) = sub_data; 210 | meta.(name) = sub_meta; 211 | 212 | elseif objtype == 2 213 | % Dataset 214 | name = regexprep(objname, '.*/', ''); 215 | 216 | dataset_loc = H5D.open(group_id, name); 217 | try 218 | sub_data = H5D.read(dataset_loc, ... 219 | 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT'); 220 | try 221 | [status, count, attr] = H5A.iterate(dataset_loc, 'H5_INDEX_NAME', 'H5_ITER_INC', 0, @getattribute, attr); 222 | catch 223 | attr = []; 224 | end 225 | H5D.close(dataset_loc); 226 | catch exc 227 | H5D.close(dataset_loc); 228 | rethrow(exc); 229 | end 230 | if (ischar(sub_data) && numel(sub_data) > 1 && sub_data(end) == 0) 231 | sub_data = sub_data(1:end - 1); 232 | end 233 | 234 | if ((isnumeric(sub_data) && inputdata.opt.dotranspose) || (iscell(sub_data) && length(sub_data) > 1)) 235 | sub_data = permute(sub_data, ndims(sub_data):-1:1); 236 | end 237 | sub_data = fix_data(sub_data, attr, inputdata.opt); 238 | if (encodename) 239 | name = encodevarname(name); 240 | else 241 | name = genvarname(name); 242 | end 243 | data.(name) = sub_data; 244 | meta.(name) = attr; 245 | end 246 | catch ME 247 | rethrow(ME); 248 | end 249 | res = struct('data', data, 'meta', meta, 'opt', inputdata.opt); 250 | 251 | % -------------------------------------------------------------------------- 252 | function data = fix_data(data, attr, opt) 253 | % Fix some common types of data to more friendly form. 254 | 255 | if isstruct(data) 256 | fields = fieldnames(data); 257 | 258 | if (length(intersect(fields, {'SparseIndex', opt.complexformat{1}})) == 2) 259 | if isnumeric(data.SparseIndex) && isnumeric(data.(opt.complexformat{1})) 260 | if (nargin > 1 && isstruct(attr)) 261 | if (isfield(attr, 'SparseArraySize')) 262 | spd = sparse(1, prod(attr.SparseArraySize)); 263 | if (isfield(data, opt.complexformat{2})) 264 | spd(data.SparseIndex) = complex(data.(opt.complexformat{1}), data.(opt.complexformat{2})); 265 | else 266 | spd(data.SparseIndex) = data.(opt.complexformat{1}); 267 | end 268 | data = reshape(spd, attr.SparseArraySize(:)'); 269 | return 270 | end 271 | end 272 | end 273 | else 274 | if (numel(opt.complexformat) == 2 && length(intersect(fields, opt.complexformat)) == 2) 275 | if isnumeric(data.(opt.complexformat{1})) && isnumeric(data.(opt.complexformat{2})) 276 | data = data.(opt.complexformat{1}) + 1j * data.(opt.complexformat{2}); 277 | end 278 | else 279 | % if complexformat is not specified or not found, try some common complex number storage formats 280 | if (length(intersect(fields, {'Real', 'Imag'})) == 2) 281 | if isnumeric(data.Real) && isnumeric(data.Imag) 282 | data = data.Real + 1j * data.Imag; 283 | end 284 | elseif (length(intersect(fields, {'real', 'imag'})) == 2) 285 | if isnumeric(data.real) && isnumeric(data.imag) 286 | data = data.real + 1j * data.imag; 287 | end 288 | elseif (length(intersect(fields, {'Re', 'Im'})) == 2) 289 | if isnumeric(data.Re) && isnumeric(data.Im) 290 | data = data.Re + 1j * data.Im; 291 | end 292 | elseif (length(intersect(fields, {'re', 'im'})) == 2) 293 | if isnumeric(data.re) && isnumeric(data.im) 294 | data = data.re + 1j * data.im; 295 | end 296 | elseif (length(intersect(fields, {'r', 'i'})) == 2) 297 | if isnumeric(data.r) && isnumeric(data.i) 298 | data = data.r + 1j * data.i; 299 | end 300 | end 301 | end 302 | end 303 | end 304 | 305 | if (isa(data, 'uint8') || isa(data, 'int8')) 306 | if (nargin > 1 && isstruct(attr)) 307 | if (isfield(attr, 'MATLABObjectClass')) 308 | data = getArrayFromByteStream(data); % use undocumented function 309 | end 310 | end 311 | end 312 | 313 | % handeling string arrays (or cell of char strings) 314 | if (iscell(data) && all(cellfun(@ischar, data))) 315 | if (exist('string') && opt.stringarray) 316 | data = string(data); 317 | end 318 | end 319 | 320 | % -------------------------------------------------------------------------- 321 | function [status, dataout] = getattribute(loc_id, attr_name, info, datain) 322 | status = 0; 323 | attr_id = H5A.open(loc_id, attr_name, 'H5P_DEFAULT'); 324 | datain.(attr_name) = H5A.read(attr_id, 'H5ML_DEFAULT'); 325 | H5A.close(attr_id); 326 | dataout = datain; 327 | -------------------------------------------------------------------------------- /mergestruct.m: -------------------------------------------------------------------------------- 1 | function s = mergestruct(s1, s2) 2 | % 3 | % s=mergestruct(s1,s2) 4 | % 5 | % merge two struct objects into one 6 | % 7 | % authors:Qianqian Fang (q.fang neu.edu) 8 | % date: 2012/12/22 9 | % 10 | % input: 11 | % s1,s2: a struct object, s1 and s2 can not be arrays 12 | % 13 | % output: 14 | % s: the merged struct object. fields in s1 and s2 will be combined in s. 15 | % 16 | % license: 17 | % BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 18 | % 19 | % -- this function is part of JSONLab toolbox (https://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) 20 | % 21 | 22 | if (~isstruct(s1) || ~isstruct(s2)) 23 | error('input parameters contain non-struct'); 24 | end 25 | if (length(s1) > 1 || length(s2) > 1) 26 | error('can not merge struct arrays'); 27 | end 28 | fn = fieldnames(s2); 29 | s = s1; 30 | for i = 1:length(fn) 31 | s.(fn{i}) = s2.(fn{i}); 32 | end 33 | -------------------------------------------------------------------------------- /regrouph5.m: -------------------------------------------------------------------------------- 1 | function data = regrouph5(root, varargin) 2 | % 3 | % data=regrouph5(root) 4 | % or 5 | % data=regrouph5(root,type) 6 | % data=regrouph5(root,{'nameA','nameB',...}) 7 | % 8 | % Processing a loadh5 restored data and merge "indexed datasets", whose 9 | % names start with an ASCII string followed by a contiguous integer 10 | % sequence number starting from 1, into a cell array. For example, 11 | % datasets {data.a1, data.a2, data.a3} will be merged into a cell/struct 12 | % array data.a with 3 elements. 13 | % 14 | % A single subfield .name1 will be renamed as .name. Items with 15 | % non-contigous numbering will not be grouped. If .name and 16 | % .name1/.name2 co-exist in the input struct, no grouping will be done. 17 | % 18 | % The grouped subfield will appear at the position of the first 19 | % pre-grouped item in the original input structure. 20 | % 21 | % author: Qianqian Fang (q.fang neu.edu) 22 | % 23 | % input: 24 | % root: the raw input HDF5 data structure (loaded from loadh5.m) 25 | % type: if type is set as a cell array of strings, it restrict the 26 | % grouping only to the subset of field names in this list; 27 | % if type is a string as 'snirf', it is the same as setting 28 | % type as {'aux','data','nirs','stim','measurementList'}. 29 | % 30 | % output: 31 | % data: a reorganized matlab structure. 32 | % 33 | % example: 34 | % a=struct('a1',rand(5),'a2','string','a3',true,'d',2+3i,'e',{'test',[],1:5}); 35 | % regrouph5(a) 36 | % saveh5(a,'test.h5'); 37 | % rawdata=loadh5('test.h5') 38 | % data=regrouph5(rawdata) 39 | % 40 | % this file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 41 | % 42 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 43 | % 44 | 45 | if (nargin < 1) 46 | help regrouph5; 47 | return 48 | end 49 | 50 | dict = {}; 51 | if (~isempty(varargin)) 52 | if (ischar(varargin{1}) && strcmpi(varargin{1}, 'snirf')) 53 | dict = {'aux', 'data', 'nirs', 'stim', 'measurementList'}; 54 | elseif (iscell(varargin{1})) 55 | dict = varargin{1}; 56 | end 57 | end 58 | 59 | data = struct; 60 | if (isstruct(root)) 61 | data = repmat(struct, size(root)); 62 | names = fieldnames(root); 63 | newnames = struct(); 64 | firstpos = struct(); 65 | 66 | for i = 1:length(names) 67 | item = regexp(names{i}, '^(.*\D)(\d+)$', 'tokens'); 68 | if (~isempty(item) && str2double(item{1}{2}) ~= 0 && ~isfield(root, item{1}{1})) 69 | if (~isfield(newnames, item{1}{1})) 70 | newnames.(item{1}{1}) = str2double(item{1}{2}); 71 | else 72 | newnames.(item{1}{1}) = [newnames.(item{1}{1}), str2double(item{1}{2})]; 73 | end 74 | if (~isfield(firstpos, item{1}{1})) 75 | firstpos.(item{1}{1}) = length(fieldnames(data(1))); 76 | end 77 | else 78 | for j = 1:length(root) 79 | if (isstruct(root(j).(names{i}))) 80 | data(j).(names{i}) = regrouph5(root(j).(names{i})); 81 | else 82 | data(j).(names{i}) = root(j).(names{i}); 83 | end 84 | end 85 | end 86 | end 87 | 88 | names = fieldnames(newnames); 89 | if (~isempty(dict)) 90 | names = intersect(names, dict); 91 | end 92 | 93 | for i = length(names):-1:1 94 | len = length(newnames.(names{i})); 95 | idx = newnames.(names{i}); 96 | if ((min(idx) ~= 1 || max(idx) ~= len) && len ~= 1) 97 | for j = 1:len 98 | dataname = sprintf('%s%d', names{i}, idx(j)); 99 | for k = 1:length(root) 100 | if (isstruct(root(k).(dataname))) 101 | data(k).(dataname) = regrouph5(root(k).(dataname)); 102 | else 103 | data(k).(dataname) = root(k).(dataname); 104 | end 105 | end 106 | end 107 | pos = firstpos.(names{i}); 108 | len = length(fieldnames(data)); 109 | data = orderfields(data, [1:pos, len, pos + 1:len - 1]); 110 | continue 111 | end 112 | for j = 1:length(data) 113 | data(j).(names{i}) = cell(1, len); 114 | end 115 | idx = sort(idx); 116 | for j = 1:len 117 | for k = 1:length(root) 118 | obj = root(k).(sprintf('%s%d', names{i}, idx(j))); 119 | if (isstruct(obj)) 120 | data(k).(names{i}){j} = regrouph5(obj); 121 | else 122 | data(k).(names{i}){j} = obj; 123 | end 124 | end 125 | end 126 | pos = firstpos.(names{i}); 127 | len = length(fieldnames(data)); 128 | data = orderfields(data, [1:pos, len, pos + 1:len - 1]); 129 | try 130 | data.(names{i}) = cell2mat(data.(names{i})); 131 | catch 132 | end 133 | end 134 | end 135 | -------------------------------------------------------------------------------- /saveh5.m: -------------------------------------------------------------------------------- 1 | function saveh5(data, fname, varargin) 2 | % 3 | % saveh5(data, outputfile) 4 | % or 5 | % saveh5(data, outputfile, options) 6 | % saveh5(data, outputfile, 'Param1',value1, 'Param2',value2,...) 7 | % 8 | % Save a MATLAB struct (array) or cell (array) into an HDF5 file 9 | % 10 | % author: Qianqian Fang (q.fang neu.edu) 11 | % 12 | % input: 13 | % data: a structure (array) or cell (array) to be stored. 14 | % fname: the output HDF5 (.h5) file name 15 | % options: (optional) a struct or Param/value pairs for user specified options 16 | % JData [0|1] use JData Specifiation to serialize complex data structures 17 | % such as complex/sparse arrays, tables, maps, graphs etc by 18 | % calling jdataencode before saving data to HDF5 19 | % RootName: the HDF5 path of the root object. If not given, the 20 | % actual variable name for the data input will be used as 21 | % the root object. The value shall not include '/'. 22 | % UnpackHex [1|0]: convert the 0x[hex code] in variable names 23 | % back to Unicode string using decodevarname.m 24 | % Compression: ['deflate'|''] - use zlib-deflate method 25 | % to compress data array 26 | % CompressArraySize: [100|int]: only to compress an array if the 27 | % total element count is larger than this number. 28 | % CompressLevel: [5|int] - a number between 1-9 to set 29 | % compression level 30 | % Chunk: a size vector or empty - breaking a large array into 31 | % small chunks of size specified by this parameter 32 | % Append [0|1]: if set to 1, new data will be appended to an 33 | % file if already exists under the user-defined 34 | % 'rootname' path; if set to 0, old data 35 | % will be overwritten if the file exists. 36 | % VariableLengthString [0|1]: if set to 1, strings and char arrays will be 37 | % saved with type 'H5T_C_S1' and size 'H5T_VARIABLE' 38 | % Scalar [1|0]: if set to 1, arrays of length 1 will be saved as 39 | % a scalar instead of a length-1 array 40 | % Transpose: [1|0] - if set to 1 (default), MATLAB arrays are 41 | % transposed (from column-major to row-major) so 42 | % that the output HDF5 dataset shows the same 43 | % dimensions as in MATLAB when reading from other 44 | % tools. 45 | % ComplexFormat: {'realKey','imagKey'}: use 'realKey' and 'imagKey' 46 | % as keywords for the real and the imaginary part of a 47 | % complex array, respectively (sparse arrays not supported); 48 | % the default values are {'Real','Imag'} 49 | % 50 | % example: 51 | % a=struct('a',rand(5),'b','string','c',true,'d',2+3i,'e',{'test',[],1:5}); 52 | % saveh5(a,'test.h5'); 53 | % saveh5(a(1),'test2.h5','rootname',''); 54 | % saveh5(a(1),'test2.h5','compression','deflate','compressarraysize',1); 55 | % saveh5(a,'test.h5j','jdata',1); 56 | % saveh5(a,'test.h5j','rootname','/newroot','append',1); 57 | % 58 | % this file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 59 | % 60 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 61 | % 62 | 63 | if (nargin < 2) 64 | error('you must provide at least two inputs'); 65 | end 66 | 67 | rootname = ['/' inputname(1)]; 68 | opt = struct; 69 | 70 | if (length(varargin) == 1 && ischar(varargin{1})) 71 | rootname = [varargin{1} '/' inputname(1)]; 72 | else 73 | opt = varargin2struct(varargin{:}); 74 | end 75 | 76 | opt.compression = jsonopt('Compression', '', opt); 77 | opt.compresslevel = jsonopt('CompressLevel', 5, opt); 78 | opt.compressarraysize = jsonopt('CompressArraySize', 100, opt); 79 | opt.unpackhex = jsonopt('UnpackHex', 1, opt); 80 | opt.dotranspose = jsonopt('Transpose', 1, opt); 81 | opt.variablelengthstring = jsonopt('VariableLengthString', 0, opt); 82 | opt.scalar = jsonopt('Scalar', 1, opt); 83 | 84 | opt.releaseid = 0; 85 | opt.isoctave = 0; 86 | vers = ver('MATLAB'); 87 | if (~isempty(vers)) 88 | opt.releaseid = datenum(vers(1).Date); 89 | opt.skipempty = (opt.releaseid < datenum('1-Jan-2015')); 90 | else 91 | opt.isoctave = 1; 92 | opt.skipempty = 0; 93 | end 94 | 95 | if (exist('OCTAVE_VERSION', 'builtin') ~= 0 && exist('h5info') == 0) 96 | try 97 | pkg load oct-hdf5; 98 | catch 99 | error(['To use EasyH5 in Octave, one must install oct-hdf5 first using\n\t' ... 100 | 'pkg install https://github.com/NeuroJSON/oct-hdf5/archive/refs/tags/git20250413.zip\n%s'], ''); 101 | end 102 | end 103 | 104 | if (isfield(opt, 'rootname')) 105 | rootname = ['/' opt.rootname]; 106 | end 107 | 108 | if (~isfield(opt, 'rootname') && ~isempty(regexp(rootname, '/$', 'once'))) 109 | rootname = [rootname 'data']; 110 | end 111 | 112 | if (jsonopt('JData', 0, opt)) 113 | data = jdataencode(data, 'Base64', 0, 'UseArrayZipSize', 0, opt); 114 | end 115 | 116 | try 117 | if (isa(fname, 'H5ML.id')) 118 | fid = fname; 119 | else 120 | if (jsonopt('append', 0, opt) && exist(fname, 'file')) 121 | fid = H5F.open(fname, 'H5F_ACC_RDWR', 'H5P_DEFAULT'); 122 | else 123 | fid = H5F.create(fname, 'H5F_ACC_TRUNC', H5P.create('H5P_FILE_CREATE'), H5P.create('H5P_FILE_ACCESS')); 124 | end 125 | end 126 | obj2h5(rootname, data, fid, 1, opt); 127 | catch ME 128 | if (exist('fid', 'var') && fid > 0) 129 | H5F.close(fid); 130 | end 131 | rethrow(ME); 132 | end 133 | 134 | if (~isa(fname, 'H5ML.id')) 135 | H5F.close(fid); 136 | end 137 | 138 | %% ------------------------------------------------------------------------- 139 | function oid = obj2h5(name, item, handle, level, varargin) 140 | 141 | if (iscell(item)) 142 | oid = cell2h5(name, item, handle, level, varargin{:}); 143 | elseif (isa(item, 'jdict')) 144 | oid = obj2h5(name, item.v(), handle, level, varargin{:}); 145 | elseif (isstruct(item)) 146 | oid = struct2h5(name, item, handle, level, varargin{:}); 147 | elseif (ischar(item) || isa(item, 'string')) 148 | oid = mat2h5(name, item, handle, level, varargin{:}); 149 | elseif (isa(item, 'containers.Map')) 150 | oid = map2h5(name, item, handle, level, varargin{:}); 151 | elseif (isa(item, 'categorical')) 152 | oid = cell2h5(name, cellstr(item), handle, level, varargin{:}); 153 | elseif (islogical(item) || isnumeric(item) || isa(item, 'timeseries')) 154 | oid = mat2h5(name, item, handle, level, varargin{:}); 155 | else 156 | oid = any2h5(name, item, handle, level, varargin{:}); 157 | end 158 | 159 | %% ------------------------------------------------------------------------- 160 | function oid = idxobj2h5(name, idx, varargin) 161 | oid = obj2h5(sprintf('%s%d', name, idx), varargin{:}); 162 | 163 | %% ------------------------------------------------------------------------- 164 | function oid = cell2h5(name, item, handle, level, varargin) 165 | 166 | num = numel(item); 167 | if (num > 1) 168 | idx = reshape(1:num, size(item)); 169 | idx = num2cell(idx); 170 | oid = cellfun(@(x, id) idxobj2h5(name, id, x, handle, level, varargin{:}), item, idx, 'UniformOutput', false); 171 | else 172 | oid = cellfun(@(x) obj2h5(name, x, handle, level, varargin{:}), item, 'UniformOutput', false); 173 | end 174 | 175 | %% ------------------------------------------------------------------------- 176 | function oid = struct2h5(name, item, handle, level, varargin) 177 | 178 | num = numel(item); 179 | if (num > 1) 180 | oid = obj2h5(name, num2cell(item), handle, level, varargin{:}); 181 | else 182 | pd = 'H5P_DEFAULT'; 183 | gcpl = H5P.create('H5P_GROUP_CREATE'); 184 | tracked = H5ML.get_constant_value('H5P_CRT_ORDER_TRACKED'); 185 | indexed = H5ML.get_constant_value('H5P_CRT_ORDER_INDEXED'); 186 | order = bitor(tracked, indexed); 187 | if (~varargin{1}.isoctave) 188 | H5P.set_link_creation_order(gcpl, order); 189 | end 190 | if (varargin{1}.unpackhex) 191 | name = decodevarname(name); 192 | end 193 | try 194 | handle = H5G.create(handle, name, pd, gcpl, pd); 195 | isnew = 1; 196 | catch 197 | isnew = 0; 198 | end 199 | 200 | names = fieldnames(item); 201 | oid = cell(1, length(names)); 202 | for i = 1:length(names) 203 | oid{i} = obj2h5(names{i}, item.(names{i}), handle, level + 1, varargin{:}); 204 | end 205 | 206 | if (isnew) 207 | H5G.close(handle); 208 | end 209 | end 210 | 211 | %% ------------------------------------------------------------------------- 212 | function oid = map2h5(name, item, handle, level, varargin) 213 | 214 | pd = 'H5P_DEFAULT'; 215 | gcpl = H5P.create('H5P_GROUP_CREATE'); 216 | tracked = H5ML.get_constant_value('H5P_CRT_ORDER_TRACKED'); 217 | indexed = H5ML.get_constant_value('H5P_CRT_ORDER_INDEXED'); 218 | order = bitor(tracked, indexed); 219 | if (~varargin{1}.isoctave) 220 | H5P.set_link_creation_order(gcpl, order); 221 | end 222 | try 223 | if (varargin{1}.unpackhex) 224 | name = decodevarname(name); 225 | end 226 | handle = H5G.create(handle, name, pd, gcpl, pd); 227 | isnew = 1; 228 | catch 229 | isnew = 0; 230 | end 231 | 232 | names = item.keys; 233 | oid = zeros(length(names)); 234 | for i = 1:length(names) 235 | oid(i) = obj2h5(names{i}, item(names{i}), handle, level + 1, varargin{:}); 236 | end 237 | 238 | if (isnew) 239 | H5G.close(handle); 240 | end 241 | 242 | %% ------------------------------------------------------------------------- 243 | function oid = mat2h5(name, item, handle, level, varargin) 244 | typemap = h5types; 245 | 246 | opt = varargin{1}; 247 | 248 | is1dvector = 0; 249 | 250 | if (isa(item, 'timeseries')) 251 | if (item.TimeInfo.Length == 1 || ... 252 | (item.TimeInfo.isUniform && item.TimeInfo.Increment == 1 && ndims(item.Data) == 3 && size(item.Data, 1) == 1 && size(item.Data, 2) == 1) || ... 253 | (isempty(item.Time) && isempty(item.Data) && size(item.Time, 2) == 1)) 254 | is1dvector = 1; 255 | item = squeeze(item.Data); 256 | else 257 | item = [item.Time, item.Data]; 258 | end 259 | end 260 | 261 | if (opt.dotranspose) 262 | item = permute(item, ndims(item):-1:1); 263 | end 264 | 265 | pd = 'H5P_DEFAULT'; 266 | gcpl = H5P.create('H5P_GROUP_CREATE'); 267 | tracked = H5ML.get_constant_value('H5P_CRT_ORDER_TRACKED'); 268 | indexed = H5ML.get_constant_value('H5P_CRT_ORDER_INDEXED'); 269 | order = bitor(tracked, indexed); 270 | if (~varargin{1}.isoctave) 271 | H5P.set_link_creation_order(gcpl, order); 272 | end 273 | 274 | if (~(isfield(opt, 'complexformat') && iscellstr(opt.complexformat) && numel(opt.complexformat) == 2) || strcmp(opt.complexformat{1}, opt.complexformat{2})) 275 | opt.complexformat = {'Real', 'Imag'}; 276 | end 277 | 278 | usefilter = opt.compression; 279 | complevel = opt.compresslevel; 280 | minsize = opt.compressarraysize; 281 | chunksize = jsonopt('Chunk', size(item), opt); 282 | 283 | if (isa(item, 'logical')) 284 | item = uint8(item); 285 | end 286 | 287 | if (~isempty(usefilter) && numel(item) >= minsize) 288 | if (isnumeric(usefilter) && usefilter(1) == 1) 289 | usefilter = 'deflate'; 290 | end 291 | if (strcmpi(usefilter, 'deflate')) 292 | pd = H5P.create('H5P_DATASET_CREATE'); 293 | h5_chunk_dims = fliplr(chunksize); 294 | H5P.set_chunk(pd, h5_chunk_dims); 295 | H5P.set_deflate(pd, complevel); 296 | opt.scalar = 0; 297 | opt.variablelengthstring = 0; 298 | else 299 | error('Filter %s is unsupported', usefilter); 300 | end 301 | end 302 | 303 | if (opt.unpackhex) 304 | name = decodevarname(name); 305 | end 306 | 307 | oid = []; 308 | 309 | if (isempty(item) && opt.skipempty) 310 | warning('The HDF5 library is older than v1.8.7, and can not save empty datasets. Skip saving "%s"', name); 311 | return 312 | end 313 | 314 | if (isreal(item) || isa(item, 'string')) 315 | if (issparse(item)) 316 | idx = find(item); 317 | oid = sparse2h5(name, struct('Size', size(item), 'SparseIndex', idx, 'Real', full(item(idx))), handle, level, varargin{:}); 318 | else 319 | itemtype = H5T.copy(typemap.(class(item))); 320 | if ((ischar(item) || isa(item, 'string')) && opt.variablelengthstring) 321 | H5T.set_size(itemtype, 'H5T_VARIABLE'); 322 | itemsize = H5S.create('H5S_SCALAR'); 323 | if (isa(item, 'string') && is1dvector) 324 | itemsize = H5S.create_simple(1, length(item), length(item)); 325 | elseif (isa(item, 'string') && length(item) > 1) 326 | itemsize = H5S.create_simple(ndims(item), fliplr(size(item)), fliplr(size(item))); 327 | end 328 | elseif (isnumeric(item) && numel(item) == 1 && ndims(item) == 2 && opt.scalar && ~is1dvector) 329 | itemsize = H5S.create('H5S_SCALAR'); 330 | else 331 | if (is1dvector) 332 | itemsize = H5S.create_simple(1, length(item), length(item)); 333 | else 334 | itemsize = H5S.create_simple(ndims(item), fliplr(size(item)), fliplr(size(item))); 335 | end 336 | end 337 | try 338 | oid = H5D.create(handle, name, itemtype, itemsize, pd); 339 | catch ME 340 | if (jsonopt('append', 0, opt) == 1) 341 | H5L.delete(handle, name, 'H5P_DEFAULT'); 342 | oid = H5D.create(handle, name, itemtype, H5S.create_simple(ndims(item), fliplr(size(item)), fliplr(size(item))), pd); 343 | end 344 | end 345 | if ((ischar(item) || isa(item, 'string')) && isfield(opt, 'variablelengthstring') && opt.variablelengthstring) 346 | if (isa(item, 'string')) 347 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', cellstr(item)); 348 | else 349 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', {item}); 350 | end 351 | else 352 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', item); 353 | end 354 | end 355 | else 356 | if (issparse(item)) 357 | idx = find(item); 358 | oid = sparse2h5(name, struct('Size', size(item), 'SparseIndex', idx, 'Real', full(real(item(idx))), 'Imag', full(imag(item(idx)))), handle, level, varargin{:}); 359 | else 360 | typeid = H5T.copy(typemap.(class(item))); 361 | elemsize = H5T.get_size(typeid); 362 | memtype = H5T.create ('H5T_COMPOUND', elemsize * 2); 363 | H5T.insert (memtype, opt.complexformat{1}, 0, typeid); 364 | H5T.insert (memtype, opt.complexformat{2}, elemsize, typeid); 365 | if (is1dvector) 366 | oid = H5D.create(handle, name, memtype, H5S.create_simple(1, length(item), length(item)), pd); 367 | else 368 | oid = H5D.create(handle, name, memtype, H5S.create_simple(ndims(item), fliplr(size(item)), fliplr(size(item))), pd); 369 | end 370 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', struct(opt.complexformat{1}, real(item), opt.complexformat{2}, imag(item))); 371 | end 372 | end 373 | if (~isempty(oid)) 374 | H5D.close(oid); 375 | end 376 | 377 | %% ------------------------------------------------------------------------- 378 | function oid = sparse2h5(name, item, handle, level, varargin) 379 | 380 | opt = varargin{1}; 381 | 382 | idx = item.SparseIndex; 383 | 384 | if (isempty(idx) && opt.skipempty) 385 | warning('The HDF5 library is older than v1.8.7, and can not save empty datasets. Skip saving "%s"', name); 386 | oid = []; 387 | return 388 | end 389 | 390 | adata = item.Size; 391 | item = rmfield(item, 'Size'); 392 | hasimag = isfield(item, 'Imag'); 393 | 394 | typemap = h5types; 395 | 396 | pd = 'H5P_DEFAULT'; 397 | 398 | usefilter = opt.compression; 399 | complevel = opt.compresslevel; 400 | minsize = opt.compressarraysize; 401 | chunksize = jsonopt('Chunk', size(item), opt); 402 | 403 | if (~isempty(usefilter) && numel(idx) >= minsize) 404 | if (isnumeric(usefilter) && usefilter(1) == 1) 405 | usefilter = 'deflate'; 406 | end 407 | if (strcmpi(usefilter, 'deflate')) 408 | pd = H5P.create('H5P_DATASET_CREATE'); 409 | h5_chunk_dims = fliplr(chunksize); 410 | H5P.set_chunk(pd, h5_chunk_dims); 411 | H5P.set_deflate(pd, complevel); 412 | else 413 | error('Filter %s is unsupported', usefilter); 414 | end 415 | end 416 | 417 | idxtypeid = H5T.copy(typemap.(class(idx))); 418 | idxelemsize = H5T.get_size(idxtypeid); 419 | datatypeid = H5T.copy(typemap.(class(item.Real))); 420 | dataelemsize = H5T.get_size(datatypeid); 421 | memtype = H5T.create ('H5T_COMPOUND', idxelemsize + dataelemsize * (1 + hasimag)); 422 | H5T.insert (memtype, 'SparseIndex', 0, idxtypeid); 423 | H5T.insert (memtype, 'Real', idxelemsize, datatypeid); 424 | if (hasimag) 425 | H5T.insert (memtype, 'Imag', idxelemsize + dataelemsize, datatypeid); 426 | end 427 | oid = H5D.create(handle, name, memtype, H5S.create_simple(ndims(idx), fliplr(size(idx)), fliplr(size(idx))), pd); 428 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', item); 429 | 430 | space_id = H5S.create_simple(ndims(adata), fliplr(size(adata)), fliplr(size(adata))); 431 | attr_size = H5A.create(oid, 'SparseArraySize', H5T.copy('H5T_NATIVE_DOUBLE'), space_id, H5P.create('H5P_ATTRIBUTE_CREATE')); 432 | H5A.write(attr_size, 'H5ML_DEFAULT', adata); 433 | H5A.close(attr_size); 434 | 435 | %% ------------------------------------------------------------------------- 436 | function oid = any2h5(name, item, handle, level, varargin) 437 | pd = 'H5P_DEFAULT'; 438 | 439 | if (varargin{1}.unpackhex) 440 | name = decodevarname(name); 441 | end 442 | 443 | rawdata = getByteStreamFromArray(item); % use undocumented matlab function 444 | oid = H5D.create(handle, name, H5T.copy('H5T_STD_U8LE'), H5S.create_simple(ndims(rawdata), size(rawdata), size(rawdata)), pd); 445 | H5D.write(oid, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', pd, rawdata); 446 | 447 | adata = class(item); 448 | space_id = H5S.create_simple(ndims(adata), size(adata), size(adata)); 449 | attr_type = H5A.create(oid, 'MATLABObjectClass', H5T.copy('H5T_C_S1'), space_id, H5P.create('H5P_ATTRIBUTE_CREATE')); 450 | H5A.write(attr_type, 'H5ML_DEFAULT', adata); 451 | H5A.close(attr_type); 452 | 453 | adata = size(item); 454 | space_id = H5S.create_simple(ndims(adata), size(adata), size(adata)); 455 | attr_size = H5A.create(oid, 'MATLABObjectSize', H5T.copy('H5T_NATIVE_DOUBLE'), space_id, H5P.create('H5P_ATTRIBUTE_CREATE')); 456 | H5A.write(attr_size, 'H5ML_DEFAULT', adata); 457 | H5A.close(attr_size); 458 | 459 | H5D.close(oid); 460 | 461 | %% ------------------------------------------------------------------------- 462 | function typemap = h5types 463 | typemap.char = 'H5T_C_S1'; 464 | typemap.string = 'H5T_C_S1'; 465 | typemap.double = 'H5T_IEEE_F64LE'; 466 | typemap.single = 'H5T_IEEE_F32LE'; 467 | typemap.logical = 'H5T_STD_U8LE'; 468 | typemap.uint8 = 'H5T_STD_U8LE'; 469 | typemap.int8 = 'H5T_STD_I8LE'; 470 | typemap.uint16 = 'H5T_STD_U16LE'; 471 | typemap.int16 = 'H5T_STD_I16LE'; 472 | typemap.uint32 = 'H5T_STD_U32LE'; 473 | typemap.int32 = 'H5T_STD_I32LE'; 474 | typemap.uint64 = 'H5T_STD_U64LE'; 475 | typemap.int64 = 'H5T_STD_I64LE'; 476 | -------------------------------------------------------------------------------- /transposemat.m: -------------------------------------------------------------------------------- 1 | function data = transposemat(input) 2 | % 3 | % data=transposemat(input) 4 | % 5 | % Iterate over struct/cell and transpose 2D or higher-dimensional numerical 6 | % array to match Octave loaded HDF5 array elements with loadh5 default setting 7 | % 8 | % author: Qianqian Fang (q.fang neu.edu) 9 | % 10 | % input: 11 | % name: a matlab variable, can be a cell, struct, containers.Map, numeric array or strings 12 | % 13 | % output: 14 | % newname: the restored original string 15 | % 16 | % example: 17 | % a=struct('a', ones(2,3), 'b', 'a string', 'c', uint8(zeros(2,3,4))); 18 | % b=transposemat(a) 19 | % 20 | % this file is part of EasyH5 Toolbox: https://github.com/NeuroJSON/easyh5 21 | % 22 | % License: GPLv3 or 3-clause BSD license, see https://github.com/NeuroJSON/easyh5 for details 23 | % 24 | 25 | if (isstruct(input)) 26 | data = structfun(@transposemat, input, 'UniformOutput', false); 27 | elseif (iscell(input)) 28 | data = cellfun(@transposemat, input, 'UniformOutput', 'false'); 29 | elseif (isa(input, 'containers.Map')) 30 | allkeys = keys(input); 31 | for i = 1:length(allkeys) 32 | input(allkeys(i)) = transposemat(allkeys(i)); 33 | end 34 | elseif (isnumeric(input) && (ndims(input) > 2 || all(size(input) > 1))) 35 | data = permute(input, ndims(input):-1:1); 36 | elseif (ischar(input) && ndims(input) == 2 && size(input, 1) == 1 && size(input, 2) > 1 && input(end) == ' ') 37 | data = input(1:end - 1); 38 | else 39 | data = input; 40 | end 41 | -------------------------------------------------------------------------------- /varargin2struct.m: -------------------------------------------------------------------------------- 1 | function opt = varargin2struct(varargin) 2 | % 3 | % opt=varargin2struct('param1',value1,'param2',value2,...) 4 | % or 5 | % opt=varargin2struct(...,optstruct,...) 6 | % 7 | % convert a series of input parameters into a structure 8 | % 9 | % authors:Qianqian Fang (q.fang neu.edu) 10 | % date: 2012/12/22 11 | % 12 | % input: 13 | % 'param', value: the input parameters should be pairs of a string and a value 14 | % optstruct: if a parameter is a struct, the fields will be merged to the output struct 15 | % 16 | % output: 17 | % opt: a struct where opt.param1=value1, opt.param2=value2 ... 18 | % 19 | % license: 20 | % BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 21 | % 22 | % -- this function is part of JSONLab toolbox (https://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) 23 | % 24 | 25 | len = length(varargin); 26 | opt = struct; 27 | if (len == 0) 28 | return 29 | end 30 | i = 1; 31 | while (i <= len) 32 | if (isstruct(varargin{i})) 33 | opt = mergestruct(opt, varargin{i}); 34 | elseif ((ischar(varargin{i}) || isa(varargin{i}, 'string')) && i < len) 35 | opt.(lower(varargin{i})) = varargin{i + 1}; 36 | i = i + 1; 37 | else 38 | error('input must be in the form of ...,''name'',value,... pairs or structs'); 39 | end 40 | i = i + 1; 41 | end 42 | --------------------------------------------------------------------------------