├── .gitignore ├── .travis.yml ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── jsonschema.go ├── jsonschema2go └── cli.go └── text ├── text.go └── text_test.go /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled Object files, Static and Dynamic libs (Shared Objects) 2 | *.o 3 | *.a 4 | *.so 5 | 6 | # Folders 7 | _obj 8 | _test 9 | 10 | # Architecture specific extensions/prefixes 11 | *.[568vq] 12 | [568vq].out 13 | 14 | *.cgo1.go 15 | *.cgo2.c 16 | _cgo_defun.c 17 | _cgo_gotypes.go 18 | _cgo_export.* 19 | 20 | _testmain.go 21 | 22 | *.exe 23 | *.test 24 | *.prof 25 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: go 2 | go: 3 | - 1.9 4 | 5 | # currently cannot customise per user fork, see: 6 | # https://github.com/travis-ci/travis-ci/issues/1094 7 | notifications: 8 | irc: 9 | channels: 10 | - "irc.mozilla.org#taskcluster-bots" 11 | on_success: change 12 | on_failure: always 13 | template: 14 | - "%{repository}#%{build_number} (%{branch} - %{commit} : %{author}): %{message}" 15 | - "Change view : %{compare_url}" 16 | - "Build details : %{build_url}" 17 | - "Commit message : %{commit_message}" 18 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Community Participation Guidelines 2 | 3 | This repository is governed by Mozilla's code of conduct and etiquette guidelines. 4 | For more details, please read the 5 | [Mozilla Community Participation Guidelines](https://www.mozilla.org/about/governance/policies/participation/). 6 | 7 | ## How to Report 8 | For more information on how to report violations of the Community Participation Guidelines, please read our '[How to Report](https://www.mozilla.org/about/governance/policies/participation/reporting/)' page. 9 | 10 | 16 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to Contribute 2 | 3 | We welcome pull requests from everyone. We do expect everyone to adhere to the [Mozilla Community Participation Guidelines][participation]. 4 | 5 | If you're trying to figure out what to work on, here are some places to find suitable projects: 6 | * [Good first bugs][goodfirstbug]: these are scoped to make it easy for first-time contributors to get their feet wet with Taskcluster code. 7 | * [Mentored bugs][bugsahoy]: these are slightly more involved projects that may require insight or guidance from someone on the Taskcluster team. 8 | * [Full list of open issues][issues]: everything else 9 | 10 | If the project you're interested in working on isn't covered by a bug or issue, or you're unsure about how to proceed on an existing issue, it's a good idea to talk to someone on the Taskcluster team before you go too far down a particular path. You can find us in the #taskcluster channel on [Mozilla's IRC server][irc] to discuss. You can also simply add a comment to the issue or bug. 11 | 12 | Once you've found an issue to work on and written a patch, submit a pull request. Some things that will increase the chance that your pull request is accepted: 13 | 14 | * Follow our [best practices][bestpractices]. 15 | * This includes [writing or updating tests][testing]. 16 | * Write a [good commit message][commit]. 17 | 18 | Welcome to the team! 19 | 20 | [participation]: https://www.mozilla.org/en-US/about/governance/policies/participation/ 21 | [issues]: ../../issues 22 | [bugsahoy]: https://www.joshmatthews.net/bugsahoy/?taskcluster=1 23 | [goodfirstbug]: http://www.joshmatthews.net/bugsahoy/?taskcluster=1&simple=1 24 | [irc]: https://wiki.mozilla.org/IRC 25 | [bestpractices]: https://docs.taskcluster.net/docs/manual/design/devel/best-practices 26 | [testing]: https://docs.taskcluster.net/docs/manual/design/devel/best-practices/testing 27 | [commit]: https://docs.taskcluster.net/docs/manual/design/devel/best-practices/commits 28 | 29 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Mozilla Public License Version 2.0 2 | ================================== 3 | 4 | 1. Definitions 5 | -------------- 6 | 7 | 1.1. "Contributor" 8 | means each individual or legal entity that creates, contributes to 9 | the creation of, or owns Covered Software. 10 | 11 | 1.2. "Contributor Version" 12 | means the combination of the Contributions of others (if any) used 13 | by a Contributor and that particular Contributor's Contribution. 14 | 15 | 1.3. "Contribution" 16 | means Covered Software of a particular Contributor. 17 | 18 | 1.4. "Covered Software" 19 | means Source Code Form to which the initial Contributor has attached 20 | the notice in Exhibit A, the Executable Form of such Source Code 21 | Form, and Modifications of such Source Code Form, in each case 22 | including portions thereof. 23 | 24 | 1.5. "Incompatible With Secondary Licenses" 25 | means 26 | 27 | (a) that the initial Contributor has attached the notice described 28 | in Exhibit B to the Covered Software; or 29 | 30 | (b) that the Covered Software was made available under the terms of 31 | version 1.1 or earlier of the License, but not also under the 32 | terms of a Secondary License. 33 | 34 | 1.6. "Executable Form" 35 | means any form of the work other than Source Code Form. 36 | 37 | 1.7. "Larger Work" 38 | means a work that combines Covered Software with other material, in 39 | a separate file or files, that is not Covered Software. 40 | 41 | 1.8. "License" 42 | means this document. 43 | 44 | 1.9. "Licensable" 45 | means having the right to grant, to the maximum extent possible, 46 | whether at the time of the initial grant or subsequently, any and 47 | all of the rights conveyed by this License. 48 | 49 | 1.10. "Modifications" 50 | means any of the following: 51 | 52 | (a) any file in Source Code Form that results from an addition to, 53 | deletion from, or modification of the contents of Covered 54 | Software; or 55 | 56 | (b) any new file in Source Code Form that contains any Covered 57 | Software. 58 | 59 | 1.11. "Patent Claims" of a Contributor 60 | means any patent claim(s), including without limitation, method, 61 | process, and apparatus claims, in any patent Licensable by such 62 | Contributor that would be infringed, but for the grant of the 63 | License, by the making, using, selling, offering for sale, having 64 | made, import, or transfer of either its Contributions or its 65 | Contributor Version. 66 | 67 | 1.12. "Secondary License" 68 | means either the GNU General Public License, Version 2.0, the GNU 69 | Lesser General Public License, Version 2.1, the GNU Affero General 70 | Public License, Version 3.0, or any later versions of those 71 | licenses. 72 | 73 | 1.13. "Source Code Form" 74 | means the form of the work preferred for making modifications. 75 | 76 | 1.14. "You" (or "Your") 77 | means an individual or a legal entity exercising rights under this 78 | License. For legal entities, "You" includes any entity that 79 | controls, is controlled by, or is under common control with You. For 80 | purposes of this definition, "control" means (a) the power, direct 81 | or indirect, to cause the direction or management of such entity, 82 | whether by contract or otherwise, or (b) ownership of more than 83 | fifty percent (50%) of the outstanding shares or beneficial 84 | ownership of such entity. 85 | 86 | 2. License Grants and Conditions 87 | -------------------------------- 88 | 89 | 2.1. Grants 90 | 91 | Each Contributor hereby grants You a world-wide, royalty-free, 92 | non-exclusive license: 93 | 94 | (a) under intellectual property rights (other than patent or trademark) 95 | Licensable by such Contributor to use, reproduce, make available, 96 | modify, display, perform, distribute, and otherwise exploit its 97 | Contributions, either on an unmodified basis, with Modifications, or 98 | as part of a Larger Work; and 99 | 100 | (b) under Patent Claims of such Contributor to make, use, sell, offer 101 | for sale, have made, import, and otherwise transfer either its 102 | Contributions or its Contributor Version. 103 | 104 | 2.2. Effective Date 105 | 106 | The licenses granted in Section 2.1 with respect to any Contribution 107 | become effective for each Contribution on the date the Contributor first 108 | distributes such Contribution. 109 | 110 | 2.3. Limitations on Grant Scope 111 | 112 | The licenses granted in this Section 2 are the only rights granted under 113 | this License. No additional rights or licenses will be implied from the 114 | distribution or licensing of Covered Software under this License. 115 | Notwithstanding Section 2.1(b) above, no patent license is granted by a 116 | Contributor: 117 | 118 | (a) for any code that a Contributor has removed from Covered Software; 119 | or 120 | 121 | (b) for infringements caused by: (i) Your and any other third party's 122 | modifications of Covered Software, or (ii) the combination of its 123 | Contributions with other software (except as part of its Contributor 124 | Version); or 125 | 126 | (c) under Patent Claims infringed by Covered Software in the absence of 127 | its Contributions. 128 | 129 | This License does not grant any rights in the trademarks, service marks, 130 | or logos of any Contributor (except as may be necessary to comply with 131 | the notice requirements in Section 3.4). 132 | 133 | 2.4. Subsequent Licenses 134 | 135 | No Contributor makes additional grants as a result of Your choice to 136 | distribute the Covered Software under a subsequent version of this 137 | License (see Section 10.2) or under the terms of a Secondary License (if 138 | permitted under the terms of Section 3.3). 139 | 140 | 2.5. Representation 141 | 142 | Each Contributor represents that the Contributor believes its 143 | Contributions are its original creation(s) or it has sufficient rights 144 | to grant the rights to its Contributions conveyed by this License. 145 | 146 | 2.6. Fair Use 147 | 148 | This License is not intended to limit any rights You have under 149 | applicable copyright doctrines of fair use, fair dealing, or other 150 | equivalents. 151 | 152 | 2.7. Conditions 153 | 154 | Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted 155 | in Section 2.1. 156 | 157 | 3. Responsibilities 158 | ------------------- 159 | 160 | 3.1. Distribution of Source Form 161 | 162 | All distribution of Covered Software in Source Code Form, including any 163 | Modifications that You create or to which You contribute, must be under 164 | the terms of this License. You must inform recipients that the Source 165 | Code Form of the Covered Software is governed by the terms of this 166 | License, and how they can obtain a copy of this License. You may not 167 | attempt to alter or restrict the recipients' rights in the Source Code 168 | Form. 169 | 170 | 3.2. Distribution of Executable Form 171 | 172 | If You distribute Covered Software in Executable Form then: 173 | 174 | (a) such Covered Software must also be made available in Source Code 175 | Form, as described in Section 3.1, and You must inform recipients of 176 | the Executable Form how they can obtain a copy of such Source Code 177 | Form by reasonable means in a timely manner, at a charge no more 178 | than the cost of distribution to the recipient; and 179 | 180 | (b) You may distribute such Executable Form under the terms of this 181 | License, or sublicense it under different terms, provided that the 182 | license for the Executable Form does not attempt to limit or alter 183 | the recipients' rights in the Source Code Form under this License. 184 | 185 | 3.3. Distribution of a Larger Work 186 | 187 | You may create and distribute a Larger Work under terms of Your choice, 188 | provided that You also comply with the requirements of this License for 189 | the Covered Software. If the Larger Work is a combination of Covered 190 | Software with a work governed by one or more Secondary Licenses, and the 191 | Covered Software is not Incompatible With Secondary Licenses, this 192 | License permits You to additionally distribute such Covered Software 193 | under the terms of such Secondary License(s), so that the recipient of 194 | the Larger Work may, at their option, further distribute the Covered 195 | Software under the terms of either this License or such Secondary 196 | License(s). 197 | 198 | 3.4. Notices 199 | 200 | You may not remove or alter the substance of any license notices 201 | (including copyright notices, patent notices, disclaimers of warranty, 202 | or limitations of liability) contained within the Source Code Form of 203 | the Covered Software, except that You may alter any license notices to 204 | the extent required to remedy known factual inaccuracies. 205 | 206 | 3.5. Application of Additional Terms 207 | 208 | You may choose to offer, and to charge a fee for, warranty, support, 209 | indemnity or liability obligations to one or more recipients of Covered 210 | Software. However, You may do so only on Your own behalf, and not on 211 | behalf of any Contributor. You must make it absolutely clear that any 212 | such warranty, support, indemnity, or liability obligation is offered by 213 | You alone, and You hereby agree to indemnify every Contributor for any 214 | liability incurred by such Contributor as a result of warranty, support, 215 | indemnity or liability terms You offer. You may include additional 216 | disclaimers of warranty and limitations of liability specific to any 217 | jurisdiction. 218 | 219 | 4. Inability to Comply Due to Statute or Regulation 220 | --------------------------------------------------- 221 | 222 | If it is impossible for You to comply with any of the terms of this 223 | License with respect to some or all of the Covered Software due to 224 | statute, judicial order, or regulation then You must: (a) comply with 225 | the terms of this License to the maximum extent possible; and (b) 226 | describe the limitations and the code they affect. Such description must 227 | be placed in a text file included with all distributions of the Covered 228 | Software under this License. Except to the extent prohibited by statute 229 | or regulation, such description must be sufficiently detailed for a 230 | recipient of ordinary skill to be able to understand it. 231 | 232 | 5. Termination 233 | -------------- 234 | 235 | 5.1. The rights granted under this License will terminate automatically 236 | if You fail to comply with any of its terms. However, if You become 237 | compliant, then the rights granted under this License from a particular 238 | Contributor are reinstated (a) provisionally, unless and until such 239 | Contributor explicitly and finally terminates Your grants, and (b) on an 240 | ongoing basis, if such Contributor fails to notify You of the 241 | non-compliance by some reasonable means prior to 60 days after You have 242 | come back into compliance. Moreover, Your grants from a particular 243 | Contributor are reinstated on an ongoing basis if such Contributor 244 | notifies You of the non-compliance by some reasonable means, this is the 245 | first time You have received notice of non-compliance with this License 246 | from such Contributor, and You become compliant prior to 30 days after 247 | Your receipt of the notice. 248 | 249 | 5.2. If You initiate litigation against any entity by asserting a patent 250 | infringement claim (excluding declaratory judgment actions, 251 | counter-claims, and cross-claims) alleging that a Contributor Version 252 | directly or indirectly infringes any patent, then the rights granted to 253 | You by any and all Contributors for the Covered Software under Section 254 | 2.1 of this License shall terminate. 255 | 256 | 5.3. In the event of termination under Sections 5.1 or 5.2 above, all 257 | end user license agreements (excluding distributors and resellers) which 258 | have been validly granted by You or Your distributors under this License 259 | prior to termination shall survive termination. 260 | 261 | ************************************************************************ 262 | * * 263 | * 6. Disclaimer of Warranty * 264 | * ------------------------- * 265 | * * 266 | * Covered Software is provided under this License on an "as is" * 267 | * basis, without warranty of any kind, either expressed, implied, or * 268 | * statutory, including, without limitation, warranties that the * 269 | * Covered Software is free of defects, merchantable, fit for a * 270 | * particular purpose or non-infringing. The entire risk as to the * 271 | * quality and performance of the Covered Software is with You. * 272 | * Should any Covered Software prove defective in any respect, You * 273 | * (not any Contributor) assume the cost of any necessary servicing, * 274 | * repair, or correction. This disclaimer of warranty constitutes an * 275 | * essential part of this License. No use of any Covered Software is * 276 | * authorized under this License except under this disclaimer. * 277 | * * 278 | ************************************************************************ 279 | 280 | ************************************************************************ 281 | * * 282 | * 7. Limitation of Liability * 283 | * -------------------------- * 284 | * * 285 | * Under no circumstances and under no legal theory, whether tort * 286 | * (including negligence), contract, or otherwise, shall any * 287 | * Contributor, or anyone who distributes Covered Software as * 288 | * permitted above, be liable to You for any direct, indirect, * 289 | * special, incidental, or consequential damages of any character * 290 | * including, without limitation, damages for lost profits, loss of * 291 | * goodwill, work stoppage, computer failure or malfunction, or any * 292 | * and all other commercial damages or losses, even if such party * 293 | * shall have been informed of the possibility of such damages. This * 294 | * limitation of liability shall not apply to liability for death or * 295 | * personal injury resulting from such party's negligence to the * 296 | * extent applicable law prohibits such limitation. Some * 297 | * jurisdictions do not allow the exclusion or limitation of * 298 | * incidental or consequential damages, so this exclusion and * 299 | * limitation may not apply to You. * 300 | * * 301 | ************************************************************************ 302 | 303 | 8. Litigation 304 | ------------- 305 | 306 | Any litigation relating to this License may be brought only in the 307 | courts of a jurisdiction where the defendant maintains its principal 308 | place of business and such litigation shall be governed by laws of that 309 | jurisdiction, without reference to its conflict-of-law provisions. 310 | Nothing in this Section shall prevent a party's ability to bring 311 | cross-claims or counter-claims. 312 | 313 | 9. Miscellaneous 314 | ---------------- 315 | 316 | This License represents the complete agreement concerning the subject 317 | matter hereof. If any provision of this License is held to be 318 | unenforceable, such provision shall be reformed only to the extent 319 | necessary to make it enforceable. Any law or regulation which provides 320 | that the language of a contract shall be construed against the drafter 321 | shall not be used to construe this License against a Contributor. 322 | 323 | 10. Versions of the License 324 | --------------------------- 325 | 326 | 10.1. New Versions 327 | 328 | Mozilla Foundation is the license steward. Except as provided in Section 329 | 10.3, no one other than the license steward has the right to modify or 330 | publish new versions of this License. Each version will be given a 331 | distinguishing version number. 332 | 333 | 10.2. Effect of New Versions 334 | 335 | You may distribute the Covered Software under the terms of the version 336 | of the License under which You originally received the Covered Software, 337 | or under the terms of any subsequent version published by the license 338 | steward. 339 | 340 | 10.3. Modified Versions 341 | 342 | If you create software not governed by this License, and you want to 343 | create a new license for such software, you may create and use a 344 | modified version of this License if you rename the license and remove 345 | any references to the name of the license steward (except to note that 346 | such modified license differs from this License). 347 | 348 | 10.4. Distributing Source Code Form that is Incompatible With Secondary 349 | Licenses 350 | 351 | If You choose to distribute Source Code Form that is Incompatible With 352 | Secondary Licenses under the terms of this version of the License, the 353 | notice described in Exhibit B of this License must be attached. 354 | 355 | Exhibit A - Source Code Form License Notice 356 | ------------------------------------------- 357 | 358 | This Source Code Form is subject to the terms of the Mozilla Public 359 | License, v. 2.0. If a copy of the MPL was not distributed with this 360 | file, You can obtain one at http://mozilla.org/MPL/2.0/. 361 | 362 | If it is not possible or desirable to put the notice in a particular 363 | file, then You may include the notice in a location (such as a LICENSE 364 | file in a relevant directory) where a recipient would be likely to look 365 | for such a notice. 366 | 367 | You may add additional accurate notices of copyright ownership. 368 | 369 | Exhibit B - "Incompatible With Secondary Licenses" Notice 370 | --------------------------------------------------------- 371 | 372 | This Source Code Form is "Incompatible With Secondary Licenses", as 373 | defined by the Mozilla Public License, v. 2.0. 374 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # jsonschema2go 2 | 3 | See https://github.com/taskcluster/taskcluster/tree/master/tools/jsonschema2go#readme 4 | -------------------------------------------------------------------------------- /jsonschema.go: -------------------------------------------------------------------------------- 1 | // Package jsonschema2go allows you to translate json schemas like this: 2 | // 3 | // { 4 | // "definitions": { 5 | // "activities": { 6 | // "description": "A subset of all known human activities", 7 | // "type": "object", 8 | // "additionalProperties": false, 9 | // "properties": { 10 | // "snooker": { 11 | // "description": "The fine sport of snooker, invented in Madras around 1885", 12 | // "type": "boolean" 13 | // }, 14 | // "cooking": { 15 | // "description": "The act of preparing food for consumption, typically involving the application of heat", 16 | // "type": "boolean" 17 | // } 18 | // }, 19 | // "required": [ 20 | // "cooking", 21 | // "snooker" 22 | // ] 23 | // } 24 | // }, 25 | // "title": "person", 26 | // "description": "A member of the animal kingdom of planet Earth, dominant briefly around 13.8 billion years after the Big Bang", 27 | // "type": "object", 28 | // "additionalProperties": false, 29 | // "properties": { 30 | // "address": { 31 | // "description": "Where the person lives", 32 | // "type": "array", 33 | // "items": { 34 | // "type": "string" 35 | // } 36 | // }, 37 | // "hobbies": { 38 | // "description": "Hobbies the person has", 39 | // "$ref": "#/definitions/activities" 40 | // }, 41 | // "dislikes": { 42 | // "description": "Activities this person dislikes", 43 | // "$ref": "#/definitions/activities" 44 | // } 45 | // }, 46 | // "required": [ 47 | // "address" 48 | // ] 49 | // } 50 | // 51 | // into generated code like this: 52 | // 53 | // // This source code file is AUTO-GENERATED by github.com/taskcluster/jsonschema2go 54 | // 55 | // package main 56 | // 57 | // type ( 58 | // // A subset of all known human activities 59 | // Activities struct { 60 | // 61 | // // The act of preparing food for consumption, typically involving the application of heat 62 | // Cooking bool `json:"cooking"` 63 | // 64 | // // The fine sport of snooker, invented in Madras around 1885 65 | // Snooker bool `json:"snooker"` 66 | // } 67 | // 68 | // // A member of the animal kingdom of planet Earth, dominant briefly around 13.8 billion years after the Big Bang 69 | // Person struct { 70 | // 71 | // // Where the person lives 72 | // Address []string `json:"address"` 73 | // 74 | // // Activities this person dislikes 75 | // Dislikes Activities `json:"dislikes,omitempty"` 76 | // 77 | // // Hobbies the person has 78 | // Hobbies Activities `json:"hobbies,omitempty"` 79 | // } 80 | // ) 81 | // 82 | // This then allows you to json.Unmarshal json data that conforms to a given 83 | // schema into the generated types. By harnessing this library as part of your 84 | // build process, you can ensure that your go types are always in sync with 85 | // your json schemas. 86 | package jsonschema2go 87 | 88 | import ( 89 | "encoding/json" 90 | "errors" 91 | "fmt" 92 | "go/format" 93 | "io" 94 | "io/ioutil" 95 | "log" 96 | "net/http" 97 | "net/url" 98 | "os" 99 | "reflect" 100 | "sort" 101 | "strconv" 102 | "strings" 103 | 104 | "github.com/ghodss/yaml" 105 | "github.com/taskcluster/jsonschema2go/text" 106 | ) 107 | 108 | type ( 109 | // JsonSubSchema represents the data stored in a json subschema. Note that 110 | // all members are backed by pointers, so that nil value can signify 111 | // non-existence. Otherwise we could not differentiate whether a zero 112 | // value is non-existence or actually the zero value. For example, if a 113 | // bool is false, we don't know if it was explictly set to false in the 114 | // json we read, or whether it was not given. Unmarshaling into a pointer 115 | // means pointer will be nil pointer if it wasn't read, or a pointer to 116 | // true/false if it was read from json. 117 | JsonSubSchema struct { 118 | AdditionalItems *bool `json:"additionalItems,omitempty"` 119 | AdditionalProperties *AdditionalProperties `json:"additionalProperties,omitempty"` 120 | AllOf *Items `json:"allOf,omitempty"` 121 | AnyOf *Items `json:"anyOf,omitempty"` 122 | Const *interface{} `json:"const,omitempty"` 123 | Default *interface{} `json:"default,omitempty"` 124 | Definitions *Properties `json:"definitions,omitempty"` 125 | Dependencies map[string]*Dependency `json:"dependencies,omitempty"` 126 | Description *string `json:"description,omitempty"` 127 | Enum []interface{} `json:"enum,omitempty"` 128 | ExclusiveMaximum *bool `json:"exclusiveMaximum,omitempty"` 129 | ExclusiveMinimum *bool `json:"exclusiveMinimum,omitempty"` 130 | Format *string `json:"format,omitempty"` 131 | ID *string `json:"$id,omitempty"` 132 | Items *JsonSubSchema `json:"items,omitempty"` 133 | Maximum *int `json:"maximum,omitempty"` 134 | MaxItems *int `json:"maxItems,omitempty"` 135 | MaxLength *int `json:"maxLength,omitempty"` 136 | MaxProperties *int `json:"maxProperties,omitempty"` 137 | Minimum *int `json:"minimum,omitempty"` 138 | MinItems *int `json:"minItems,omitempty"` 139 | MinLength *int `json:"minLength,omitempty"` 140 | MinProperties *int `json:"minProperties,omitempty"` 141 | MultipleOf *int `json:"multipleOf,omitempty"` 142 | OneOf *Items `json:"oneOf,omitempty"` 143 | Pattern *string `json:"pattern,omitempty"` 144 | PatternProperties *Properties `json:"patternProperties,omitempty"` 145 | Properties *Properties `json:"properties,omitempty"` 146 | Ref *string `json:"$ref,omitempty"` 147 | Required []string `json:"required,omitempty"` 148 | Schema *string `json:"$schema,omitempty"` 149 | Title *string `json:"title,omitempty"` 150 | Type *string `json:"type,omitempty"` 151 | UniqueItems *bool `json:"uniqueItems,omitempty"` 152 | 153 | // non-json fields used for sorting/tracking 154 | 155 | // TypeName is the name of the generated go type that represents this 156 | // JsonSubSchema, e.g. HawkSignatureAuthenticationRequest. If this 157 | // JsonSubSchema does not represent a struct (for example if it 158 | // represents a string, an int, an undefined object, etc), then 159 | // TypeName will be an empty string. 160 | TypeName string `json:"TYPE_NAME"` 161 | 162 | // If this schema is a schema inside a `properties` map of strings to 163 | // schemas of a parent json subschema, PropertyName will be the key 164 | // used in that parent schema to refer to this schema. 165 | // 166 | // If this schema is inside an array (under "items"). 167 | // 168 | // Otherwise, PropertyName will be an empty string. 169 | PropertyName string `json:"PROPERTY_NAME"` 170 | SourceURL string `json:"SOURCE_URL"` 171 | RefSchemaURL string `json:"REF_SCHEMA_URL,omitempty"` 172 | RefSubSchema *JsonSubSchema `json:"REF_SUBSCHEMA,omitempty"` 173 | IsRequired bool `json:"IS_REQUIRED"` 174 | } 175 | 176 | Items struct { 177 | Items []*JsonSubSchema 178 | SourceURL string 179 | } 180 | 181 | Properties struct { 182 | Properties map[string]*JsonSubSchema 183 | MemberNames map[string]string 184 | SortedPropertyNames []string 185 | SourceURL string 186 | } 187 | 188 | AdditionalProperties struct { 189 | Boolean *bool 190 | Properties *JsonSubSchema 191 | } 192 | 193 | Dependency struct { 194 | SchemaDependency *JsonSubSchema 195 | PropertyDependency *[]string 196 | } 197 | 198 | canPopulate interface { 199 | postPopulate(*Job) error 200 | setSourceURL(string) 201 | prepare(*Job) error 202 | } 203 | 204 | NameGenerator func(name string, exported bool, blacklist map[string]bool) (identifier string) 205 | 206 | Job struct { 207 | Package string 208 | ExportTypes bool 209 | HideStructMembers bool 210 | URLs []string 211 | result *Result 212 | TypeNameGenerator NameGenerator 213 | MemberNameGenerator NameGenerator 214 | SkipCodeGen bool 215 | TypeNameBlacklist StringSet 216 | DisableNestedStructs bool 217 | } 218 | 219 | Result struct { 220 | SourceCode []byte 221 | SchemaSet *SchemaSet 222 | } 223 | 224 | // SchemaSet contains the JsonSubSchemas objects read when performing a Job. 225 | SchemaSet struct { 226 | all map[string]*JsonSubSchema 227 | used map[string]*JsonSubSchema 228 | populated []canPopulate 229 | TypeNames StringSet 230 | } 231 | 232 | StringSet map[string]bool 233 | ) 234 | 235 | // Ensure url contains "#" by adding it to end if needed 236 | func sanitizeURL(url string) string { 237 | if strings.ContainsRune(url, '#') { 238 | return url 239 | } 240 | return url + "#" 241 | } 242 | 243 | func (schemaSet *SchemaSet) SubSchema(url string) *JsonSubSchema { 244 | return schemaSet.all[sanitizeURL(url)] 245 | } 246 | 247 | func (schemaSet *SchemaSet) SortedSanitizedURLs() []string { 248 | keys := make([]string, len(schemaSet.used)) 249 | i := 0 250 | for k := range schemaSet.used { 251 | keys[i] = k 252 | i++ 253 | } 254 | sort.Strings(keys) 255 | return keys 256 | } 257 | 258 | // May panic - this is recovered by fmt package, but care should be taken to 259 | // capture panics when calling String() directly 260 | func (subSchema JsonSubSchema) String() string { 261 | v, err := json.Marshal(subSchema) 262 | if err != nil { 263 | panic(err) 264 | } 265 | b, err := yaml.JSONToYAML(v) 266 | if err != nil { 267 | panic(err) 268 | } 269 | return string(b) 270 | } 271 | 272 | func (jsonSubSchema *JsonSubSchema) typeDefinition(disableNested bool, topLevel bool, extraPackages StringSet, rawMessageTypes StringSet) (comment, typ string) { 273 | // Ignore all other properties if this has a $ref, and only redirect to the referened schema. 274 | // See https://tools.ietf.org/html/draft-handrews-json-schema-01#section-8.3: 275 | // `All other properties in a "$ref" object MUST be ignored.` 276 | if p := jsonSubSchema.RefSubSchema; p != nil { 277 | return p.typeDefinition(disableNested, topLevel, extraPackages, rawMessageTypes) 278 | } 279 | comment = "\n" 280 | if d := jsonSubSchema.Description; d != nil { 281 | comment += text.Indent(*d, "// ") 282 | } 283 | if comment[len(comment)-1:] != "\n" { 284 | comment += "\n" 285 | } 286 | if c := jsonSubSchema.Const; c != nil { 287 | comment += "//\n// Constant value: " 288 | switch (*c).(type) { 289 | case float64: 290 | comment += fmt.Sprintf("%v\n", *c) 291 | default: 292 | comment += fmt.Sprintf("%q\n", *c) 293 | } 294 | } 295 | if enum := jsonSubSchema.Enum; enum != nil { 296 | comment += "//\n// Possible values:\n" 297 | for _, i := range enum { 298 | switch i.(type) { 299 | case float64: 300 | comment += fmt.Sprintf("// * %v\n", i) 301 | default: 302 | comment += fmt.Sprintf("// * %q\n", i) 303 | } 304 | } 305 | } 306 | 307 | // Create comments for metadata in a single paragraph. Only start new 308 | // paragraph if we discover after inspecting all possible metadata, that 309 | // something has been specified. If there is no metadata, no need to create 310 | // a new paragraph. 311 | var metadata string 312 | if def := jsonSubSchema.Default; def != nil { 313 | var value string 314 | switch (*def).(type) { 315 | case bool: 316 | value = strconv.FormatBool((*def).(bool)) 317 | case float64: 318 | value = strconv.FormatFloat((*def).(float64), 'g', -1, 64) 319 | default: 320 | v, err := json.MarshalIndent(*def, "", " ") 321 | if err != nil { 322 | panic(fmt.Sprintf("couldn't marshal %+v", *def)) 323 | } 324 | value = string(v) 325 | } 326 | indentedDefault := text.Indent(value+"\n", "// ") 327 | metadata += "// Default: " + indentedDefault[15:] 328 | } 329 | if regex := jsonSubSchema.Pattern; regex != nil { 330 | metadata += "// Syntax: " + *regex + "\n" 331 | } 332 | if minItems := jsonSubSchema.MinLength; minItems != nil { 333 | metadata += "// Min length: " + strconv.Itoa(*minItems) + "\n" 334 | } 335 | if maxItems := jsonSubSchema.MaxLength; maxItems != nil { 336 | metadata += "// Max length: " + strconv.Itoa(*maxItems) + "\n" 337 | } 338 | if minimum := jsonSubSchema.Minimum; minimum != nil { 339 | metadata += "// Mininum: " + strconv.Itoa(*minimum) + "\n" 340 | } 341 | if maximum := jsonSubSchema.Maximum; maximum != nil { 342 | metadata += "// Maximum: " + strconv.Itoa(*maximum) + "\n" 343 | } 344 | if allOf := jsonSubSchema.AllOf; allOf != nil { 345 | metadata += "// All of:\n" 346 | for _, o := range allOf.Items { 347 | metadata += "// * " + o.getTypeName() + "\n" 348 | } 349 | } 350 | if anyOf := jsonSubSchema.AnyOf; anyOf != nil { 351 | metadata += "// Any of:\n" 352 | for _, o := range anyOf.Items { 353 | metadata += "// * " + o.getTypeName() + "\n" 354 | } 355 | } 356 | if oneOf := jsonSubSchema.OneOf; oneOf != nil { 357 | metadata += "// One of:\n" 358 | for _, o := range oneOf.Items { 359 | metadata += "// * " + o.getTypeName() + "\n" 360 | } 361 | } 362 | // Here we check if metadata was specified, and only create new 363 | // paragraph (`//\n`) if something was. 364 | if len(metadata) > 0 { 365 | comment += "//\n" + metadata 366 | } 367 | typ = "json.RawMessage" 368 | if p := jsonSubSchema.Type; p != nil { 369 | typ = *p 370 | } 371 | switch typ { 372 | case "array": 373 | typ = "[]interface{}" 374 | if jsonSubSchema.Items != nil { 375 | arrayComment, arrayType := jsonSubSchema.Items.typeDefinition(disableNested, false, extraPackages, rawMessageTypes) 376 | typ = "[]" + arrayType 377 | // only add array comments if target schema is a primitive type 378 | if jsonSubSchema.Items.TargetSchema().TypeName == "" { 379 | // arrayComment already contains leading newline char (\n) 380 | comment += "//\n// Array items:" + arrayComment 381 | } 382 | } 383 | case "object": 384 | if jsonSubSchema.AnyOf != nil || jsonSubSchema.AllOf != nil || jsonSubSchema.OneOf != nil { 385 | typ = "json.RawMessage" 386 | break 387 | } 388 | ap := jsonSubSchema.AdditionalProperties 389 | noExtraProperties := ap != nil && ap.Boolean != nil && !*ap.Boolean 390 | if noExtraProperties { 391 | // If we are sure no additional properties are allowed, we can 392 | // generate a struct with all allowed property names. 393 | if !topLevel && disableNested { 394 | typ = jsonSubSchema.getTypeName() 395 | } else { 396 | typ = jsonSubSchema.Properties.AsStruct(disableNested, extraPackages, rawMessageTypes) 397 | } 398 | } else if ap != nil && ap.Properties != nil && jsonSubSchema.Properties == nil { 399 | // In the special case no properties have been specified, but 400 | // additionalProperties is an object, we can create a 401 | // map[string]. 402 | subComment, subType := ap.Properties.typeDefinition(disableNested, false, extraPackages, rawMessageTypes) 403 | typ = "map[string]" + subType 404 | // only add subcomments if target schema is a primitive type 405 | if ap.Properties.TargetSchema().TypeName == "" { 406 | // subComment already contains leading newline char (\n) 407 | comment += "//\n// Map entries:" + subComment 408 | } 409 | } else { 410 | // Either *arbitrarily structured* additional properties are 411 | // allowed, or the additional properties are of a fixed form, but 412 | // the explicitly listed properties may not conform to that form, 413 | // so fall back to the most general option to ensure it can hold 414 | // both listed properties and additional properties. 415 | if s := jsonSubSchema.Properties; s != nil { 416 | comment += "//\n// Defined properties:\n//\n" 417 | comment += text.Indent(s.AsStruct(disableNested, extraPackages, rawMessageTypes), "// ") + "\n" 418 | } 419 | if ap != nil && ap.Properties != nil { 420 | comment += "//\n// Additional properties:\n" 421 | subComment, subType := ap.Properties.typeDefinition(disableNested, true, extraPackages, rawMessageTypes) 422 | comment += text.Indent(subComment, "// ") 423 | comment += text.Indent(subType, "// ") + "\n" 424 | } else { 425 | comment += "//\n// Additional properties allowed\n" 426 | } 427 | typ = "json.RawMessage" 428 | } 429 | case "number": 430 | typ = "float64" 431 | case "integer": 432 | typ = "int64" 433 | case "boolean": 434 | typ = "bool" 435 | // json type string maps to go type string, so only need to test case of when 436 | // string is a json date-time, so we can convert to go type Time... 437 | case "string": 438 | if f := jsonSubSchema.Format; f != nil { 439 | if *f == "date-time" { 440 | typ = "tcclient.Time" 441 | extraPackages["tcclient \"github.com/taskcluster/taskcluster/clients/client-go/v24\""] = true 442 | } 443 | } 444 | } 445 | 446 | if URL := jsonSubSchema.SourceURL; URL != "" { 447 | u, err := url.Parse(URL) 448 | if err == nil && u.Scheme != "file" { 449 | comment += "//\n// See " + URL + "\n" 450 | } 451 | } 452 | for strings.Index(comment, "\n//\n") == 0 { 453 | comment = "\n" + comment[4:] 454 | } 455 | 456 | switch typ { 457 | case "json.RawMessage": 458 | extraPackages["\"encoding/json\""] = true 459 | if topLevel { 460 | // Special case: we have here a top level RawMessage such as 461 | // queue.PostArtifactRequest - therefore need to implement 462 | // Marshal and Unmarshal methods. See: 463 | // http://play.golang.org/p/FKHSUmWVFD vs 464 | // http://play.golang.org/p/erjM6ptIYI 465 | extraPackages["\"errors\""] = true 466 | rawMessageTypes[jsonSubSchema.TypeName] = true 467 | } 468 | } 469 | return comment, typ 470 | } 471 | 472 | func (p Properties) String() string { 473 | result := "" 474 | for _, i := range p.SortedPropertyNames { 475 | result += "Property '" + i + "' =\n" + text.Indent(p.Properties[i].String(), " ") 476 | } 477 | return result 478 | } 479 | 480 | func (p *Properties) prepare(job *Job) error { 481 | log.Printf("In PREPARE (properties): %v", p.SourceURL) 482 | for _, j := range p.SortedPropertyNames { 483 | if p.Properties[j].TargetSchema().Properties != nil { 484 | if job.DisableNestedStructs { 485 | job.add(p.Properties[j].TargetSchema()) 486 | } 487 | } 488 | } 489 | return nil 490 | } 491 | 492 | func (p *Properties) postPopulate(job *Job) error { 493 | log.Printf("In POSTPOPULATE (properties): %v", p.SourceURL) 494 | job.result.SchemaSet.populated = append(job.result.SchemaSet.populated, p) 495 | // now all data should be loaded, let's sort the p.Properties 496 | if p.Properties != nil { 497 | p.SortedPropertyNames = make([]string, 0, len(p.Properties)) 498 | for propertyName := range p.Properties { 499 | p.SortedPropertyNames = append(p.SortedPropertyNames, propertyName) 500 | // subschemas need to have SourceURL set 501 | p.Properties[propertyName].setSourceURL(p.SourceURL + "/" + propertyName) 502 | p.Properties[propertyName].PropertyName = propertyName 503 | } 504 | sort.Strings(p.SortedPropertyNames) 505 | members := make(StringSet, len(p.SortedPropertyNames)) 506 | p.MemberNames = make(map[string]string, len(p.SortedPropertyNames)) 507 | for _, j := range p.SortedPropertyNames { 508 | p.MemberNames[j] = job.MemberNameGenerator(j, !job.HideStructMembers, members) 509 | // subschemas also need to be triggered to postPopulate... 510 | err := p.Properties[j].postPopulate(job) 511 | if err != nil { 512 | return err 513 | } 514 | } 515 | } else { 516 | return fmt.Errorf("WEIRD - NO PROPERTIES in %v", p.SourceURL) 517 | } 518 | return nil 519 | } 520 | 521 | func (job *Job) SetTypeName(subSchema *JsonSubSchema, blacklist map[string]bool) { 522 | if r := subSchema.Ref; r != nil { 523 | log.Printf("Not setting type name for %v - has $ref to %v", subSchema.SourceURL, subSchema.RefSubSchema.SourceURL) 524 | job.SetTypeName(subSchema.RefSubSchema, blacklist) 525 | return 526 | } 527 | if subSchema.TypeName != "" { 528 | log.Printf("Type name already set to '%v' for %v", subSchema.TypeName, subSchema.SourceURL) 529 | return 530 | } 531 | log.Printf("Setting type name for %v", subSchema.SourceURL) 532 | // Type names only need to be set for objects and arrays, everything else is a primitive type 533 | subSchema.TypeName = job.TypeNameGenerator(subSchema.TypeNameRaw(), job.ExportTypes, blacklist) 534 | if subSchema.Items != nil { 535 | log.Printf("Type %v is an array - will set type for items too...", subSchema.SourceURL) 536 | subSchema.Items.TargetSchema().PropertyName = subSchema.PropertyName + " entry" 537 | job.SetTypeName(subSchema.Items, blacklist) 538 | } 539 | } 540 | 541 | func (p *Properties) setSourceURL(url string) { 542 | p.SourceURL = url 543 | } 544 | 545 | func (i *Items) UnmarshalJSON(bytes []byte) (err error) { 546 | err = json.Unmarshal(bytes, &i.Items) 547 | return 548 | } 549 | 550 | func (p *Properties) UnmarshalJSON(bytes []byte) (err error) { 551 | err = json.Unmarshal(bytes, &p.Properties) 552 | return 553 | } 554 | 555 | func (d *Dependency) UnmarshalJSON(bytes []byte) (err error) { 556 | s, j := &[]string{}, new(JsonSubSchema) 557 | if err = json.Unmarshal(bytes, s); err == nil { 558 | d.PropertyDependency = s 559 | return 560 | } 561 | if err = json.Unmarshal(bytes, j); err == nil { 562 | d.SchemaDependency = j 563 | } 564 | return 565 | } 566 | 567 | func (aP *AdditionalProperties) UnmarshalJSON(bytes []byte) (err error) { 568 | b, p := new(bool), new(JsonSubSchema) 569 | if err = json.Unmarshal(bytes, b); err == nil { 570 | aP.Boolean = b 571 | return 572 | } 573 | if err = json.Unmarshal(bytes, p); err == nil { 574 | aP.Properties = p 575 | } 576 | return 577 | } 578 | 579 | func (aP AdditionalProperties) String() string { 580 | if aP.Boolean != nil { 581 | return strconv.FormatBool(*aP.Boolean) 582 | } 583 | return aP.Properties.String() 584 | } 585 | 586 | func (items Items) String() string { 587 | result := "" 588 | for i, j := range items.Items { 589 | result += fmt.Sprintf("Item '%v' =\n", i) + text.Indent(j.String(), " ") 590 | } 591 | return result 592 | } 593 | 594 | func (items *Items) prepare(job *Job) error { 595 | log.Printf("In PREPARE (items): %v", items.SourceURL) 596 | for _, j := range (*items).Items { 597 | // add to schemas so we get a type generated for it in source code 598 | job.add(j.TargetSchema()) 599 | } 600 | return nil 601 | } 602 | 603 | func (items *Items) postPopulate(job *Job) error { 604 | log.Printf("In POSTPOPULATE (items): %v", items.SourceURL) 605 | job.result.SchemaSet.populated = append(job.result.SchemaSet.populated, items) 606 | for i, j := range (*items).Items { 607 | j.setSourceURL(items.SourceURL + "[" + strconv.Itoa(i) + "]") 608 | err := j.postPopulate(job) 609 | if err != nil { 610 | return err 611 | } 612 | } 613 | return nil 614 | } 615 | 616 | func (subSchema *JsonSubSchema) TypeNameRaw() string { 617 | switch { 618 | case subSchema.RefSubSchema != nil: 619 | return subSchema.RefSubSchema.TypeNameRaw() 620 | case subSchema.Title != nil && *subSchema.Title != "" && len(*subSchema.Title) < 40: 621 | return *subSchema.Title 622 | case subSchema.PropertyName != "" && len(subSchema.PropertyName) < 40: 623 | return subSchema.PropertyName 624 | case subSchema.Description != nil && *subSchema.Description != "" && len(*subSchema.Description) < 40: 625 | return *subSchema.Description 626 | default: 627 | return "var" 628 | } 629 | } 630 | 631 | func (job *Job) add(subSchema *JsonSubSchema) { 632 | // if we have already included in the schema set, nothing to do... 633 | if _, ok := job.result.SchemaSet.used[subSchema.SourceURL]; ok { 634 | log.Printf("Not adding %v", subSchema.SourceURL) 635 | return 636 | } 637 | log.Printf("Adding %v", subSchema.SourceURL) 638 | job.result.SchemaSet.used[subSchema.SourceURL] = subSchema 639 | job.SetTypeName(subSchema, job.TypeNameBlacklist) 640 | job.result.SchemaSet.TypeNames[subSchema.TypeName] = true 641 | } 642 | 643 | func (items *Items) setSourceURL(url string) { 644 | items.SourceURL = url 645 | } 646 | 647 | func (subSchema *JsonSubSchema) postPopulateIfNotNil(canPopulate canPopulate, job *Job, suffix string) error { 648 | if reflect.ValueOf(canPopulate).IsValid() { 649 | if !reflect.ValueOf(canPopulate).IsNil() { 650 | canPopulate.setSourceURL(subSchema.SourceURL + suffix) 651 | err := canPopulate.postPopulate(job) 652 | if err != nil { 653 | return err 654 | } 655 | } 656 | } 657 | return nil 658 | } 659 | 660 | func (subSchema *JsonSubSchema) link(job *Job) (err error) { 661 | if ref := subSchema.Ref; ref != nil && *ref != "" { 662 | subSchema.RefSubSchema = job.result.SchemaSet.all[subSchema.RefSchemaURL] 663 | if subSchema.RefSubSchema == nil { 664 | return fmt.Errorf("Subschema %v not loaded when updating %v", subSchema.RefSchemaURL, subSchema.SourceURL) 665 | } 666 | log.Printf("Linked %v to %v", subSchema.SourceURL, subSchema.RefSchemaURL) 667 | } else { 668 | log.Printf("Nothing to link in %v", subSchema.SourceURL) 669 | } 670 | return nil 671 | } 672 | 673 | func (subSchema *JsonSubSchema) prepare(job *Job) (err error) { 674 | log.Printf("In PREPARE (subschema): %v", subSchema.SourceURL) 675 | 676 | // If this subschema has Items (anyOf, oneOf, allOf) then we should "copy 677 | // down" properties from this schema into them, since they inherit the 678 | // values in this schema if they don't override them. 679 | subSchema.AllOf.MergeIn(subSchema, map[string]bool{"AllOf": true, "ID": true}) 680 | subSchema.AnyOf.MergeIn(subSchema, map[string]bool{"AnyOf": true, "ID": true}) 681 | subSchema.OneOf.MergeIn(subSchema, map[string]bool{"OneOf": true, "ID": true}) 682 | 683 | subSchema.Type = subSchema.inferType() 684 | 685 | // Mark subschema properties that are in required list as being required (IsRequired property) 686 | for _, req := range subSchema.Required { 687 | if subSchema.Properties != nil { 688 | if subSubSchema, ok := subSchema.Properties.Properties[req]; ok { 689 | subSubSchema.IsRequired = true 690 | } else { 691 | panic(fmt.Sprintf("Schema %v has a required property %v but this property definition cannot be found", subSchema.SourceURL, req)) 692 | } 693 | } 694 | } 695 | 696 | if job.DisableNestedStructs { 697 | // If this subschema is an array of objects, then add the object type to the top level types 698 | if subSchema.Items != nil && subSchema.Items.TargetSchema().Properties != nil { 699 | job.add(subSchema.Items.TargetSchema()) 700 | } 701 | // If this subschema is a map of strings to objects, then add the object type to the top level types 702 | if subSchema.AdditionalProperties != nil && subSchema.AdditionalProperties.Properties != nil && subSchema.AdditionalProperties.Properties.TargetSchema().Properties != nil { 703 | job.add(subSchema.AdditionalProperties.Properties.TargetSchema()) 704 | } 705 | } 706 | return nil 707 | } 708 | 709 | func (subSchema *JsonSubSchema) postPopulate(job *Job) (err error) { 710 | log.Printf("In POSTPOPULATE (subschema): %v", subSchema.SourceURL) 711 | job.result.SchemaSet.populated = append(job.result.SchemaSet.populated, subSchema) 712 | 713 | // Since setSourceURL(string) must be called before postPopulate(*Job), we 714 | // can rely on subSchema.SourceURL being already set. 715 | job.result.SchemaSet.all[subSchema.SourceURL] = subSchema 716 | 717 | // Call postPopulate on sub items of this schema... Use an ARRAY not a MAP 718 | // so we can be sure subSchema.Definitions is processed before anything 719 | // that might reference it 720 | type Subcomponent struct { 721 | subPath string 722 | subItem canPopulate 723 | } 724 | 725 | subcomponents := []Subcomponent{ 726 | {"/definitions", subSchema.Definitions}, 727 | {"/allOf", subSchema.AllOf}, 728 | {"/anyOf", subSchema.AnyOf}, 729 | {"/oneOf", subSchema.OneOf}, 730 | {"/items", subSchema.Items}, 731 | {"/properties", subSchema.Properties}, 732 | } 733 | if subSchema.AdditionalProperties != nil { 734 | subcomponents = append(subcomponents, Subcomponent{"/additionalProperties", subSchema.AdditionalProperties.Properties}) 735 | } 736 | 737 | for _, s := range subcomponents { 738 | err = subSchema.postPopulateIfNotNil(s.subItem, job, s.subPath) 739 | if err != nil { 740 | return 741 | } 742 | } 743 | 744 | // If we have a $ref pointing to another schema, keep a reference so we can 745 | // discover TypeName later when we generate the type definition 746 | if ref := subSchema.Ref; ref != nil && *ref != "" { 747 | // relative references within current document are relatively simple... 748 | if strings.HasPrefix(*ref, "#") { 749 | subSchema.RefSchemaURL = subSchema.SourceURL[:strings.Index(subSchema.SourceURL, "#")] + *ref 750 | return 751 | } 752 | // looks like it's pointing to a different document - better make sure we've loaded/cached it 753 | // first need to determine if id property has been specified for a base url to resolve against... 754 | // see https://json-schema.org/understanding-json-schema/structuring.html#the-id-property 755 | var absURL string 756 | // note json schemas are nested, so we need to get to the root json schema of the document we're in 757 | // using the source URL we can strip off the internal path within the document (everything after '#') 758 | docURLRoot := strings.SplitN(subSchema.SourceURL, "#", 2) 759 | docBaseSchema := job.result.SchemaSet.SubSchema(docURLRoot[0] + "#") 760 | if id := docBaseSchema.ID; id != nil && *id != "" { 761 | // '$id' property is specified in doc, so let's use it! 762 | var refURL *url.URL 763 | refURL, err = url.Parse(*ref) 764 | if err != nil { 765 | return 766 | } 767 | var idURL *url.URL 768 | idURL, err = url.Parse(*id) 769 | if err != nil { 770 | return 771 | } 772 | absURL = idURL.ResolveReference(refURL).String() 773 | } else { 774 | // no '$id' property is specified, we must assume this is an absolute URL 775 | absURL = *subSchema.Ref 776 | } 777 | // make sure the doc is loaded (if in cache it won't be loaded again) 778 | subSchema.RefSubSchema, err = job.cacheJsonSchema(absURL) 779 | if err != nil { 780 | return 781 | } 782 | subSchema.RefSchemaURL = sanitizeURL(absURL) 783 | } 784 | return 785 | } 786 | 787 | func (subSchema *JsonSubSchema) TargetSchema() *JsonSubSchema { 788 | if ref := subSchema.RefSubSchema; ref != nil { 789 | return ref.TargetSchema() 790 | } 791 | return subSchema 792 | } 793 | 794 | // MergeIn copies attributes from subSchema into the subschemas in items.Items 795 | // when they are not currently set. 796 | func (items *Items) MergeIn(subSchema *JsonSubSchema, skipFields StringSet) { 797 | if items == nil || len(items.Items) == 0 { 798 | // nothing to do 799 | return 800 | } 801 | p := reflect.ValueOf(subSchema).Elem() 802 | // loop through all struct fields of Jsonsubschema 803 | for i := 0; i < p.NumField(); i++ { 804 | // don't copy fields that are blacklisted, or that aren't pointers 805 | if skipFields[p.Type().Field(i).Name] || p.Field(i).Kind() != reflect.Ptr { 806 | continue 807 | } 808 | // loop through all items (e.g. the list of oneOf schemas) 809 | for _, item := range items.Items { 810 | c := reflect.ValueOf(item).Elem() 811 | // only replace destination value if it is currently nil 812 | if destination, source := c.Field(i), p.Field(i); destination.IsNil() { 813 | 814 | // To copy the pointer, we would just: 815 | // destination.Set(source) 816 | // However, we want to make copies of the entries, rather than 817 | // copy the pointers, so that future modifications of a copied 818 | // subschema won't update the source schema. Note: this is only 819 | // a top-level copy, not a deep copy, but is better than nothing. 820 | 821 | // dereference the pointer to get the value 822 | targetValue := reflect.Indirect(source) 823 | if targetValue.IsValid() { 824 | // create a new value to store it 825 | newValue := reflect.New(targetValue.Type()).Elem() 826 | // copy the value into the new value 827 | newValue.Set(targetValue) 828 | // create a new pointer to point to the new value 829 | newPointer := reflect.New(targetValue.Addr().Type()).Elem() 830 | // set that pointer to the address of the new value 831 | newPointer.Set(newValue.Addr()) 832 | // copy the new pointer to the destination 833 | destination.Set(newPointer) 834 | } 835 | } 836 | // If we wanted to "move" instead of "copy", we could reset source 837 | // to nil with: 838 | // source.Set(reflect.Zero(source.Type())) 839 | } 840 | } 841 | } 842 | 843 | func (subSchema *JsonSubSchema) setSourceURL(url string) { 844 | subSchema.SourceURL = url 845 | } 846 | 847 | func (job *Job) loadJsonSchema(URL string) (subSchema *JsonSubSchema, err error) { 848 | log.Printf("Loading %v", URL) 849 | var body io.ReadCloser 850 | if strings.HasPrefix(URL, "file://") { 851 | body, err = os.Open(URL[7 : len(URL)-1]) // need to strip trailing '#' 852 | if err != nil { 853 | return 854 | } 855 | } else { 856 | var u *url.URL 857 | u, err = url.Parse(URL) 858 | if err != nil { 859 | return 860 | } 861 | var resp *http.Response 862 | // TODO: may be better to use https://golang.org/pkg/net/http/#NewFileTransport here?? 863 | switch u.Scheme { 864 | case "http", "https": 865 | resp, err = http.Get(URL) 866 | if err != nil { 867 | return subSchema, err 868 | } 869 | body = resp.Body 870 | default: 871 | return nil, fmt.Errorf("Unknown scheme '%s' for URL '%s'", u.Scheme, URL) 872 | } 873 | } 874 | defer body.Close() 875 | data, err := ioutil.ReadAll(body) 876 | if err != nil { 877 | return 878 | } 879 | // json is valid YAML, so we can safely convert, even if it is already json 880 | j, err := yaml.YAMLToJSON(data) 881 | if err != nil { 882 | return 883 | } 884 | subSchema = new(JsonSubSchema) 885 | err = json.Unmarshal(j, subSchema) 886 | if err != nil { 887 | return 888 | } 889 | subSchema.setSourceURL(sanitizeURL(URL)) 890 | err = subSchema.postPopulate(job) 891 | return 892 | } 893 | 894 | func (job *Job) cacheJsonSchema(url string) (*JsonSubSchema, error) { 895 | // if url is not provided, there is nothing to download 896 | if url == "" { 897 | return nil, errors.New("Empty url in cacheJsonSchema") 898 | } 899 | sanitizedURL := sanitizeURL(url) 900 | // only fetch if we haven't fetched already... 901 | if _, ok := job.result.SchemaSet.all[sanitizedURL]; ok { 902 | log.Printf("Schema already cached: %v", url) 903 | return job.result.SchemaSet.SubSchema(sanitizedURL), nil 904 | } 905 | 906 | // The URL we load here could be a subschema nested inside a root document, e.g. 907 | // https://foo.com/schema.json#/definitions/bar/monkey 908 | // Therefore we need to load the schema from the URL portion up to the '#' char 909 | // (this char is guaranteed to be present) but return the particular subschema 910 | // that is located at the path underneath. 911 | 912 | // Containing document URL (sanitized URL up to and including '#' char) 913 | rootSchemaURL := sanitizedURL[:strings.Index(sanitizedURL, "#")+1] 914 | 915 | // Path to subschema from root of parent document (sanitized URL after '#' char) 916 | subschemaPath := sanitizedURL[strings.Index(sanitizedURL, "#")+1:] 917 | 918 | _, err := job.loadJsonSchema(rootSchemaURL) 919 | if err != nil { 920 | return nil, err 921 | } 922 | 923 | // check that the required subschema is contained in the document we loaded 924 | subschema, found := job.result.SchemaSet.all[sanitizedURL] 925 | if !found { 926 | return nil, fmt.Errorf("Subschema %v not found under URL %v", subschemaPath, rootSchemaURL) 927 | } 928 | return subschema, nil 929 | } 930 | 931 | // This is where we generate nested and compoound types in go to represent json payloads 932 | // which are used as inputs and outputs for the REST API endpoints, and also for Pulse 933 | // message bodies for the Exchange APIs. 934 | // Returns the generated code content, and a map of keys of extra packages to import, e.g. 935 | // a generated type might use time.Time, so if not imported, this would have to be added. 936 | // using a map of strings -> bool to simulate a set - true => include 937 | func generateGoTypes(disableNested bool, schemaSet *SchemaSet) (string, StringSet, StringSet) { 938 | extraPackages := make(StringSet) 939 | rawMessageTypes := make(StringSet) 940 | content := "type (" // intentionally no \n here since each type starts with one already 941 | // Loop through all json schemas that were found referenced inside the API json schemas... 942 | typeDefinitions := make(map[string]string) 943 | typeNames := make([]string, 0, len(schemaSet.used)) 944 | for _, i := range schemaSet.used { 945 | log.Printf("Type name: '%v' - %v", i.getTypeName(), i.SourceURL) 946 | var newComment, newType string 947 | newComment, newType = i.typeDefinition(disableNested, true, extraPackages, rawMessageTypes) 948 | typeDefinitions[i.TypeName] = text.Indent(newComment+i.TypeName+" "+newType, "\t") 949 | typeNames = append(typeNames, i.getTypeName()) 950 | } 951 | sort.Strings(typeNames) 952 | for _, t := range typeNames { 953 | content += typeDefinitions[t] + "\n" 954 | } 955 | return content + ")\n\n", extraPackages, rawMessageTypes 956 | } 957 | 958 | func (job *Job) Execute() (*Result, error) { 959 | // Generate normalised names for schemas. Keep a record of generated type 960 | // names, so that we don't reuse old names. Set acts like a set 961 | // of strings. 962 | job.result = new(Result) 963 | job.result.SchemaSet = &SchemaSet{ 964 | all: make(map[string]*JsonSubSchema), 965 | used: make(map[string]*JsonSubSchema), 966 | populated: make([]canPopulate, 0, len(job.URLs)), 967 | TypeNames: make(StringSet), 968 | } 969 | if job.TypeNameBlacklist == nil { 970 | job.TypeNameBlacklist = make(StringSet) 971 | } 972 | if job.TypeNameGenerator == nil { 973 | job.TypeNameGenerator = text.GoIdentifierFrom 974 | } 975 | if job.MemberNameGenerator == nil { 976 | job.MemberNameGenerator = text.GoIdentifierFrom 977 | } 978 | for _, URL := range job.URLs { 979 | j, err := job.cacheJsonSchema(URL) 980 | if err != nil { 981 | return nil, err 982 | } 983 | // note we don't add inside cacheJsonSchema/loadJsonSchema 984 | // since we don't want to add e.g. top level items if only 985 | // definitions inside the schema are referenced 986 | job.add(j.TargetSchema()) 987 | } 988 | for _, subSchema := range job.result.SchemaSet.all { 989 | err := subSchema.link(job) 990 | if err != nil { 991 | return nil, err 992 | } 993 | } 994 | for _, cp := range job.result.SchemaSet.populated { 995 | err := cp.prepare(job) 996 | if err != nil { 997 | return nil, err 998 | } 999 | } 1000 | 1001 | var err error 1002 | if job.SkipCodeGen { 1003 | return job.result, err 1004 | } 1005 | types, extraPackages, rawMessageTypes := generateGoTypes(job.DisableNestedStructs, job.result.SchemaSet) 1006 | content := `// This source code file is AUTO-GENERATED by github.com/taskcluster/jsonschema2go 1007 | 1008 | package ` + job.Package + ` 1009 | 1010 | ` 1011 | extraPackagesContent := "" 1012 | for j, k := range extraPackages { 1013 | if k { 1014 | extraPackagesContent += text.Indent(""+j+"\n", "\t") 1015 | } 1016 | } 1017 | 1018 | if extraPackagesContent != "" { 1019 | content += `import ( 1020 | ` + extraPackagesContent + `) 1021 | 1022 | ` 1023 | } 1024 | content += types 1025 | content += jsonRawMessageImplementors(rawMessageTypes) 1026 | // format it 1027 | job.result.SourceCode, err = format.Source([]byte(content)) 1028 | if err != nil { 1029 | err = fmt.Errorf("Formatting error: %v\n%v", err, content) 1030 | } 1031 | return job.result, err 1032 | // imports should be good, so no need to run 1033 | // https://godoc.org/golang.org/x/tools/imports#Process 1034 | } 1035 | 1036 | func jsonRawMessageImplementors(rawMessageTypes StringSet) string { 1037 | // first sort the order of the rawMessageTypes since when we rebuild, we 1038 | // don't want to generate functions in a different order and introduce 1039 | // diffs against the previous version 1040 | sortedRawMessageTypes := make([]string, len(rawMessageTypes)) 1041 | i := 0 1042 | for goType := range rawMessageTypes { 1043 | sortedRawMessageTypes[i] = goType 1044 | i++ 1045 | } 1046 | sort.Strings(sortedRawMessageTypes) 1047 | content := "" 1048 | for _, goType := range sortedRawMessageTypes { 1049 | content += ` 1050 | 1051 | // MarshalJSON calls json.RawMessage method of the same name. Required since 1052 | // ` + goType + ` is of type json.RawMessage... 1053 | func (this *` + goType + `) MarshalJSON() ([]byte, error) { 1054 | x := json.RawMessage(*this) 1055 | return (&x).MarshalJSON() 1056 | } 1057 | 1058 | // UnmarshalJSON is a copy of the json.RawMessage implementation. 1059 | func (this *` + goType + `) UnmarshalJSON(data []byte) error { 1060 | if this == nil { 1061 | return errors.New("` + goType + `: UnmarshalJSON on nil pointer") 1062 | } 1063 | *this = append((*this)[0:0], data...) 1064 | return nil 1065 | }` 1066 | } 1067 | return content 1068 | } 1069 | 1070 | func (s *Properties) AsStruct(disableNested bool, extraPackages StringSet, rawMessageTypes StringSet) (typ string) { 1071 | typ = fmt.Sprintf("struct {\n") 1072 | if s != nil { 1073 | for _, j := range s.SortedPropertyNames { 1074 | // recursive call to build structs inside structs 1075 | var subComment, subType string 1076 | subMember := s.MemberNames[j] 1077 | subComment, subType = s.Properties[j].typeDefinition(disableNested, false, extraPackages, rawMessageTypes) 1078 | jsonStructTagOptions := "" 1079 | if !s.Properties[j].IsRequired { 1080 | jsonStructTagOptions = ",omitempty" 1081 | } 1082 | // struct member name and type, as part of struct definition 1083 | typ += text.Indent(fmt.Sprintf("%v%v %v `json:\"%v%v\"`", subComment, subMember, subType, j, jsonStructTagOptions), "\t") + "\n" 1084 | } 1085 | } 1086 | typ += "}" 1087 | return 1088 | } 1089 | 1090 | func (jsonSubSchema *JsonSubSchema) getTypeName() string { 1091 | if jsonSubSchema.Ref != nil { 1092 | return jsonSubSchema.RefSubSchema.getTypeName() 1093 | } 1094 | return jsonSubSchema.TypeName 1095 | } 1096 | 1097 | // inferType is a cheeky little function that tries to set the type, if it can 1098 | // infer it from other information, such as if all OneOf subschemas share the 1099 | // same type, for example. 1100 | func (subSchema *JsonSubSchema) inferType() *string { 1101 | 1102 | // 1) If already set, nothing to do... 1103 | if subSchema.Type != nil { 1104 | return subSchema.Type 1105 | } 1106 | 1107 | // 2) See if we can infer from existence of `properties` or `items` 1108 | var inferredType string 1109 | switch { 1110 | case subSchema.Properties != nil: 1111 | inferredType = "object" 1112 | case subSchema.Items != nil: 1113 | inferredType = "array" 1114 | } 1115 | if inferredType != "" { 1116 | return &inferredType 1117 | } 1118 | 1119 | // 3) If all items in subSchema.AllOf/subSchema.AnyOf/subSchema.OneOf have 1120 | // same type, we can infer that is the type 1121 | for _, items := range []*Items{ 1122 | subSchema.AllOf, 1123 | subSchema.AnyOf, 1124 | subSchema.OneOf, 1125 | } { 1126 | if items != nil { 1127 | for _, subSubSchema := range items.Items { 1128 | subType := subSubSchema.inferType() 1129 | if subType == nil { 1130 | return nil 1131 | } 1132 | if inferredType == "" { 1133 | inferredType = *subType 1134 | continue 1135 | } 1136 | if inferredType != *subType { 1137 | return nil 1138 | } 1139 | } 1140 | return &inferredType 1141 | } 1142 | } 1143 | 1144 | // 4) If const is set, infer from that 1145 | if subSchema.Const != nil { 1146 | return jsonSchemaTypeFromValue(*subSchema.Const) 1147 | } 1148 | 1149 | // 5) If an enum, see if all entries have same type 1150 | for _, enumItem := range subSchema.Enum { 1151 | enumType := jsonSchemaTypeFromValue(enumItem) 1152 | if inferredType == "" { 1153 | inferredType = *enumType 1154 | continue 1155 | } 1156 | if inferredType != *enumType { 1157 | return nil 1158 | } 1159 | } 1160 | if inferredType != "" { 1161 | return &inferredType 1162 | } 1163 | 1164 | // 6) Cannot infer type 1165 | return nil 1166 | } 1167 | 1168 | func jsonSchemaTypeFromValue(v interface{}) *string { 1169 | var inferredType string 1170 | switch t := v.(type) { 1171 | case bool: 1172 | inferredType = "boolean" 1173 | case float64: 1174 | inferredType = "number" 1175 | case string: 1176 | inferredType = "string" 1177 | case []interface{}: 1178 | inferredType = "array" 1179 | case map[string]interface{}: 1180 | inferredType = "object" 1181 | case nil: 1182 | inferredType = "null" 1183 | default: 1184 | log.Fatalf("What the? %v", t) 1185 | } 1186 | return &inferredType 1187 | } 1188 | -------------------------------------------------------------------------------- /jsonschema2go/cli.go: -------------------------------------------------------------------------------- 1 | // jsonschema2go is the command invoked by go generate in order to generate the go client library. 2 | package main 3 | 4 | import ( 5 | "bufio" 6 | "encoding/json" 7 | "fmt" 8 | "io" 9 | "log" 10 | "os" 11 | 12 | docopt "github.com/docopt/docopt-go" 13 | "github.com/taskcluster/jsonschema2go" 14 | ) 15 | 16 | func readStringStrip(reader *bufio.Reader, delimeter byte) (string, error) { 17 | token, err := reader.ReadString(delimeter) 18 | if err != nil { 19 | return "", err 20 | } 21 | // strip delimeter from end of string 22 | if token != "" { 23 | token = token[:len(token)-1] 24 | } 25 | return token, nil 26 | } 27 | 28 | func parseStandardIn() []string { 29 | results := []string{} 30 | reader := bufio.NewReader(os.Stdin) 31 | for { 32 | url, err := readStringStrip(reader, '\n') 33 | if err == io.EOF { 34 | break 35 | } 36 | exitOnFail(err) 37 | results = append(results, url) 38 | } 39 | return results 40 | } 41 | 42 | var ( 43 | version = "jsonschema2go 1.0" 44 | usage = ` 45 | jsonschema2go 46 | jsonschema2go generates go source code from json schema inputs. Specifically, 47 | it returns a []byte of source code that can be written to a file, for all 48 | objects found in the provided json schemas, plus any schemas that they 49 | reference. It will automatically download json schema definitions referred to 50 | in the provided schemas, if there are cross references to external json schemas 51 | hosted on an available url (i.e. $ref property of json schema). You pass urls 52 | via standard in (one per line), e.g. by generating a list of schema urls and 53 | then piping to jsonschema2go -o . 54 | 55 | The go type names will be "normalised" from the json subschema Title element. 56 | 57 | Example: 58 | cat urls.txt | jsonschema2go -o main 59 | 60 | Usage: 61 | jsonschema2go -o GO-PACKAGE-NAME 62 | jsonschema2go --help 63 | 64 | Options: 65 | -h --help Display this help text. 66 | -o GO-PACKAGE-NAME The package name to use in the generated file. 67 | ` 68 | ) 69 | 70 | func main() { 71 | // Parse the docopt string and exit on any error or help message. 72 | arguments, err := docopt.Parse(usage, nil, true, version, false, true) 73 | exitOnFail(err) 74 | job := &jsonschema2go.Job{ 75 | Package: arguments["-o"].(string), 76 | ExportTypes: true, 77 | URLs: parseStandardIn(), 78 | DisableNestedStructs: true, 79 | } 80 | result, err := job.Execute() 81 | if err != nil { 82 | log.Printf("%#v", err) 83 | switch j := err.(type) { 84 | case *json.UnmarshalTypeError: 85 | log.Printf("Error: %v", j.Error()) 86 | log.Printf("Field: %v", j.Field) 87 | } 88 | } 89 | exitOnFail(err) 90 | // simply output the generated file name, in the case of success, for 91 | // super-easy parsing 92 | fmt.Println(string(result.SourceCode)) 93 | } 94 | 95 | func exitOnFail(err error) { 96 | if err != nil { 97 | fmt.Printf("%v\n%T\n", err, err) 98 | panic(err) 99 | } 100 | } 101 | -------------------------------------------------------------------------------- /text/text.go: -------------------------------------------------------------------------------- 1 | // Package text contains utility functions for manipulating raw text strings 2 | package text 3 | 4 | import ( 5 | "fmt" 6 | "strings" 7 | "unicode" 8 | "unicode/utf8" 9 | 10 | "github.com/fatih/camelcase" 11 | ) 12 | 13 | // See https://golang.org/ref/spec#Keywords 14 | var reservedKeyWords = map[string]bool{ 15 | "break": true, 16 | "case": true, 17 | "chan": true, 18 | "const": true, 19 | "continue": true, 20 | "default": true, 21 | "defer": true, 22 | "else": true, 23 | "fallthrough": true, 24 | "for": true, 25 | "func": true, 26 | "go": true, 27 | "goto": true, 28 | "if": true, 29 | "import": true, 30 | "interface": true, 31 | "map": true, 32 | "package": true, 33 | "range": true, 34 | "return": true, 35 | "select": true, 36 | "struct": true, 37 | "switch": true, 38 | "type": true, 39 | "var": true, 40 | } 41 | 42 | // taken from https://github.com/golang/lint/blob/32a87160691b3c96046c0c678fe57c5bef761456/lint.go#L702 43 | var commonInitialisms = map[string]bool{ 44 | "API": true, 45 | "ASCII": true, 46 | "CPU": true, 47 | "CSS": true, 48 | "DNS": true, 49 | "EOF": true, 50 | "GUID": true, 51 | "HTML": true, 52 | "HTTP": true, 53 | "HTTPS": true, 54 | "ID": true, 55 | "IP": true, 56 | "JSON": true, 57 | "LHS": true, 58 | "OS": true, 59 | "QPS": true, 60 | "RAM": true, 61 | "RHS": true, 62 | "RPC": true, 63 | "SLA": true, 64 | "SMTP": true, 65 | "SQL": true, 66 | "SSH": true, 67 | "TCP": true, 68 | "TLS": true, 69 | "TTL": true, 70 | "UDP": true, 71 | "UI": true, 72 | "UID": true, 73 | "UUID": true, 74 | "URI": true, 75 | "URL": true, 76 | "UTF8": true, 77 | "VM": true, 78 | "XML": true, 79 | "XSRF": true, 80 | "XSS": true, 81 | } 82 | 83 | // Indent indents a block of text with an indent string. It does this by 84 | // placing the given indent string at the front of every line, except on the 85 | // last line, if the last line has no characters. This special treatment 86 | // simplifies the generation of nested text structures. 87 | func Indent(text, indent string) string { 88 | if text == "" { 89 | return text 90 | } 91 | if text[len(text)-1:] == "\n" { 92 | result := "" 93 | for _, j := range strings.Split(text[:len(text)-1], "\n") { 94 | result += indent + j + "\n" 95 | } 96 | return result 97 | } 98 | result := "" 99 | for _, j := range strings.Split(strings.TrimRight(text, "\n"), "\n") { 100 | result += indent + j + "\n" 101 | } 102 | return result[:len(result)-1] 103 | } 104 | 105 | // Underline returns the provided text together with a new line character and a 106 | // line of "=" characters whose length is equal to the maximum line length in 107 | // the provided text, followed by a final newline character. 108 | func Underline(text string) string { 109 | var maxlen int 110 | for _, j := range strings.Split(text, "\n") { 111 | if len(j) > maxlen { 112 | maxlen = len(j) 113 | } 114 | } 115 | return text + "\n" + strings.Repeat("=", maxlen) + "\n" 116 | } 117 | 118 | // Returns a string of the same length, filled with "*"s. 119 | func StarOut(text string) string { 120 | return strings.Repeat("*", len(text)) 121 | } 122 | 123 | // GoIdentifierFrom provides a mechanism to mutate an arbitrary descriptive 124 | // string (name) into a Go identifier (variable name, function name, etc) that 125 | // e.g. can be used in generated code, taking into account a blacklist of names 126 | // that should not be used, plus the blacklist of the go language reserved key 127 | // words (https://golang.org/ref/spec#Keywords), in order to guarantee that a 128 | // new name is created which will not conflict with an existing type. 129 | // 130 | // Identifier syntax: https://golang.org/ref/spec#Identifiers 131 | // 132 | // Strategy to convert arbitrary unicode string to a valid identifier: 133 | // 134 | // 1) Ensure name is valid UTF-8; if not, replace it with empty string 135 | // 136 | // 2) Split name into arrays of allowed runes (words), by considering a run of 137 | // disallowed unicode characters to act as a separator, where allowed runes 138 | // include unicode letters, unicode numbers, and '_' character (disallowed 139 | // runes are discarded) 140 | // 141 | // 3) Split words further into sub words, by decomposing camel case words as 142 | // per https://github.com/fatih/camelcase#usage-and-examples 143 | // 144 | // 4) Designate the case of all subwords of all words to be uppercase, with the 145 | // exception of the first subword of the first word, which should be lowercase 146 | // if exported is false, otherwise uppercase 147 | // 148 | // 5) For each subword of each word, adjust as follows: if designated as 149 | // lowercase, lowercase all characters of the subword; if designated as 150 | // uppercase, then if recognised as a common "initialism", then uppercase all 151 | // the characters of the subword, otherwise uppercase only the first character 152 | // of the subword. Common "Initialisms" are defined as per: 153 | // https://github.com/golang/lint/blob/32a87160691b3c96046c0c678fe57c5bef761456/lint.go#L702 154 | // 155 | // 6) Rejoin subwords to form a single word 156 | // 157 | // 7) Rejoin words into a single string 158 | // 159 | // 8) If the string starts with a number, add a leading `_` 160 | // 161 | // 9) If the string is the empty string or "_", set as "Identifier" 162 | // 163 | // 10) If the resulting identifier is in the given blacklist, or the list of 164 | // reserved key words (https://golang.org/ref/spec#Keywords), append the lowest 165 | // integer possible, >= 1, that results in no blacklist conflict 166 | // 167 | // 11) Add the new name to the given blacklist 168 | // 169 | // Note, the `map[string]bool` construction is simply a mechanism to implement 170 | // set semantics; a value of `true` signifies inclusion in the set. 171 | // Non-existence is equivalent to existence with a value of `false`; therefore 172 | // it is recommended to only store `true` values. 173 | func GoIdentifierFrom(name string, exported bool, blacklist map[string]bool) (identifier string) { 174 | if !utf8.ValidString(name) { 175 | name = "" 176 | } 177 | for i, word := range strings.FieldsFunc( 178 | name, 179 | func(c rune) bool { 180 | return !unicode.IsLetter(c) && !unicode.IsNumber(c) && c != '_' 181 | }, 182 | ) { 183 | caseAdaptedWord := "" 184 | for j, subWord := range camelcase.Split(word) { 185 | caseAdaptedWord += fixCase(subWord, i == 0 && j == 0 && !exported) 186 | } 187 | identifier += caseAdaptedWord 188 | } 189 | 190 | if strings.IndexFunc( 191 | identifier, 192 | func(c rune) bool { 193 | return unicode.IsNumber(c) 194 | }, 195 | ) == 0 { 196 | identifier = "_" + identifier 197 | } 198 | 199 | if identifier == "" || identifier == "_" { 200 | identifier = "Identifier" 201 | } 202 | 203 | // If name already exists, add an integer suffix to name. Start with "1" and increment 204 | // by 1 until an unused name is found. Example: if name FooBar was generated four times 205 | // , the first instance would be called FooBar, then the next would be FooBar1, the next 206 | // FooBar2 and the last would be assigned a name of FooBar3. We do this to guarantee we 207 | // don't use duplicate names for different logical entities. 208 | for k, baseName := 1, identifier; blacklist[identifier] || reservedKeyWords[identifier]; { 209 | identifier = fmt.Sprintf("%v%v", baseName, k) 210 | k++ 211 | } 212 | blacklist[identifier] = true 213 | return 214 | } 215 | 216 | func fixCase(word string, makeLower bool) string { 217 | if word == "" { 218 | return "" 219 | } 220 | if makeLower { 221 | return strings.ToLower(word) 222 | } 223 | upper := strings.ToUpper(word) 224 | if commonInitialisms[upper] { 225 | return upper 226 | } 227 | firstRune, size := utf8.DecodeRuneInString(word) 228 | remainingString := word[size:] 229 | return string(unicode.ToUpper(firstRune)) + remainingString 230 | } 231 | 232 | // Returns the indefinite article (in English) for a the given noun, which is 233 | // 'an' for nouns beginning with a vowel, otherwise 'a'. 234 | func IndefiniteArticle(noun string) string { 235 | if strings.ContainsRune("AEIOUaeiou", rune(noun[0])) { 236 | return "an" 237 | } 238 | return "a" 239 | } 240 | -------------------------------------------------------------------------------- /text/text_test.go: -------------------------------------------------------------------------------- 1 | package text_test 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/taskcluster/jsonschema2go/text" 7 | ) 8 | 9 | func ExampleIndent_basic() { 10 | fmt.Println("1.") 11 | fmt.Println(text.Indent("", "....")) 12 | fmt.Println("2.") 13 | fmt.Println(text.Indent("\n", "....")) 14 | fmt.Println("3.") 15 | fmt.Println(text.Indent("line one\nline two", "....")) 16 | fmt.Println("4.") 17 | fmt.Println(text.Indent("line one\nline two\n", "....")) 18 | fmt.Println("5.") 19 | fmt.Println(text.Indent("line one\nline two\n\n", "....")) 20 | fmt.Println("Done") 21 | 22 | // Output: 23 | // 1. 24 | // 25 | // 2. 26 | // .... 27 | // 28 | // 3. 29 | // ....line one 30 | // ....line two 31 | // 4. 32 | // ....line one 33 | // ....line two 34 | // 35 | // 5. 36 | // ....line one 37 | // ....line two 38 | // .... 39 | // 40 | // Done 41 | } 42 | 43 | func ExampleIndent_nested() { 44 | fmt.Println(text.Indent("func A(foo string) {\n"+text.Indent("a := []string{\n"+text.Indent("\"x\",\n\"y\",\n\"z\",\n", "\t")+"}\n", "\t")+"}\n", "=> ")) 45 | fmt.Println("Done") 46 | 47 | // Output: 48 | // => func A(foo string) { 49 | // => a := []string{ 50 | // => "x", 51 | // => "y", 52 | // => "z", 53 | // => } 54 | // => } 55 | // 56 | // Done 57 | } 58 | 59 | func ExampleUnderline_basic() { 60 | fmt.Println(text.Underline("Taskcluster Client") + "Please see https://docs.taskcluster.net/manual/tools/clients") 61 | 62 | // Output: 63 | // Taskcluster Client 64 | // ================== 65 | // Please see https://docs.taskcluster.net/manual/tools/clients 66 | } 67 | 68 | func ExampleUnderline_multiline() { 69 | fmt.Println(text.Underline("Taskcluster Client\nGo (golang) Implementation\n13 Jan 2016") + "Please see http://taskcluster.github.io/taskcluster-client-go") 70 | 71 | // Output: 72 | // Taskcluster Client 73 | // Go (golang) Implementation 74 | // 13 Jan 2016 75 | // ========================== 76 | // Please see http://taskcluster.github.io/taskcluster-client-go 77 | } 78 | 79 | func ExampleIndefiniteArticle() { 80 | for _, noun := range []string{ 81 | "ant", 82 | "dog", 83 | "emu", 84 | "fish", 85 | "gopher", 86 | "hippopotamus", 87 | "owl", 88 | } { 89 | fmt.Println(text.IndefiniteArticle(noun), noun) 90 | } 91 | 92 | // Output: 93 | // an ant 94 | // a dog 95 | // an emu 96 | // a fish 97 | // a gopher 98 | // a hippopotamus 99 | // an owl 100 | } 101 | 102 | func ExampleGoIdentifierFrom() { 103 | blacklist := make(map[string]bool) 104 | fmt.Println(text.GoIdentifierFrom("Azure Artifact Request", true, blacklist)) 105 | fmt.Println(text.GoIdentifierFrom("AzureArtifactRequest", true, blacklist)) 106 | fmt.Println(text.GoIdentifierFrom("Azure artifact request", false, blacklist)) 107 | fmt.Println(text.GoIdentifierFrom("azure-artifact request", false, blacklist)) 108 | fmt.Println(text.GoIdentifierFrom("azure-artifact request", true, blacklist)) 109 | fmt.Println(text.GoIdentifierFrom("List Artifacts Response", true, blacklist)) 110 | fmt.Println(text.GoIdentifierFrom("hello, 世;;;((```[]!@#$界", false, blacklist)) 111 | fmt.Println(text.GoIdentifierFrom(".-4$sjdb2##f \n\txxßßß", true, blacklist)) 112 | fmt.Println(text.GoIdentifierFrom("", false, blacklist)) 113 | fmt.Println(text.GoIdentifierFrom("", true, blacklist)) 114 | fmt.Println(text.GoIdentifierFrom("_", false, blacklist)) 115 | fmt.Println(text.GoIdentifierFrom("grüß", true, blacklist)) 116 | fmt.Println(text.GoIdentifierFrom("333", false, blacklist)) 117 | fmt.Println(text.GoIdentifierFrom("3_33", true, blacklist)) 118 | fmt.Println(text.GoIdentifierFrom("ü", true, blacklist)) 119 | fmt.Println(text.GoIdentifierFrom("üö33", true, blacklist)) 120 | fmt.Println(text.GoIdentifierFrom("Üö33", false, blacklist)) 121 | fmt.Println(text.GoIdentifierFrom("Üö33", true, blacklist)) 122 | fmt.Println(text.GoIdentifierFrom("\xe2\x28\xa1", true, blacklist)) 123 | fmt.Println(text.GoIdentifierFrom("provisioner id", true, blacklist)) 124 | fmt.Println(text.GoIdentifierFrom("provisioner ide", true, blacklist)) 125 | fmt.Println(text.GoIdentifierFrom("provisionerId", true, blacklist)) 126 | fmt.Println(text.GoIdentifierFrom("provisionerId parent", true, blacklist)) 127 | fmt.Println(text.GoIdentifierFrom("provisionerId parent ", true, blacklist)) 128 | fmt.Println(text.GoIdentifierFrom("urlEndpoint", false, blacklist)) 129 | fmt.Println(text.GoIdentifierFrom("uRLEndpoint", false, blacklist)) 130 | fmt.Println(text.GoIdentifierFrom("URLEndpoint", true, blacklist)) 131 | fmt.Println(text.GoIdentifierFrom("UrlEndpoint", true, blacklist)) 132 | fmt.Println(text.GoIdentifierFrom("UrlEndpoint", false, blacklist)) 133 | fmt.Println(text.GoIdentifierFrom("PDFDocument", false, blacklist)) 134 | fmt.Println(text.GoIdentifierFrom("continue", false, blacklist)) 135 | 136 | // Output: 137 | // AzureArtifactRequest 138 | // AzureArtifactRequest1 139 | // azureArtifactRequest 140 | // azureArtifactRequest1 141 | // AzureArtifactRequest2 142 | // ListArtifactsResponse 143 | // hello世界 144 | // _4Sjdb2FXxßßß 145 | // Identifier 146 | // Identifier1 147 | // Identifier2 148 | // Grüß 149 | // _333 150 | // _3_33 151 | // Ü 152 | // Üö33 153 | // üö33 154 | // Üö331 155 | // Identifier3 156 | // ProvisionerID 157 | // ProvisionerIde 158 | // ProvisionerID1 159 | // ProvisionerIDParent 160 | // ProvisionerIDParent1 161 | // urlEndpoint 162 | // uRLEndpoint 163 | // URLEndpoint 164 | // URLEndpoint1 165 | // urlEndpoint1 166 | // pdfDocument 167 | // continue1 168 | } 169 | --------------------------------------------------------------------------------