├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md └── bin └── json_schema /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. 6 | 7 | ## Our Standards 8 | 9 | Examples of behavior that contributes to creating a positive environment include: 10 | 11 | * Using welcoming and inclusive language 12 | * Being respectful of differing viewpoints and experiences 13 | * Gracefully accepting constructive criticism 14 | * Focusing on what is best for the community 15 | * Showing empathy towards other community members 16 | 17 | Examples of unacceptable behavior by participants include: 18 | 19 | * The use of sexualized language or imagery and unwelcome sexual attention or advances 20 | * Trolling, insulting/derogatory comments, and personal or political attacks 21 | * Public or private harassment 22 | * Publishing others' private information, such as a physical or electronic address, without explicit permission 23 | * Other conduct which could reasonably be considered inappropriate in a professional setting 24 | 25 | ## Our Responsibilities 26 | 27 | Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. 28 | 29 | Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. 30 | 31 | ## Scope 32 | 33 | This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. 34 | 35 | ## Enforcement 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at noah@onemorebug.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. 38 | 39 | Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. 40 | 41 | ## Attribution 42 | 43 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] 44 | 45 | [homepage]: http://contributor-covenant.org 46 | [version]: http://contributor-covenant.org/version/1/4/ 47 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2015-2019 Noah Sussman New Media, LLC 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); you 4 | may not use this library except in compliance with the License. You 5 | may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 12 | implied. See the License for the specific language governing 13 | permissions and limitations under the License. 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | JSON schema dump command 2 | ======== 3 | 4 | _A wrapper for [jq](https://stedolan.github.io/jq/). Dumps the schemas 5 | (as determined by `jq`) from a JSON document._ 6 | 7 | A `jq` wrapper that prints back a high-level view of the schema of a 8 | JSON document. 9 | 10 | `json_schema` is a bash script that does two things: 11 | 12 | 1. List unique paths using valid `jq` queries that you can then use 13 | to retrieve those paths if you wish. 14 | 2. Count how many times each _unique_ path occurs in the document. 15 | 16 | ### List the unique paths in a JSON document 17 | 18 | Each unique path in the JSON document is printed alongside a count of 19 | how many times that unique path occurs in the document. 20 | 21 | ### Count how many times each unique path occurs 22 | 23 | For each unique path, a count of occurrences is printed to the left 24 | of the path. 25 | 26 | The path itself is shown as a valid `jq` expression that you can then 27 | use to access resources at paths you have identified. 28 | 29 | ## Synopsis 30 | 31 | Use it like this: 32 | 33 | json_schema foo.json 34 | 35 | If the contents of `foo.json` were the following: 36 | 37 | { 38 | "a": "foo", 39 | "b": "bar", 40 | "c": [1, 2, 3, {"hello": "world"}, 4, 5], 41 | "d": [] 42 | } 43 | 44 | Then that should produce output like this: 45 | 46 | 1 .["a"] 47 | 1 .["b"] 48 | 1 .["c"] 49 | 6 .["c"][] 50 | 1 .["c"][]["hello"] 51 | 1 .["d"] 52 | 53 | 54 | ### Sample output from NASA near-Earth objects report 55 | 56 | This is what `json_schema` looks like when applied to a larger JSON document. 57 | 58 | curl -so near_earth_asteroids.json \ 59 | https://data.nasa.gov/resource/2vr3-k9wn.json 60 | 61 | json_schema near_earth_asteroids.json 62 | 63 | Then the output should look like: 64 | 65 | 202 .[] 66 | 202 .[]["designation"] 67 | 202 .[]["discovery_date"] 68 | 181 .[]["h_mag"] 69 | 202 .[]["i_deg"] 70 | 202 .[]["moid_au"] 71 | 202 .[]["orbit_class"] 72 | 200 .[]["period_yr"] 73 | 202 .[]["pha"] 74 | 202 .[]["q_au_1"] 75 | 200 .[]["q_au_2"] 76 | 77 | As you can see, `json_schema` presents a compact, high-level view of a 78 | document that is composed of over 200 records. For instance, it is now 79 | possible to see that the field `h_mag` is used much less than the other 80 | fields in the `near_earth_asteroids.json` document. 81 | -------------------------------------------------------------------------------- /bin/json_schema: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | # Copyright 2014-2019 Noah Sussman New Media, LLC 4 | # Copyright 2019-2020 Noah Sussman New Media, LLC and Daniel Beecham 5 | 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | 18 | 19 | # Set 'strict mode'; 'set -e' will make the whole script exit if a 20 | # command fails instead of continuing on the next line, 'set -u' 21 | # will treat references to unset variables as an error and exit 22 | # immediately. 23 | set -eu 24 | 25 | if test ! "$(command -v jq)"; then 26 | echo "ERROR: jq is not in your path!" 27 | exit 1 28 | fi 29 | 30 | jq --raw-output \ 31 | 'paths 32 | | map(. as $item 33 | | type 34 | | if . == "number" then 35 | "[]" 36 | else 37 | "[\"" + $item + "\"]" 38 | end) 39 | | join("") 40 | | "." + .' \ 41 | "$@" \ 42 | | sort \ 43 | | uniq -c 44 | --------------------------------------------------------------------------------