├── .gitignore ├── .travis.yml ├── LICENSE.md ├── README.md ├── csv-dialect-description-format.json ├── data-package.json ├── definitions.json ├── fiscal-data-package.json ├── fiscal-data-package.jsonld ├── index.html ├── json-table-schema.json ├── registry.csv ├── registry.json ├── tabular-data-package.json └── tests ├── __init__.py ├── test_registry.py ├── test_registry_json.py └── test_schemas.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | node_modules/* 3 | .idea 4 | .cache/ 5 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - 3.4 4 | sudo: false 5 | install: 6 | - pip install jsonschema 7 | - pip install requests 8 | script: 9 | - python -m unittest discover 10 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # JSON Schemas for Frictionless Data Specifications 2 | 3 | [![Build Status](http://travis-ci.org/frictionlessdata/schemas.svg?branch=master)](http://travis-ci.org/frictionlessdata/schemas) 4 | 5 | JSON Schemas, and a registry, for the Data Package family of specifications. Read more about Data Packages at [Frictionless Data](http://frictionlessdata.io/). 6 | 7 | The schemas are implemented using [JSON Schema](http://json-schema.org/), a specification which provides a simple declarative format for describing the structure of JSON documents. 8 | 9 | The registry is implemented as simple CSV file, and there are libraries in [JavaScript](http://github.com/frictionlessdata/datapackage-registry-js) and [Python](https://github.com/frictionlessdata/datapackage-py) that work with the registry directly. 10 | 11 | ## The schemas 12 | 13 | Here you'll find schemas for [Data Package](http://frictionlessdata.io/data-packages/), various Data Package Profiles, [JSON Table Schemas](http://frictionlessdata.io/json-table-schema/), [CSV Dialect Description Format](http://frictionlessdata.io/csv-dialect/) and more. 14 | 15 | Note that some of the schemas also feature information for [json-editor](http://github.com/jdorn/json-editor) - useful for building web forms and other UI components dynamically from a schema. We use this extensively in [DataPackagist](http://github.com/okfn/datapackagist) to build UIs for creating Data Packages. 16 | 17 | ## The registry 18 | 19 | The registry enables consumers to get access to schemas and documentation for the family of Data Package specifications, and related specifications like JSON Table Schema and CSV Dialect Description Format. See [Frictionless Data](http://frictionlessdata.io/) for more information. 20 | 21 | ### Contributing 22 | 23 | Yes we welcome and encourage additions to the registry! Any spec that is added must meet the following criteria: 24 | 25 | * Be related to the Data Packages family of specifications. 26 | * Have a publicly-accessible web page describing the specification. 27 | * Have a JSON Schema file that describes the specification. 28 | 29 | See the existing entries in the registry, and then take the following steps to add a new entry: 30 | 31 | 1. Make a new pull request called `registry/{NAME_OF_SPECIFICATION}` 32 | 2. The pull request features a JSON Schema file for the new specification, and adds the spec to `registry.csv` 33 | 3. Write a brief description of the spec as part of the pull request. 34 | -------------------------------------------------------------------------------- /csv-dialect-description-format.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "title": "CSVDDF", 4 | "description": "JSON Schema for validating CSVDDF dialect structures", 5 | "type": "object", 6 | "properties": { 7 | "delimiter": { 8 | "type": "string" 9 | }, 10 | "doublequote": { 11 | "type": "boolean" 12 | }, 13 | "lineterminator": { 14 | "type": "string" 15 | }, 16 | "quotechar": { 17 | "type": "string" 18 | }, 19 | "skipinitialspace": { 20 | "type": "boolean" 21 | } 22 | }, 23 | "required": [ 24 | "delimiter", 25 | "doublequote", 26 | "lineterminator", 27 | "quotechar", 28 | "skipinitialspace" 29 | ] 30 | } 31 | -------------------------------------------------------------------------------- /data-package.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "title": "Data Package", 4 | "description": "Data Package is a simple specification for data access and delivery.", 5 | "type": "object", 6 | "required": [ "name", "resources" ], 7 | "properties": { 8 | "name": { 9 | "$ref": "definitions.json#/define/name", 10 | "propertyOrder": 10 11 | }, 12 | "title": { 13 | "$ref": "definitions.json#/define/title", 14 | "propertyOrder": 20 15 | }, 16 | "description": { 17 | "$ref": "definitions.json#/define/description", 18 | "format": "textarea", 19 | "propertyOrder": 30 20 | }, 21 | "homepage": { 22 | "$ref": "definitions.json#/define/homepage", 23 | "propertyOrder": 40 24 | }, 25 | "version": { 26 | "$ref": "definitions.json#/define/version", 27 | "propertyOrder": 50 28 | }, 29 | "license": { 30 | "$ref": "definitions.json#/define/license", 31 | "propertyOrder": 60 32 | }, 33 | "author": { 34 | "$ref": "definitions.json#/define/author", 35 | "propertyOrder": 70 36 | }, 37 | "contributors": { 38 | "$ref": "definitions.json#/define/contributors", 39 | "propertyOrder": 80, 40 | "options": { "hidden": true } 41 | }, 42 | "resources": { 43 | "title": "Resources", 44 | "description": "The data resources that this package describes.", 45 | "type": "array", 46 | "propertyOrder": 90, 47 | "minItems": 0, 48 | "items": { 49 | "type": "object", 50 | "properties": { 51 | "name": { 52 | "$ref": "definitions.json#/define/name", 53 | "propertyOrder": 10 54 | }, 55 | "title": { 56 | "$ref": "definitions.json#/define/title", 57 | "propertyOrder": 20 58 | }, 59 | "description": { 60 | "$ref": "definitions.json#/define/description", 61 | "propertyOrder": 30, 62 | "format": "textarea" 63 | }, 64 | "schema": { 65 | "$ref": "definitions.json#/define/schema", 66 | "propertyOrder": 40 67 | }, 68 | "url": { 69 | "$ref": "definitions.json#/define/url", 70 | "propertyOrder": 50 71 | }, 72 | "path": { 73 | "$ref": "definitions.json#/define/path", 74 | "propertyOrder": 60 75 | }, 76 | "data": { 77 | "$ref": "definitions.json#/define/data", 78 | "propertyOrder": 70 79 | }, 80 | "format": { 81 | "$ref": "definitions.json#/define/format", 82 | "propertyOrder": 80 83 | }, 84 | "mediatype": { 85 | "$ref": "definitions.json#/define/mediatype", 86 | "propertyOrder": 90 87 | }, 88 | "encoding": { 89 | "$ref": "definitions.json#/define/encoding", 90 | "propertyOrder": 100 91 | }, 92 | "bytes": { 93 | "$ref": "definitions.json#/define/bytes", 94 | "propertyOrder": 110, 95 | "options": { "hidden": true } 96 | }, 97 | "hash": { 98 | "$ref": "definitions.json#/define/hash", 99 | "propertyOrder": 120, 100 | "options": { "hidden": true } 101 | }, 102 | "dialect": { 103 | "$ref": "definitions.json#/define/dialect", 104 | "propertyOrder": 130, 105 | "options": { "hidden": true } 106 | }, 107 | "sources": { 108 | "$ref": "definitions.json#/define/sources", 109 | "propertyOrder": 140, 110 | "options": { "hidden": true } 111 | }, 112 | "license": { 113 | "$ref": "definitions.json#/define/license", 114 | "description": "The license under which the resource is published.", 115 | "propertyOrder": 150, 116 | "options": { "hidden": true } 117 | } 118 | }, 119 | "anyOf": [ 120 | { "title": "url required", "required": ["url"] }, 121 | { "title": "path required", "required": ["path"] }, 122 | { "title": "data required", "required": ["data"] } 123 | ] 124 | } 125 | }, 126 | "keywords": { 127 | "$ref": "definitions.json#/define/keywords", 128 | "propertyOrder": 100 129 | }, 130 | "sources": { 131 | "$ref": "definitions.json#/define/sources", 132 | "propertyOrder": 110, 133 | "options": { "hidden": true } 134 | }, 135 | "image": { 136 | "$ref": "definitions.json#/define/image", 137 | "propertyOrder": 120, 138 | "options": { "hidden": true } 139 | }, 140 | "dataDependencies": { 141 | "$ref": "definitions.json#/define/dataDependencies", 142 | "propertyOrder": 140, 143 | "options": { "hidden": true } 144 | } 145 | } 146 | } 147 | -------------------------------------------------------------------------------- /definitions.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "define": { 4 | "schema": { 5 | "title": "Schema", 6 | "description": "The JSON Table Schema that describes of this resource.", 7 | "type": "object", 8 | "properties": { 9 | "fields": { 10 | "type": "array", 11 | "minItems": 0, 12 | "items": { 13 | "type": "object", 14 | "properties": { 15 | "name": { 16 | "type": "string", 17 | "propertyOrder": 10 18 | }, 19 | "title": { 20 | "type": "string", 21 | "propertyOrder": 20 22 | }, 23 | "description": { 24 | "type": "string", 25 | "propertyOrder": 30 26 | }, 27 | "type": { 28 | "type": "string", 29 | "enum": [ 30 | "string", 31 | "number", 32 | "integer", 33 | "boolean", 34 | "object", 35 | "array", 36 | "datetime", 37 | "date", 38 | "time", 39 | "duration", 40 | "geopoint", 41 | "geojson", 42 | "any" 43 | ], 44 | "propertyOrder": 40 45 | }, 46 | "format": { 47 | "type": "string", 48 | "propertyOrder": 50 49 | } 50 | } 51 | } 52 | } 53 | } 54 | }, 55 | "name": { 56 | "title": "Name", 57 | "description": "An identifier for this package. Lower case characters with '.', '_' and '-' are allowed.", 58 | "type": "string", 59 | "pattern": "^([a-z0-9._-])+$" 60 | }, 61 | "title": { 62 | "title": "Title", 63 | "description": "A human-readable title.", 64 | "type": "string" 65 | }, 66 | "description": { 67 | "title": "Description", 68 | "description": "A text description.", 69 | "type": "string" 70 | }, 71 | "homepage": { 72 | "title": "Home Page", 73 | "description": "The URL for this data package's website.", 74 | "type": "string" 75 | }, 76 | "version": { 77 | "title": "Version", 78 | "description": "A unique version number for this package.", 79 | "type": "string" 80 | }, 81 | "url": { 82 | "title": "URL", 83 | "description": "The URL for this resource.", 84 | "type": "string" 85 | }, 86 | "path": { 87 | "title": "Path", 88 | "description": "The relative path to this resource.", 89 | "type": "string" 90 | }, 91 | "data": { 92 | "title": "Data", 93 | "description": "The inline data for this resource.", 94 | "anyOf": [ 95 | { "type": "string" }, 96 | { "type": "array" }, 97 | { "type": "object" } 98 | ] 99 | }, 100 | "format": { 101 | "title": "Format", 102 | "description": "The file format of this resource.", 103 | "type": "string" 104 | }, 105 | "mediatype": { 106 | "title": "Media Type", 107 | "description": "The media type of this resource.", 108 | "type": "string", 109 | "pattern": "^(.+)/(.+)$" 110 | }, 111 | "encoding": { 112 | "title": "Encoding", 113 | "description": "The file encoding of this resource.", 114 | "type": "string" 115 | }, 116 | "bytes": { 117 | "title": "Bytes", 118 | "description": "The size of this resource in bytes.", 119 | "type": "integer" 120 | }, 121 | "hash": { 122 | "title": "Hash", 123 | "type": "string", 124 | "description": "The MD5 hash of this resource. Indicate other hashing algorithms with the {algorithm}:{hash} format.", 125 | "pattern": "^([^:]+:[a-fA-F0-9]+|[a-fA-F0-9]{32}|)$" 126 | }, 127 | "dialect": { 128 | "title": "Dialect", 129 | "description": "The dialect of this resource file type.", 130 | "type": "object" 131 | }, 132 | "author": { 133 | "title": "Author", 134 | "description": "A contributor to this package.", 135 | "oneOf": [ 136 | { "type": "string" }, 137 | { 138 | "type": "object", 139 | "properties": { 140 | "name": { "type": "string" }, 141 | "email": { "type": "string" }, 142 | "web": { "type": "string" } 143 | }, 144 | "required": [ "name" ] 145 | } 146 | ] 147 | }, 148 | "contributors": { 149 | "title": "Contributors", 150 | "description": "The contributors to this package.", 151 | "type": "array", 152 | "items": { 153 | "$ref": "#/define/author" 154 | } 155 | }, 156 | "license": { 157 | "title": "License", 158 | "description": "The license under which this package is published.", 159 | "oneOf": [ 160 | { "type": "string" }, 161 | { 162 | "type": "object", 163 | "properties": { 164 | "type": { "type": "string" }, 165 | "url": { "type": "string" } 166 | }, 167 | "anyOf": [ 168 | { "title": "type required", "required": [ "type" ] }, 169 | { "title": "url required", "required": [ "url" ] } 170 | ] 171 | } 172 | ] 173 | }, 174 | "sources": { 175 | "title": "Sources", 176 | "description": "The raw sources for this resource.", 177 | "type": "array", 178 | "minItems": 0, 179 | "items": { 180 | "type": "object", 181 | "properties": { 182 | "name": { "type": "string" }, 183 | "web": { "type": "string" }, 184 | "email": { "type": "string" } 185 | }, 186 | "anyOf": [ 187 | { "title": "name required", "required": ["name"] }, 188 | { "title": "web required", "required": ["web"] }, 189 | { "title": "email required", "required": ["email"] } 190 | ] 191 | } 192 | }, 193 | "keywords": { 194 | "title": "Keywords", 195 | "description": "A list of keywords that describe this package.", 196 | "type": "array", 197 | "items": { "type": "string" } 198 | }, 199 | "image": { 200 | "title": "Image", 201 | "description": "A image to represent this package.", 202 | "type": "string" 203 | }, 204 | "dataDependencies": { 205 | "title": "Data Dependencies", 206 | "description": "Additional Data Packages required to install this package.", 207 | "type": "object" 208 | }, 209 | "countryCode": { 210 | "title": "ISO 3166-1 Alpha-2 Country code", 211 | "description": "A valid 2-digit ISO country code (ISO 3166-1 alpha-2), or, an array of valid ISO codes.", 212 | "oneOf": [ 213 | { "type": "string", "pattern": "^[A-Z]{2}$" }, 214 | { 215 | "type": "array", 216 | "minItems": 1, 217 | "items": { "type": "string", "pattern": "^[A-Z]{2}$" } 218 | } 219 | ] 220 | } 221 | } 222 | } 223 | -------------------------------------------------------------------------------- /fiscal-data-package.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "title": "Fiscal Data Package", 4 | "description": "Fiscal Data Package is a simple specification for data access and delivery of fiscal data.", 5 | "type": "object", 6 | "required": [ "name", "title", "resources", "model" ], 7 | "properties": { 8 | "name": { 9 | "$ref": "definitions.json#/define/name", 10 | "propertyOrder": 10 11 | }, 12 | "title": { 13 | "$ref": "definitions.json#/define/title", 14 | "propertyOrder": 20 15 | }, 16 | "description": { 17 | "$ref": "definitions.json#/define/description", 18 | "format": "textarea", 19 | "propertyOrder": 30 20 | }, 21 | "homepage": { 22 | "$ref": "definitions.json#/define/homepage", 23 | "propertyOrder": 40 24 | }, 25 | "version": { 26 | "$ref": "definitions.json#/define/version", 27 | "propertyOrder": 50 28 | }, 29 | "license": { 30 | "$ref": "definitions.json#/define/license", 31 | "propertyOrder": 60 32 | }, 33 | "author": { 34 | "$ref": "definitions.json#/define/author", 35 | "propertyOrder": 70 36 | }, 37 | "contributors": { 38 | "$ref": "definitions.json#/define/contributors", 39 | "propertyOrder": 80, 40 | "options": { "hidden": true } 41 | }, 42 | "resources": { 43 | "title": "Resources", 44 | "description": "The data resources that this package describes.", 45 | "type": "array", 46 | "propertyOrder": 90, 47 | "minItems": 0, 48 | "items": { 49 | "type": "object", 50 | "properties": { 51 | "name": { 52 | "$ref": "definitions.json#/define/name", 53 | "propertyOrder": 10 54 | }, 55 | "title": { 56 | "$ref": "definitions.json#/define/title", 57 | "propertyOrder": 20 58 | }, 59 | "description": { 60 | "$ref": "definitions.json#/define/description", 61 | "propertyOrder": 30, 62 | "format": "textarea" 63 | }, 64 | "schema": { 65 | "$ref": "definitions.json#/define/schema", 66 | "propertyOrder": 40 67 | }, 68 | "url": { 69 | "$ref": "definitions.json#/define/url", 70 | "propertyOrder": 50 71 | }, 72 | "path": { 73 | "$ref": "definitions.json#/define/path", 74 | "propertyOrder": 60 75 | }, 76 | "data": { 77 | "$ref": "definitions.json#/define/data", 78 | "propertyOrder": 70 79 | }, 80 | "format": { 81 | "$ref": "definitions.json#/define/format", 82 | "propertyOrder": 80 83 | }, 84 | "mediatype": { 85 | "$ref": "definitions.json#/define/mediatype", 86 | "propertyOrder": 90 87 | }, 88 | "encoding": { 89 | "$ref": "definitions.json#/define/encoding", 90 | "propertyOrder": 100 91 | }, 92 | "bytes": { 93 | "$ref": "definitions.json#/define/bytes", 94 | "propertyOrder": 110, 95 | "options": { "hidden": true } 96 | }, 97 | "hash": { 98 | "$ref": "definitions.json#/define/hash", 99 | "propertyOrder": 120, 100 | "options": { "hidden": true } 101 | }, 102 | "dialect": { 103 | "$ref": "definitions.json#/define/dialect", 104 | "propertyOrder": 130, 105 | "options": { "hidden": true } 106 | }, 107 | "sources": { 108 | "$ref": "definitions.json#/define/sources", 109 | "propertyOrder": 140, 110 | "options": { "hidden": true } 111 | }, 112 | "license": { 113 | "$ref": "definitions.json#/define/license", 114 | "description": "The license under which the resource is published.", 115 | "propertyOrder": 150, 116 | "options": { "hidden": true } 117 | } 118 | }, 119 | "anyOf": [ 120 | { "title": "url required", "required": ["url"] }, 121 | { "title": "path required", "required": ["path"] }, 122 | { "title": "data required", "required": ["data"] } 123 | ] 124 | } 125 | }, 126 | "keywords": { 127 | "$ref": "definitions.json#/define/keywords", 128 | "propertyOrder": 100 129 | }, 130 | "sources": { 131 | "$ref": "definitions.json#/define/sources", 132 | "propertyOrder": 110, 133 | "options": { "hidden": true } 134 | }, 135 | "image": { 136 | "$ref": "definitions.json#/define/image", 137 | "propertyOrder": 120, 138 | "options": { "hidden": true } 139 | }, 140 | "base": { 141 | "$ref": "definitions.json#/define/base", 142 | "propertyOrder": 130, 143 | "options": { "hidden": true } 144 | }, 145 | "dataDependencies": { 146 | "$ref": "definitions.json#/define/dataDependencies", 147 | "propertyOrder": 140, 148 | "options": { "hidden": true } 149 | }, 150 | "countryCode": { "$ref": "definitions.json#/define/countryCode" }, 151 | "granularity": { 152 | "title": "Granularity of resources", 153 | "description": "A keyword that represents the type of spend data, eiter aggregated or transactional", 154 | "type": "string", 155 | "enum": ["aggregated", "transactional"] 156 | }, 157 | "fiscalPeriod": { 158 | "title": "Fiscal period for the budget", 159 | "description": "The fiscal period of the dataset", 160 | "type": "object", 161 | "properties": { 162 | "start": { 163 | "type": "string", 164 | "pattern": "^\\d{4}-(0?[1-9]|1[012])-(0?[1-9]|[12]\\d|3[01])" 165 | }, 166 | "end": { 167 | "type": "string", 168 | "pattern": "^\\d{4}-(0?[1-9]|1[012])-(0?[1-9]|[12]\\d|3[01])" 169 | } 170 | }, 171 | "required": ["start"] 172 | }, 173 | "model": { 174 | "title": "Model", 175 | "description": "The 'logical model' for the data", 176 | "type": "object", 177 | "properties": { 178 | "measures": { 179 | "title": "Measures", 180 | "description": "Measures are numerical and correspond to financial amounts in the source data.", 181 | "type": "object", 182 | "patternProperties": { 183 | "^\\w+": { 184 | "type": "object", 185 | "properties": { 186 | "source": { "type": "string" }, 187 | "resource": { "type": "string" }, 188 | "currency": { "type": "string", "pattern": "^[A-Z]{3}$" }, 189 | "factor": { "type": "number" }, 190 | "direction": { 191 | "title": "Direction of the spending", 192 | "description": "A keyword that represents the direction of the spend, either expenditure or revenue.", 193 | "type": "string", 194 | "enum": [ "expenditure", "revenue" ] 195 | }, 196 | "phase": { 197 | "title": "Budget phase", 198 | "description": "A keyword that represents the phase of the data, can be proposed for a budget proposal, approved for an approved budget, adjusted for modified budget or executed for the enacted budget", 199 | "type": "string", 200 | "enum": ["proposed", "approved", "adjusted", "executed"] 201 | } 202 | }, 203 | "required": [ "source", "currency" ] 204 | } 205 | } 206 | }, 207 | "dimensions": { 208 | "title": "Dimensions", 209 | "description": "Dimensions are groups of related fields. Dimensions cover all items other than the measure.", 210 | "type": "object", 211 | "patternProperties": { 212 | "^\\w+": { 213 | "type": "object", 214 | "properties": { 215 | "attributes": { 216 | "title": "Attributes", 217 | "description": "Attribute objects that make up the dimension", 218 | "type": "object", 219 | "minItems": 1, 220 | "patternProperties": { 221 | "^\\w+": { 222 | "type": "object", 223 | "properties": { 224 | "source": { "type": "string" }, 225 | "resource": { "type": "string" }, 226 | "constant": { 227 | "oneOf": [ 228 | { "type": "string" }, 229 | { "type": "number" } 230 | ] 231 | }, 232 | "parent": { "type": "string" }, 233 | "labelfor": { "type": "string" } 234 | }, 235 | "required": [ "source" ] 236 | } 237 | } 238 | }, 239 | "primaryKey": { 240 | "title": "Primary Key", 241 | "description": "Either an array of strings corresponding to the name attributes in a set of field objects in the fields array or a single string corresponding to one of these names. The value of primaryKey indicates the primary key or primary keys for the dimension.", 242 | "oneOf": [ 243 | { "type": "string" }, 244 | { 245 | "type": "array", 246 | "minItems": 1, 247 | "items": { "type": "string" } 248 | } 249 | ] 250 | }, 251 | "dimensionType": { 252 | "title": "Dimension Type", 253 | "description": "Describes what kind of a dimension it is.", 254 | "type": "string", 255 | "enum": [ 256 | "datetime", 257 | "entity", 258 | "classification", 259 | "activity", 260 | "fact", 261 | "location", 262 | "other" 263 | ] 264 | }, 265 | "classificationType": { 266 | "title": "Classification Type", 267 | "description": "The type of the classification.", 268 | "enum": [ "functional", "administrative", "economic" ] 269 | } 270 | }, 271 | "required": [ "attributes", "primaryKey" ] 272 | } 273 | } 274 | } 275 | }, 276 | "required": [ "measures", "dimensions" ] 277 | } 278 | } 279 | } 280 | -------------------------------------------------------------------------------- /fiscal-data-package.jsonld: -------------------------------------------------------------------------------- 1 | { 2 | "@context": { 3 | "@vocab": "http://schemas.frictionlessdata.io/fiscal-data-package#" 4 | } 5 | } 6 | -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Data Package Schemas | Open Knowledge 10 | 11 | 12 |

Data Package Schemas

13 |

See the registry.

14 | 15 | 16 | -------------------------------------------------------------------------------- /json-table-schema.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "title": "JSON Table Schema", 4 | "description": "JSON Table Schema is a specification for describing tabular data.", 5 | "type": "object", 6 | "properties": { 7 | "fields": { 8 | "type": "array", 9 | "minItems": 1, 10 | "items": { 11 | "type": "object", 12 | "properties": { 13 | "name": { "type": "string" }, 14 | "title": { "type": "string" }, 15 | "description": { "type": "string" }, 16 | "type": { 17 | "enum": [ 18 | "string", 19 | "number", 20 | "integer", 21 | "date", 22 | "time", 23 | "datetime", 24 | "boolean", 25 | "object", 26 | "geopoint", 27 | "geojson", 28 | "array", 29 | "duration", 30 | "any" 31 | ] 32 | }, 33 | "rdfType": {"type": "string" }, 34 | "format": { "type": "string" }, 35 | "constraints": { 36 | "type": "object", 37 | "properties": { 38 | "required": { "type": "boolean" }, 39 | "minLength": { "type": "integer" }, 40 | "maxLength": { "type": "integer" }, 41 | "unique": { "type": "boolean" }, 42 | "pattern": { "type": "string" }, 43 | "minimum": { 44 | "oneOf": [ { "type": "string" }, { "type": "number" } ] 45 | }, 46 | "maximum": { 47 | "oneOf": [ {"type": "string"}, {"type": "number"} ] 48 | }, 49 | "enum": { 50 | "type": "array", 51 | "minItems": 1, 52 | "uniqueItems": true 53 | } 54 | } 55 | } 56 | }, 57 | "required": ["name"] 58 | } 59 | }, 60 | "primaryKey": { 61 | "oneOf": [ { "type": "string" }, { "type": "array" } ] 62 | }, 63 | "foreignKeys": { 64 | "type": "array", 65 | "items": { 66 | "type": "object", 67 | "required": [ "fields", "reference" ], 68 | "properties": { 69 | "fields": { "oneOf": [ { "type": "string" }, { "type": "array" } ] }, 70 | "reference": { 71 | "type": "object", 72 | "required": [ "resource", "fields" ], 73 | "properties": { 74 | "resource": { "type": "string" }, 75 | "fields": { "oneOf": [ { "type": "string" }, { "type": "array" } ] } 76 | } 77 | } 78 | } 79 | } 80 | } 81 | }, 82 | "required": [ "fields" ] 83 | } 84 | -------------------------------------------------------------------------------- /registry.csv: -------------------------------------------------------------------------------- 1 | id,title,schema,schema_path,specification 2 | base,Data Package,http://schemas.datapackages.org/data-package.json,data-package.json,http://dataprotocols.org/data-packages 3 | tabular,Tabular Data Package,http://schemas.datapackages.org/tabular-data-package.json,tabular-data-package.json,http://dataprotocols.org/tabular-data-package/ 4 | fiscal,Fiscal Data Package,http://schemas.datapackages.org/fiscal-data-package.json,fiscal-data-package.json,http://fiscal.dataprotocols.org/spec/ 5 | -------------------------------------------------------------------------------- /registry.json: -------------------------------------------------------------------------------- 1 | { 2 | "datapackage": { 3 | "title": "Data Package", 4 | "schema": "https://schemas.frictionlessdata.io/data-package.json", 5 | "schema_path": "data-package.json", 6 | "specification": "https://specs.frictionlessdata.io/data-package/" 7 | }, 8 | "tabular-datapackage": { 9 | "title": "Tabular Data Package", 10 | "schema": "https://schemas.frictionlessdata.io/tabular-data-package.json", 11 | "schema_path": "tabular-data-package.json", 12 | "specification": "https://specs.frictionlessdata.io/tabular-data-package/" 13 | }, 14 | "fiscal-datapackage": { 15 | "title": "Fiscal Data Package", 16 | "schema": "https://schemas.frictionlessdata.io/fiscal-data-package.json", 17 | "schema_path": "fiscal-data-package.json", 18 | "specification": "https://specs.frictionlessdata.io/fiscal-data-package/" 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /tabular-data-package.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "title": "Tabular Data Package", 4 | "description": "Tabular Data Package is a simple specification for data access and delivery of tabular data.", 5 | "type": "object", 6 | "required": [ "name", "resources" ], 7 | "properties": { 8 | "name": { 9 | "$ref": "definitions.json#/define/name", 10 | "propertyOrder": 10 11 | }, 12 | "title": { 13 | "$ref": "definitions.json#/define/title", 14 | "propertyOrder": 20 15 | }, 16 | "description": { 17 | "$ref": "definitions.json#/define/description", 18 | "format": "textarea", 19 | "propertyOrder": 30 20 | }, 21 | "homepage": { 22 | "$ref": "definitions.json#/define/homepage", 23 | "propertyOrder": 40 24 | }, 25 | "version": { 26 | "$ref": "definitions.json#/define/version", 27 | "propertyOrder": 50 28 | }, 29 | "license": { 30 | "$ref": "definitions.json#/define/license", 31 | "propertyOrder": 60 32 | }, 33 | "author": { 34 | "$ref": "definitions.json#/define/author", 35 | "propertyOrder": 70 36 | }, 37 | "contributors": { 38 | "$ref": "definitions.json#/define/contributors", 39 | "propertyOrder": 80, 40 | "options": { "hidden": true } 41 | }, 42 | "resources": { 43 | "title": "Resources", 44 | "description": "The data resources that this package describes.", 45 | "type": "array", 46 | "propertyOrder": 90, 47 | "minItems": 0, 48 | "items": { 49 | "type": "object", 50 | "properties": { 51 | "name": { 52 | "$ref": "definitions.json#/define/name", 53 | "propertyOrder": 10 54 | }, 55 | "title": { 56 | "$ref": "definitions.json#/define/title", 57 | "propertyOrder": 20 58 | }, 59 | "description": { 60 | "$ref": "definitions.json#/define/description", 61 | "propertyOrder": 30, 62 | "format": "textarea" 63 | }, 64 | "schema": { 65 | "$ref": "definitions.json#/define/schema", 66 | "propertyOrder": 40 67 | }, 68 | "url": { 69 | "$ref": "definitions.json#/define/url", 70 | "propertyOrder": 50 71 | }, 72 | "path": { 73 | "$ref": "definitions.json#/define/path", 74 | "propertyOrder": 60 75 | }, 76 | "data": { 77 | "$ref": "definitions.json#/define/data", 78 | "propertyOrder": 70 79 | }, 80 | "format": { 81 | "$ref": "definitions.json#/define/format", 82 | "propertyOrder": 80 83 | }, 84 | "mediatype": { 85 | "$ref": "definitions.json#/define/mediatype", 86 | "propertyOrder": 90 87 | }, 88 | "encoding": { 89 | "$ref": "definitions.json#/define/encoding", 90 | "propertyOrder": 100 91 | }, 92 | "bytes": { 93 | "$ref": "definitions.json#/define/bytes", 94 | "propertyOrder": 110, 95 | "options": { "hidden": true } 96 | }, 97 | "hash": { 98 | "$ref": "definitions.json#/define/hash", 99 | "propertyOrder": 120, 100 | "options": { "hidden": true } 101 | }, 102 | "dialect": { 103 | "$ref": "definitions.json#/define/dialect", 104 | "propertyOrder": 130, 105 | "options": { "hidden": true } 106 | }, 107 | "sources": { 108 | "$ref": "definitions.json#/define/sources", 109 | "propertyOrder": 140, 110 | "options": { "hidden": true } 111 | }, 112 | "license": { 113 | "$ref": "definitions.json#/define/license", 114 | "description": "The license under which the resource is published.", 115 | "propertyOrder": 150, 116 | "options": { "hidden": true } 117 | } 118 | }, 119 | "anyOf": [ 120 | { "title": "url required", "required": ["url"] }, 121 | { "title": "path required", "required": ["path"] }, 122 | { "title": "data required", "required": ["data"] } 123 | ], 124 | "required": [ "schema" ] 125 | } 126 | }, 127 | "keywords": { 128 | "$ref": "definitions.json#/define/keywords", 129 | "propertyOrder": 100 130 | }, 131 | "sources": { 132 | "$ref": "definitions.json#/define/sources", 133 | "propertyOrder": 110, 134 | "options": { "hidden": true } 135 | }, 136 | "image": { 137 | "$ref": "definitions.json#/define/image", 138 | "propertyOrder": 120, 139 | "options": { "hidden": true } 140 | }, 141 | "dataDependencies": { 142 | "$ref": "definitions.json#/define/dataDependencies", 143 | "propertyOrder": 140, 144 | "options": { "hidden": true } 145 | } 146 | } 147 | } 148 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frictionlessdata/schemas/e37c16854a13f8c1eb3494c224be49bb16b53429/tests/__init__.py -------------------------------------------------------------------------------- /tests/test_registry.py: -------------------------------------------------------------------------------- 1 | import os 2 | import csv 3 | import urllib 4 | import unittest 5 | 6 | BASE_PATH = os.path.abspath( 7 | os.path.join( 8 | os.path.dirname(__file__), 9 | '..' 10 | ) 11 | ) 12 | REGISTRY_PATH = os.path.join(BASE_PATH, 'registry.csv') 13 | 14 | 15 | class TestRegistry(unittest.TestCase): 16 | def test_registry_has_the_expected_headers(self): 17 | expected_headers = ( 18 | 'id', 19 | 'title', 20 | 'schema', 21 | 'schema_path', 22 | 'specification', 23 | ) 24 | 25 | with open(REGISTRY_PATH, 'r', newline='') as f: 26 | headers = next(csv.reader(f)) 27 | 28 | self.assertEqual(sorted(headers), sorted(expected_headers)) 29 | 30 | def test_registry_schemas_have_the_required_attributes(self): 31 | required_attributes = ( 32 | 'id', 33 | 'title', 34 | 'schema', 35 | 'schema_path', 36 | 'specification', 37 | ) 38 | 39 | with open(REGISTRY_PATH, 'r', newline='') as f: 40 | registry = csv.DictReader(f) 41 | msg = "Schema '{0}' doesn't define required attribute '{1}'" 42 | 43 | for schema in registry: 44 | for key, value in schema.items(): 45 | if key in required_attributes: 46 | assert value != '', msg.format(schema['id'], key) 47 | 48 | def test_registry_schemas_have_unique_ids(self): 49 | with open(REGISTRY_PATH, 'r', newline='') as f: 50 | registry = csv.DictReader(f) 51 | ids = [schema['id'] for schema in registry] 52 | 53 | assert len(ids) == len(set(ids)), "The schemas IDs aren't unique" 54 | 55 | def test_schema_paths_exist_and_are_files(self): 56 | with open(REGISTRY_PATH, 'r', newline='') as f: 57 | registry = csv.DictReader(f) 58 | 59 | for entry in registry: 60 | schema_path = entry['schema_path'] 61 | msg = "schema_path '{0}' of schema '{1}' isn't a file" 62 | msg = msg.format(schema_path, entry['id']) 63 | path = os.path.join(BASE_PATH, schema_path) 64 | assert os.path.isfile(path), msg 65 | 66 | def test_schema_urls_exist(self): 67 | is_successful = lambda req: req.status >= 200 and req.status < 400 68 | is_redirect = lambda req: req.status >= 300 and req.status < 400 69 | 70 | with open(REGISTRY_PATH, 'r', newline='') as f: 71 | registry = csv.DictReader(f) 72 | 73 | for entry in registry: 74 | try: 75 | url = entry['schema'] 76 | res = self._make_head_request(url) 77 | msg = "Error fetching schema_url '{0}' of schema '{1}'" 78 | msg = msg.format(url, entry['id']) 79 | assert (is_successful(res) or is_redirect(res)), msg 80 | except urllib.error.URLError as e: 81 | raise Exception(msg) from e 82 | 83 | def test_specification_urls_exist(self): 84 | is_successful = lambda req: req.status >= 200 and req.status < 400 85 | is_redirect = lambda req: req.status >= 300 and req.status < 400 86 | 87 | with open(REGISTRY_PATH, 'r', newline='') as f: 88 | registry = csv.DictReader(f) 89 | 90 | for entry in registry: 91 | try: 92 | url = entry['schema'] 93 | res = self._make_head_request(url) 94 | msg = "Error fetching specification '{0}' of schema '{1}'" 95 | msg = msg.format(url, entry['id']) 96 | assert (is_successful(res) or is_redirect(res)), msg 97 | except urllib.error.URLError as e: 98 | raise Exception(msg) from e 99 | 100 | def _make_head_request(self, url): 101 | req = urllib.request.Request(url, method='HEAD') 102 | return urllib.request.urlopen(req) 103 | -------------------------------------------------------------------------------- /tests/test_registry_json.py: -------------------------------------------------------------------------------- 1 | import io 2 | import os 3 | import json 4 | import unittest 5 | import requests 6 | 7 | BASE_PATH = os.path.abspath( 8 | os.path.join( 9 | os.path.dirname(__file__), 10 | '..' 11 | ) 12 | ) 13 | REGISTRY_PATH = os.path.join(BASE_PATH, 'registry.json') 14 | 15 | 16 | class TestRegistryJson(unittest.TestCase): 17 | def test_registry_has_the_expected_headers(self): 18 | expected_headers = ( 19 | 'title', 20 | 'schema', 21 | 'schema_path', 22 | 'specification', 23 | ) 24 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 25 | for id, item in registry.items(): 26 | self.assertEqual(sorted(item.keys()), sorted(expected_headers)) 27 | 28 | def test_registry_schemas_have_the_required_attributes(self): 29 | required_attributes = ( 30 | 'title', 31 | 'schema', 32 | 'schema_path', 33 | 'specification', 34 | ) 35 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 36 | for id, item in registry.items(): 37 | for attr in required_attributes: 38 | self.assertTrue(attr in item) 39 | 40 | def test_registry_schemas_have_unique_ids(self): 41 | ids = [] 42 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 43 | for id, item in registry.items(): 44 | self.assertNotIn(id, ids) 45 | ids.append(id) 46 | 47 | def test_schema_paths_exist_and_are_files(self): 48 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 49 | for id, item in registry.items(): 50 | schema_path = item['schema_path'] 51 | msg = "schema_path '{0}' of schema '{1}' isn't a file" 52 | msg = msg.format(schema_path, id) 53 | path = os.path.join(BASE_PATH, schema_path) 54 | assert os.path.isfile(path), msg 55 | 56 | def test_schema_urls_exist(self): 57 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 58 | for id, item in registry.items(): 59 | url = item['schema'] 60 | res = requests.head(url) 61 | msg = "Error fetching schema_url '{0}' of schema '{1}'" 62 | msg = msg.format(url, id) 63 | assert res.ok, msg 64 | 65 | def test_specification_urls_exist(self): 66 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 67 | for id, item in registry.items(): 68 | url = item['schema'] 69 | res = requests.head(url) 70 | msg = "Error fetching specification '{0}' of schema '{1}'" 71 | msg = msg.format(url, id) 72 | assert res.ok, msg 73 | 74 | def test_main_schemas_present(self): 75 | registry = json.load(io.open(REGISTRY_PATH, encoding='utf-8')) 76 | ids = list(registry.keys()) 77 | self.assertIn('datapackage', ids) 78 | self.assertIn('tabular-datapackage', ids) 79 | self.assertIn('fiscal-datapackage', ids) 80 | -------------------------------------------------------------------------------- /tests/test_schemas.py: -------------------------------------------------------------------------------- 1 | import os 2 | import glob 3 | import json 4 | import unittest 5 | import jsonschema 6 | 7 | BASE_PATH = os.path.abspath( 8 | os.path.join( 9 | os.path.dirname(__file__), 10 | '..' 11 | ) 12 | ) 13 | 14 | 15 | class TestSchemas(unittest.TestCase): 16 | def test_json_files_must_be_valid(self): 17 | json_glob = os.path.join(BASE_PATH, '*.json') 18 | json_paths = glob.glob(json_glob) 19 | 20 | for json_path in json_paths: 21 | try: 22 | with open(json_path, 'r') as f: 23 | json.load(f) 24 | except ValueError as e: 25 | msg = "File '{0}' isn\'t a valid JSON." 26 | raise ValueError(msg.format(json_path)) from e 27 | 28 | def test_json_files_must_be_valid_json_schemas(self): 29 | json_glob = os.path.join(BASE_PATH, '*.json') 30 | json_paths = glob.glob(json_glob) 31 | 32 | for json_path in json_paths: 33 | with open(json_path, 'r') as f: 34 | schema = json.load(f) 35 | try: 36 | validator_class = jsonschema.validators.validator_for(schema) 37 | validator = validator_class(schema) 38 | validator.check_schema(schema) 39 | except jsonschema.exceptions.SchemaError as e: 40 | msg = "File '{0}' isn\'t a valid JSON Schema." 41 | raise ValueError(msg.format(json_path)) from e 42 | --------------------------------------------------------------------------------