├── CNAME ├── CODE_OF_CONDUCT.md ├── README.md ├── compare.html ├── css └── styles.css ├── images ├── CustomShapeFileRealLogo.jpg ├── ban-shapefile.png ├── csv.png ├── geojsonicon_400x400.png ├── geopkg.png ├── gml-sticker-tag-open.jpg ├── icon.ico ├── kml.png ├── prefer-geopackage.png └── spatialite-logo.png ├── index.html └── testsuite ├── README.md ├── bin └── runtests.py ├── killtheshapefile └── __init__.py └── outputs └── stats.json /CNAME: -------------------------------------------------------------------------------- 1 | switchfromshapefile.org -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. 6 | 7 | ## Our Standards 8 | 9 | Examples of behavior that contributes to creating a positive environment include: 10 | 11 | * Using welcoming and inclusive language 12 | * Being respectful of differing viewpoints and experiences 13 | * Gracefully accepting constructive criticism 14 | * Focusing on what is best for the community 15 | * Showing empathy towards other community members 16 | 17 | Examples of unacceptable behavior by participants include: 18 | 19 | * The use of sexualized language or imagery and unwelcome sexual attention or advances 20 | * Trolling, insulting/derogatory comments, and personal or political attacks 21 | * Public or private harassment 22 | * Publishing others' private information, such as a physical or electronic address, without explicit permission 23 | * Other conduct which could reasonably be considered inappropriate in a professional setting 24 | 25 | ## Our Responsibilities 26 | 27 | Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. 28 | 29 | Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. 30 | 31 | ## Scope 32 | 33 | This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. 34 | 35 | ## Enforcement 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at info@opengeolabs.cz. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. 38 | 39 | Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. 40 | 41 | ## Attribution 42 | 43 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] 44 | 45 | [homepage]: http://contributor-covenant.org 46 | [version]: http://contributor-covenant.org/version/1/4/ 47 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # shapefilemustdie 2 | 3 | Repository with source code of the web page http://switchfromshapefile.org 4 | 5 | ## Contributing 6 | 7 | Contribute by sending standard pull request to `gh-pages` branch 8 | 9 | ## License 10 | 11 | Creative commons share alike 12 | -------------------------------------------------------------------------------- /compare.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Shapefile must die! The facts. 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 21 | 22 | 23 | 24 | Fork me on GitHub 25 |
26 |

Shapefile vs. GeoPackage vs. GML vs. GeoJSON

27 | 28 |

29 |

30 | 31 |

Metodic

32 | 58 |

59 | NOTE: Graphs are using only GeoPackage vs. Shapefile 60 | times. It was not possible to work with GeoJSON in size of 4GB. GML proofed 61 | to be working, but the times were generally very slow, it was meaning less 62 | to visualise them. 63 |

64 |

Repeat the measurment

65 | You can use the script, which is part of the GitHub 67 | repository. You can either use standard dataset or 68 | apply it on yours. 69 | 70 |

Results

71 |
72 |

73 | It was measured on an EC2 c5a.4xlarge instance (SSD, 32GB RAM, AMD® Epyc 7r32 16 cores) 74 |

75 | 76 |

File size [GB]

77 |
78 | 79 |

80 |

81 | 82 | 83 |

Read all data sequentially [ogrinfo ...]

84 |
85 | 86 |

87 | Test time, till all features are read sequentially using simple 88 | ogrinfo command. 89 | 

 90 |         ogrinfo gis.osm_buildings_a_free_1.shp gis.osm_buildings_a_free_1
 91 |
92 |

93 | 94 |

Getting file metadata [ogrinfo -so ...]

95 |
96 | 97 |

98 | Test time, till file metadata are displayed using 99 | ogrinfo command. 100 | 

101 |         ogrinfo -so \
102 |             gis.osm_buildings_a_free_1.shp gis.osm_buildings_a_free_1
103 |
104 |

105 | 106 |

Find feature using FID [ogrinfo -fid ...]

107 |
108 | 109 |

110 | Test time, till feature with given fid is 111 | identified using ogrinfo command. 112 | 

113 |         ogrinfo -fid 63347 \
114 |             gis.osm_buildings_a_free_1.shp gis.osm_buildings_a_free_1
115 |
116 |

117 | 118 |

The area of interest [ogrinfo -spat ...]

119 |
120 | 121 |

122 | Test time, till feature with given spat condition 123 | is identified using ogrinfo command. 124 | 

125 |         ogrinfo -spat 5.421254 52.129629 5.421254 52.129629 \
126 |             gis.osm_buildings_a_free_1.shp gis.osm_buildings_a_free_1
127 |
128 |

129 | 130 |

Find feature using where condition [ogrinfo -where ...]

131 |
132 | 133 |

134 | Test time, till feature with given where condition 135 | is identified using ogrinfo command. 136 | 

137 |         ogrinfo -where "name='OBS De Hobbit'" \
138 |             gis.osm_buildings_a_free_1.shp gis.osm_buildings_a_free_1
139 |
140 |

141 | 142 |
143 | 144 |

145 | Last modification: 2017-10-22
146 | Initially created by: Jachym Cepicky, 147 | OpenGeoLabs s.r.o.
148 | 149 | Creative Commons License 150 |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License
151 | Contribute: On GitHub 152 |

153 | 154 |
155 | 156 | 157 | 158 | 159 | 160 | 272 | 273 | 274 | -------------------------------------------------------------------------------- /css/styles.css: -------------------------------------------------------------------------------- 1 | .format-logo { 2 | float: right; 3 | width: 200px; 4 | } 5 | 6 | .short { 7 | } 8 | -------------------------------------------------------------------------------- /images/CustomShapeFileRealLogo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/CustomShapeFileRealLogo.jpg -------------------------------------------------------------------------------- /images/ban-shapefile.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/ban-shapefile.png -------------------------------------------------------------------------------- /images/csv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/csv.png -------------------------------------------------------------------------------- /images/geojsonicon_400x400.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/geojsonicon_400x400.png -------------------------------------------------------------------------------- /images/geopkg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/geopkg.png -------------------------------------------------------------------------------- /images/gml-sticker-tag-open.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/gml-sticker-tag-open.jpg -------------------------------------------------------------------------------- /images/icon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/icon.ico -------------------------------------------------------------------------------- /images/kml.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/kml.png -------------------------------------------------------------------------------- /images/prefer-geopackage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/prefer-geopackage.png -------------------------------------------------------------------------------- /images/spatialite-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenGeoLabs/shapefilemustdie/d1c40fffe8e2f4ec2586d716d5b20f1f33bff892/images/spatialite-logo.png -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Shapefile must die! 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 25 | 26 | 27 | 28 | Fork me on GitHub 29 |
30 |

Switch from Shapefile

31 | 32 |

33 | ESRI 34 | Shapefile is a file format for 36 | storing geospatial vector data. It has been around since the early 37 | 1990s and is still the most commonly used vector data exchange format. 38 |

39 |

While Shapefiles have enabled many successful activities over the years, they also 40 | have a number of limitations that complicate software development and reduce efficiency. 41 |

42 |

43 | We, members of the geospatial IT industry, believe that it is time to stop using Shapefiles 44 | as the primary vector data exchange format and to replace them with a format that takes 45 | advantage of the huge advances that have been made since Shapefile was introduced. 46 |

47 |

Read more:

48 | 53 | 54 |
55 | 56 |

The good side

57 |

58 | Shapefile does a lot of things right. 59 | Here are some reasons why Shapefile is so heavily used: 60 |

61 | 75 | 76 | 118 | 119 | 130 | 131 | 152 | 153 | 163 | 164 | 173 | 174 | 189 | 190 | 203 | 204 | 205 | 226 | 227 | 235 | 236 | 244 | 245 | 253 | 254 | 264 | 265 | 281 | 282 | 283 | 292 | 293 | 300 | 301 | 311 | 312 |
313 | 314 |

Alternatives

315 | 316 |

317 | What are the alternatives to the Shapefile format? To be honest, no alternative 318 | format has overthrown the Shapefile hegemony yet. Some formats nearly 319 | took over (KML, GML, GeoJSON), but their usage was limited to relatively 320 | narrow use cases only. 321 |

322 |

323 | Although there are more then 324 | 80 vector data formats in use out there, only a few can be 325 | considered as candidates for Shapefile replacement. Please note, that we do 326 | take only open (preferably community) formats into account. 327 |

328 | List of some Shapefile alternatives 329 | 338 |

https://t.co/6JZZRiP8q5 featuring two formats as @shapefile replacement. Which do you prefer? #switchfromshapefile #geojson vs. #geopackage

— Jachym Cepicky (@jachymc) October 5, 2017
339 | 340 | 341 |
342 | 343 | 395 | 396 | 424 | 425 | 461 |
462 | 463 |

OGC GML

464 | 465 |

466 | Another OGC Standard. 467 |

468 |

Features

469 | 475 |

Description

476 |

477 | GML was picked as the main distribution vector data format the European 478 | INSPIRE initiative. It's a very complex format, and its direct usage in 479 | GIS software is limited. Its main use is as a data exchange format that needs to be 480 | ingested into the user's system (e.g. into a database) to be fully usable. 481 |

482 |

483 | GML is currently often used for open data datasets, since it is 484 | technology-neutral and a supported OGC Standard. 485 |

486 |

A major downside to GML is that it is an insanely complex standard. 487 | Few software packages support the entire standard and support for individual parts of the standard varies widely.

488 |

489 | We believe that GML is a candidate for Shapefile 490 | replacement for data interchange in situations where data is too complex to be represented by GeoJSON. 491 | However, for the vast majority of datasets GML is overkill. 492 |

493 |
494 | 495 | 541 | 542 | 543 | 567 | 568 | 595 | 596 | 625 | 626 | 627 |
628 | 629 |

630 | Last modification: 2017-10-08
631 | Initially created by: Jachym Cepicky, 632 | OpenGeoLabs s.r.o.
633 | 634 | Creative Commons License 635 |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License
636 | Contribute: On GitHub 637 |

638 | 639 |
640 | 641 | 642 | 643 | 644 | 645 | 646 | -------------------------------------------------------------------------------- /testsuite/README.md: -------------------------------------------------------------------------------- 1 | # File format comparing tool 2 | 3 | ## Get help 4 | 5 | ``` 6 | python3 bin/runtests.py --help 7 | ``` 8 | 9 | ## Run the standard datast 10 | 11 | **NOTE:** You need at least 20GB of free space on your HDD! 12 | 13 | ``` 14 | python3 bin/runtests.py --tempdir /tmp/ 15 | ``` 16 | The script will download standard test dataset from GitHub, perform all the 17 | tests, write output into `outputs` directory 18 | 19 | ## Run on custom dataset 20 | 21 | The dataset should be zipped Shapefile. 22 | 23 | ``` 24 | python3 bin/runtests.py --tempdir /tmp/ --input myshp.zip \ 25 | --output /tmp/testoutput --spat 52.0 23.3 52.0, 23.3 --fid 10 \ 26 | --where "name='value'" --repeat 5 27 | ``` 28 | Result will be stored in the `/tmp/testoutput` directory 29 | -------------------------------------------------------------------------------- /testsuite/bin/runtests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import argparse 4 | from killtheshapefile import prepare_data 5 | from killtheshapefile import make_stats 6 | import logging 7 | import killtheshapefile 8 | import os 9 | 10 | 11 | def main(): 12 | logger = logging.getLogger("killtheshapefile") 13 | logger.setLevel(logging.INFO) 14 | 15 | parser = argparse.ArgumentParser() 16 | parser.add_argument("--tempdir", required=False, help="temporary directory", type=str) 17 | parser.add_argument("--output", default="./outputs/", help="output directory", type=str) 18 | parser.add_argument("--input", required=False, help="input zipped shapefile", type=argparse.FileType("r")) 19 | parser.add_argument("--where", required=False, help="-where config for ogrinfo", type=str) 20 | parser.add_argument("--fid", required=False, help="-fid config for ogrinfo", type=int) 21 | parser.add_argument("--spat", required=False, metavar="coord", 22 | help="-spat config for ogrinfo", type=float, nargs=4) 23 | parser.add_argument("--repeat", required=False, metavar="NUMBER", 24 | help="Repeat time measure X times", type=int, default=1) 25 | args = parser.parse_args() 26 | 27 | if args.output == "./outputs/": 28 | output = os.path.abspath(os.path.join( 29 | os.path.split(killtheshapefile.__file__)[0], 30 | args.output 31 | )) 32 | else: 33 | output = args.output 34 | 35 | if not os.path.isdir(output): 36 | os.mkdir(output) 37 | 38 | formats = prepare_data(args.tempdir, args.input.name if args.input else None) 39 | stats = make_stats(formats, output, args.repeat, args.where, args.fid, args.spat) 40 | 41 | if __name__ == "__main__": 42 | main() 43 | -------------------------------------------------------------------------------- /testsuite/killtheshapefile/__init__.py: -------------------------------------------------------------------------------- 1 | import requests 2 | import tempfile 3 | import subprocess 4 | import re 5 | import zipfile 6 | import os 7 | import logging 8 | from timeit import timeit 9 | import json 10 | 11 | DATA_URL="https://github.com/OpenGeoLabs/shapefilemustdie/releases/download/v1.0.0/test-data.zip" 12 | #DATA_URL="file:///home/jachym/tmp/x/test-data.zip" 13 | 14 | FORMATS=( 15 | ["ESRI Shapefile", "shp"], 16 | ["GPKG", "gpkg"], 17 | ["GML", "gml"], 18 | ["GeoJSON", "geojson"], 19 | #["FileGDB", "gdb"], 20 | ["FlatGeobuf", "fgb"] 21 | ) 22 | 23 | TEMP_DIR = None 24 | 25 | def create_tempdir(parent=None): 26 | 27 | global TEMP_DIR 28 | tempdir = tempfile.mkdtemp(prefix="killtheshapefile", dir=parent) 29 | TEMP_DIR = tempdir 30 | logging.info("Created temporary file {}".format(TEMP_DIR)) 31 | return tempdir 32 | 33 | def download_data(tempdir, inpt=None): 34 | 35 | if not inpt: 36 | r = requests.get(DATA_URL, stream=True) 37 | d = r.headers['content-disposition'] 38 | fname = re.findall("filename=(.+)", d)[0] 39 | localfile = os.path.join(tempdir, fname) 40 | logging.info("Downloading data") 41 | 42 | with open(localfile, 'wb') as f: 43 | for chunk in r.iter_content(chunk_size=1024): 44 | f.write(chunk) 45 | else: 46 | localfile=inpt 47 | 48 | zip_ref = zipfile.ZipFile(localfile, 'r') 49 | zip_ref.extractall(tempdir) 50 | zip_ref.close() 51 | files = os.listdir(tempdir) 52 | shps = [f for f in files if f.find("shp") > 0] 53 | return shps[0] 54 | 55 | def convert_to(inpt, frmt, tempdir): 56 | 57 | logging.info("Converting {} to {}".format(inpt, frmt[0])) 58 | basename = os.path.basename(inpt) 59 | root_name = os.path.splitext(basename)[0] 60 | fullpath_in = os.path.join(tempdir, inpt) 61 | dest_filename = os.path.join(tempdir, "{}.{}".format(root_name, frmt[1])) 62 | 63 | cmdline = ["ogr2ogr"] 64 | if frmt[0] == "FlatGeobuf": 65 | cmdline.extend(["-nlt", "GEOMETRY"]) 66 | cmdline.extend(["-f", frmt[0], dest_filename, fullpath_in]) 67 | subprocess.run(cmdline) 68 | 69 | return [frmt[0], frmt[1], dest_filename] 70 | 71 | 72 | def makeformats(tempdir, shp): 73 | 74 | formats = [] 75 | for frmt in FORMATS: 76 | if frmt[0] != FORMATS[0]: 77 | result = convert_to(shp, frmt, tempdir) 78 | formats.append(result) 79 | return formats 80 | 81 | def get_size(formats): 82 | 83 | size = {} 84 | for frmt in formats: 85 | if frmt[0] == "ESRI Shapefile": 86 | name, ext = os.path.splitext(frmt[-1]) 87 | size[frmt[0]] = os.stat(frmt[-1]).st_size +\ 88 | os.stat(name + ".dbf").st_size + \ 89 | os.stat(name + ".shx").st_size 90 | else: 91 | size[frmt[0]] = os.stat(frmt[-1]).st_size 92 | 93 | return size 94 | 95 | def get_metadata_time(formats, repeat=1): 96 | 97 | duration = {} 98 | for frmt in formats: 99 | fn = frmt[-1] 100 | frmt_name = frmt[0] 101 | 102 | if frmt_name == "GeoJSON": 103 | duration[frmt_name] = None 104 | continue 105 | 106 | name, ext = os.path.splitext(fn) 107 | layer = os.path.basename(name) 108 | cmdline = ["ogrinfo", "-qq"] 109 | if frmt_name == "FlatGeobuf": 110 | cmdline.extend(["-oo", "VERIFY_BUFFERS=NO"]) 111 | cmdline.extend(["-so", fn, layer]) 112 | time = timeit( 113 | stmt = "subprocess.run(" + str(cmdline)+ ", stdout=subprocess.PIPE)", 114 | setup = "import subprocess", number = repeat) 115 | duration[frmt_name] = time 116 | 117 | return duration 118 | 119 | def get_fid_time(formats, fid = None, repeat=1): 120 | 121 | if fid is None: 122 | fid = str(63347) 123 | else: 124 | fid = str(fid) 125 | 126 | duration = {} 127 | for frmt in formats: 128 | fn = frmt[-1] 129 | frmt_name = frmt[0] 130 | 131 | if frmt_name == "GeoJSON": 132 | duration[frmt_name] = None 133 | continue 134 | 135 | name, ext = os.path.splitext(fn) 136 | layer = os.path.basename(name) 137 | cmdline = ["ogrinfo", "-qq"] 138 | if frmt_name == "FlatGeobuf": 139 | cmdline.extend(["-oo", "VERIFY_BUFFERS=NO"]) 140 | cmdline.extend(["-fid", fid, fn, layer]) 141 | time = timeit( 142 | stmt = "subprocess.run(" + str(cmdline)+ ", stdout=subprocess.PIPE)", 143 | setup = "import subprocess", number = repeat) 144 | duration[frmt_name] = time 145 | 146 | return duration 147 | 148 | def get_where_time(formats, where=None, repeat=1): 149 | 150 | if not where: 151 | where = "name='OBS De Hobbit'" 152 | 153 | duration = {} 154 | for frmt in formats: 155 | fn = frmt[-1] 156 | frmt_name = frmt[0] 157 | 158 | if frmt_name == "GeoJSON": 159 | duration[frmt_name] = None 160 | continue 161 | 162 | name, ext = os.path.splitext(fn) 163 | layer = os.path.basename(name) 164 | cmdline = ["ogrinfo", "-qq"] 165 | if frmt_name == "FlatGeobuf": 166 | cmdline.extend(["-oo", "VERIFY_BUFFERS=NO"]) 167 | cmdline.extend(["-where", where, fn, layer]) 168 | time = timeit( 169 | stmt = "subprocess.run(" + str(cmdline)+ ", stdout=subprocess.PIPE)", 170 | setup = "import subprocess", number = repeat) 171 | duration[frmt_name] = time 172 | 173 | return duration 174 | 175 | def get_spat_time(formats, spat=None, repeat=1): 176 | 177 | if not spat: 178 | spat = ["5.421254", "52.129629", "5.421254", "52.129629"] 179 | else: 180 | spat = [str(c) for c in spat] 181 | 182 | duration = {} 183 | for frmt in formats: 184 | fn = frmt[-1] 185 | frmt_name = frmt[0] 186 | 187 | if frmt_name == "GeoJSON": 188 | duration[frmt_name] = None 189 | continue 190 | 191 | name, ext = os.path.splitext(fn) 192 | layer = os.path.basename(name) 193 | cmdline = ["ogrinfo", "-qq"] 194 | if frmt_name == "FlatGeobuf": 195 | cmdline.extend(["-oo", "VERIFY_BUFFERS=NO"]) 196 | cmdline.extend(["-spat"] + spat + [fn, layer]) 197 | time = timeit( 198 | stmt = "subprocess.run(" + str(cmdline) + ", stdout=subprocess.PIPE)", 199 | setup = "import subprocess", number = repeat) 200 | duration[frmt_name] = time 201 | 202 | return duration 203 | 204 | def get_seq_time(formats, repeat=1): 205 | 206 | duration = {} 207 | for frmt in formats: 208 | fn = frmt[-1] 209 | frmt_name = frmt[0] 210 | 211 | if frmt_name == "GeoJSON": 212 | duration[frmt_name] = None 213 | continue 214 | 215 | name, ext = os.path.splitext(fn) 216 | layer = os.path.basename(name) 217 | cmdline = ["ogrinfo", "-qq"] 218 | if frmt_name == "FlatGeobuf": 219 | cmdline.extend(["-oo", "VERIFY_BUFFERS=NO"]) 220 | cmdline.extend([fn, layer]) 221 | time = timeit( 222 | stmt = "subprocess.run(" + str(cmdline) + ", stdout=subprocess.PIPE)", 223 | setup = "import subprocess", number = repeat) 224 | duration[frmt_name] = time 225 | 226 | return duration 227 | 228 | def prepare_data(tempdir=None, inpt=None): 229 | tempdir = create_tempdir(tempdir) 230 | shp = download_data(tempdir, inpt) 231 | formats = makeformats(tempdir, shp) 232 | return formats 233 | 234 | def make_stats(formats, output, repeat=1, where=None, fid=None, spat=None): 235 | 236 | stats = {} 237 | stats["size"] = get_size(formats) 238 | stats["metadata"] = get_metadata_time(formats, repeat) 239 | stats["fid"] = get_fid_time(formats, fid, repeat) 240 | stats["where"] = get_where_time(formats, where, repeat) 241 | stats["spat"] = get_spat_time(formats, spat, repeat) 242 | stats["seq"] = get_seq_time(formats, repeat) 243 | with open(os.path.join(output, "stats.json"), "w") as out: 244 | out.write(json.dumps(stats, indent=2)) 245 | -------------------------------------------------------------------------------- /testsuite/outputs/stats.json: -------------------------------------------------------------------------------- 1 | { 2 | "spat": { 3 | "GPKG": 0.048057263978989795, 4 | "GML": 343.3025719569996, 5 | "ESRI Shapefile": 16.999171105999267, 6 | "GeoJSON": null 7 | }, 8 | "seq": { 9 | "GPKG": 241.53338077000808, 10 | "GML": 604.6719867950014, 11 | "ESRI Shapefile": 237.03450499402243, 12 | "GeoJSON": null 13 | }, 14 | "fid": { 15 | "GPKG": 2.938916106009856, 16 | "GML": 241.60755829000846, 17 | "ESRI Shapefile": 0.34073773300042376, 18 | "GeoJSON": null 19 | }, 20 | "where": { 21 | "GPKG": 7.732582016004017, 22 | "GML": 377.7504133799812, 23 | "ESRI Shapefile": 40.549172239989275, 24 | "GeoJSON": null 25 | }, 26 | "size": { 27 | "GPKG": 2830307328, 28 | "GML": 6509654779, 29 | "ESRI Shapefile": 3479576609, 30 | "GeoJSON": 4024587722 31 | }, 32 | "metadata": { 33 | "GPKG": 3.129738196992548, 34 | "GML": 258.6620598669979, 35 | "ESRI Shapefile": 0.37446932701277547, 36 | "GeoJSON": null 37 | } 38 | } --------------------------------------------------------------------------------