└── README.md /README.md: -------------------------------------------------------------------------------- 1 | Cheat sheet for GDAL/OGR command-line geodata tools 2 | 3 | Vector operations 4 | --- 5 | 6 | __Get vector information__ 7 | 8 | ogrinfo -so input.shp layer-name 9 | 10 | Or, for all layers 11 | 12 | ogrinfo -al -so input.shp 13 | 14 | __Print vector extent__ 15 | 16 | ogrinfo input.shp layer-name | grep Extent 17 | 18 | __Get information about a specific feature__ 19 | 20 | ogrinfo input.shp -fid 12 -geom=NO 21 | 22 | The `geom=NO` flag prevents this command from returning a well-known text (WKT) representation of the feature's geometry. 23 | 24 | __Run an SQLite command on a layer__ 25 | 26 | ogrinfo content -dialect sqlite -sql "SELECT COUNT(*) FROM input WHERE type = 'pond'" 27 | 28 | This will count "pond" features in `content/input.shp`. 29 | 30 | __List vector drivers__ 31 | 32 | ogr2ogr --formats 33 | 34 | __Convert between vector formats__ 35 | 36 | ogr2ogr -f "GeoJSON" output.json input.shp 37 | 38 | __Print count of features with attributes matching a given pattern__ 39 | 40 | ogrinfo input.shp layer-name | grep "Search Pattern" | sort | uniq -c 41 | 42 | __Read from a zip file__ 43 | 44 | This assumes that archive.zip is in the current directory. This example just extracts the file, but any ogr2ogr operation should work. It's also possible to write to existing zip files. 45 | 46 | ogr2ogr -f 'GeoJSON' dest.geojson /vsizip/archive.zip/zipped_dir/in.geojson 47 | 48 | __Clip vectors by bounding box__ 49 | 50 | ogr2ogr -f "ESRI Shapefile" output.shp input.shp -clipsrc 51 | 52 | __Clip one vector by another__ 53 | 54 | ogr2ogr -clipsrc clipping_polygon.shp output.shp input.shp 55 | 56 | __Reproject vector:__ 57 | 58 | ogr2ogr output.shp -t_srs "EPSG:4326" input.shp 59 | 60 | __Add an index to a shapefile__ 61 | 62 | Add an index on an attribute: 63 | 64 | ogrinfo example.shp -sql "CREATE INDEX ON example USING fieldname" 65 | 66 | Add a spatial index: 67 | 68 | ogrinfo example.shp -sql "CREATE SPATIAL INDEX ON example" 69 | 70 | __Merge features in a vector file by attribute ("dissolve")__ 71 | 72 | ogr2ogr -f "ESRI Shapefile" dissolved.shp input.shp -dialect sqlite \ 73 | -sql "select ST_union(Geometry) Geometry, common_attribute FROM input GROUP BY common_attribute" 74 | 75 | __Merge features ("dissolve") using a buffer to avoid slivers__ 76 | 77 | ogr2ogr -f "ESRI Shapefile" dissolved.shp input.shp -dialect sqlite \ 78 | -sql "select ST_union(ST_buffer(Geometry,0.001)),common_attribute from input GROUP BY common_attribute" 79 | 80 | __Merge vector files:__ 81 | 82 | ogr2ogr merged.shp input1.shp 83 | ogr2ogr -update -append merged.shp input2.shp -nln merged 84 | 85 | __Extract from a vector file based on query__ 86 | 87 | To extract features with STATENAME 'New York','New Hampshire', etc. from states.shp 88 | 89 | ogr2ogr states_subset.shp states.shp -where 'STATENAME like "New%"' 90 | 91 | To extract type 'pond' from water.shp 92 | 93 | ogr2ogr ponds.shp water.shp -where "type = 'pond'" 94 | 95 | __Subset & filter all shapefiles in a directory__ 96 | 97 | Assumes that filename and name of layer of interest are the same... 98 | 99 | basename -s.shp *.shp | xargs -n1 -I % ogr2ogr %-subset.shp %.shp \ 100 | -sql "SELECT field1, field2 FROM '%' WHERE field1 = 'value-of-interest'" 101 | 102 | __Extract data from a PostGis database to a GeoJSON file__ 103 | 104 | ogr2ogr -f "GeoJSON" file.geojson PG:"host=localhost dbname=database user=user password=password" \ 105 | -sql "SELECT * from table_name" 106 | 107 | __Extract data from an ESRI REST API__ 108 | 109 | Services that use ESRI maps are sometimes powered by a REST server that can provide data in OGR can consume. Finding the correct end point can be tricky and may take some false starts. 110 | 111 | ogr2ogr -f GeoJSON output.geojson "http:/example.com/arcgis/rest/services/SERVICE/LAYER/MapServer/0/query?f=json&returnGeometry=true&etc=..." OGRGeoJSON 112 | 113 | __Get the difference between two vector files__ 114 | 115 | Given two files that both have an id field, this will produce a vector file with the part of `file1.shp` that doesn't intersect with `file2.shp`: 116 | 117 | ogr2ogr diff.shp inputfolder -dialect sqlite \ 118 | -sql "SELECT ST_Difference(a.Geometry, b.Geometry) AS Geometry, a.id \ 119 | FROM file1 a LEFT JOIN file2 b USING (id) WHERE a.Geometry != b.Geometry" 120 | 121 | This assumes that `file2.shp` and `file2.shp` are both in `inputfolder`. 122 | 123 | __Spatial join:__ 124 | 125 | A spatial join transfers properties from one vector layer to another based on a [spatial relationship](http://postgis.net/docs/manual-2.0/reference.html#Spatial_Relationships_Measurements) between the features. Fields from the join layer can be [aggregated](https://www.sqlite.org/lang_aggfunc.html) in the output. 126 | 127 | Given a set of points (`natural/trees.shp`) and a set of polygons (`natural/parks.shp`), create a polygon layer with the geometries from parks.shp and summaries of some columns in trees.shp: 128 | 129 | ogr2ogr -f 'ESRI Shapefile' output.shp natural -dialect sqlite \ 130 | -sql "SELECT p.Geometry, p.id id, SUM(t.field1) field1_sum, AVG(t.field2) field2_avg 131 | FROM parks p, trees t WHERE ST_Contains(p.Geometry, t.Geometry) GROUP BY p.id" 132 | 133 | Note that features that from parks.shp that don't overlap with trees.shp won't be in the new file. 134 | 135 | __Voronoi diagram__ 136 | 137 | The PostGIS [`ST_VoronojDiagram`](http://www.gaia-gis.it/gaia-sins/spatialite-sql-4.2.0.html#p14c) function is useful for creating voronoi geometries. However, it only creates one multipolygon, which is less useful if one wants to use the attributes of the point geometry. The following creates the Voronoi diagram, breaks up the multipolygob. Then, after adding spatial indices, a spatial join adds the attribute data from the input file to the voronoi polygons: 138 | 139 | ogr2ogr tmp.shp input.shp -f 'ESRI Shapefile' -lco SPATIAL_INDEX=YES \ 140 | -dialect sqlite -sql "WITH RECURSIVE cnt(x) AS ( \ 141 | SELECT 1 UNION ALL SELECT x+1 FROM cnt \ 142 | LIMIT (SELECT NumGeometries(Geometry) FROM geom) \ 143 | ), geom AS ( \ 144 | SELECT ST_VoronojDiagram(ST_Union(Geometry)) Geometry FROM input \ 145 | ) SELECT GeometryN(geom.Geometry, x) Geometry FROM cnt, geom" 146 | ogrinfo input.shp -sql "CREATE SPATIAL INDEX ON input" 147 | ogr2ogr voronoi.shp . -f 'ESRI Shapefile' -dialect sqlite -sql "SELECT v.Geometry, a.* \ 148 | FROM tmp v LEFT JOIN input a ON ST_Contains(v.Geometry, a.Geometry)" 149 | 150 | 151 | Raster operations 152 | --- 153 | __Get raster information__ 154 | 155 | gdalinfo input.tif 156 | 157 | __List raster drivers__ 158 | 159 | gdal_translate --formats 160 | 161 | __Force creation of world file (requires libgeotiff)__ 162 | 163 | listgeo -tfw mappy.tif 164 | 165 | __Report PROJ.4 projection info, including bounding box (requires libgeotiff)__ 166 | 167 | listgeo -proj4 mappy.tif 168 | 169 | __Convert between raster formats__ 170 | 171 | gdal_translate -of "GTiff" input.grd output.tif 172 | 173 | __Convert 16-bit bands (Int16 or UInt16) to Byte type__ 174 | (Useful for Landsat 8 imagery...) 175 | 176 | gdal_translate -of "GTiff" -co "COMPRESS=LZW" -scale 0 65535 0 255 -ot Byte input_uint16.tif output_byte.tif 177 | 178 | You can change '0' and '65535' to your image's actual min/max values to preserve more color variation or to apply the scaling to other band types - find that number with: 179 | 180 | gdalinfo -mm input.tif | grep Min/Max 181 | 182 | __Convert a directory of raster files of the same format to another raster format__ 183 | 184 | basename -s.img *.img | xargs -n1 -I % gdal_translate -of "GTiff" %.img %.tif 185 | 186 | __Reproject raster:__ 187 | 188 | gdalwarp -t_srs "EPSG:102003" input.tif output.tif 189 | 190 | Be sure to add _-r bilinear_ if reprojecting elevation data to prevent funky banding artifacts. 191 | 192 | __Georeference an unprojected image with known bounding coordinates:__ 193 | 194 | gdal_translate -of GTiff -a_ullr \ 195 | -a_srs EPSG:4269 input.png output.tif 196 | 197 | __Clip raster by bounding box__ 198 | 199 | gdalwarp -te input.tif clipped_output.tif 200 | 201 | __Clip raster to SHP / NoData for pixels beyond polygon boundary__ 202 | 203 | gdalwarp -dstnodata -cutline input_polygon.shp input.tif clipped_output.tif 204 | 205 | __Crop raster dimensions to vector bounding box__ 206 | 207 | gdalwarp -cutline cropper.shp -crop_to_cutline input.tif cropped_output.tif 208 | 209 | __Merge rasters__ 210 | 211 | gdal_merge.py -o merged.tif input1.tif input2.tif 212 | 213 | Alternatively, 214 | 215 | gdalwarp input1.tif input2.tif merged.tif 216 | 217 | Or, to preserve nodata values: 218 | 219 | gdalwarp input1.tif input2.tif merged.tif -srcnodata -dstnodata 220 | 221 | __Stack grayscale bands into a georeferenced RGB__ 222 | 223 | Where LC81690372014137LGN00 is a Landsat 8 ID and B4, B3 and B2 correspond to R,G,B bands respectively: 224 | 225 | gdal_merge.py -co "PHOTOMETRIC=RGB" -separate LC81690372014137LGN00_B{4,3,2}.tif -o LC81690372014137LGN00_rgb.tif 226 | 227 | __Fix an RGB TIF whose bands don't know they're RGB__ 228 | 229 | gdal_merge.py -co "PHOTOMETRIC=RGB" input.tif -o output_rgb.tif 230 | 231 | __Export a raster for Google Earth__ 232 | 233 | gdal_translate -of KMLSUPEROVERLAY input.tif output.kmz -co FORMAT=JPEG 234 | 235 | __Raster calculation (map algebra)__ 236 | 237 | Average two rasters: 238 | 239 | gdal_calc.py -A input1.tif -B input2.tif --outfile=output.tif --calc="(A/2+B/2)" 240 | 241 | Add two rasters: 242 | 243 | gdal_calc.py -A input1.tif -B input2.tif --outfile=output.tif --calc="A+B" 244 | 245 | etc. 246 | 247 | __Create a hillshade from a DEM__ 248 | 249 | gdaldem hillshade -of PNG input.tif hillshade.png 250 | 251 | Change light direction: 252 | 253 | gdaldem hillshade -of PNG -az 135 input.tif hillshade_az135.png 254 | 255 | Use correct vertical scaling in meters if input is projected in degrees 256 | 257 | gdaldem hillshade -s 111120 -of PNG input_WGS1984.tif hillshade.png 258 | 259 | __Apply color ramp to a DEM__ 260 | First, create a color-ramp.txt file: 261 | _(Height, Red, Green, Blue)_ 262 | 263 | 0 110 220 110 264 | 900 240 250 160 265 | 1300 230 220 170 266 | 1900 220 220 220 267 | 2500 250 250 250 268 | 269 | Then apply those colors to a DEM: 270 | 271 | gdaldem color-relief input.tif color_ramp.txt color-relief.tif 272 | 273 | __Create slope-shading from a DEM__ 274 | First, make a slope raster from DEM: 275 | 276 | gdaldem slope input.tif slope.tif 277 | 278 | Second, create a color-slope.txt file: 279 | _(Slope angle, Red, Green, Blue)_ 280 | 281 | 0 255 255 255 282 | 90 0 0 0 283 | 284 | Finally, color the slope raster based on angles in color-slope.txt: 285 | 286 | gdaldem color-relief slope.tif color-slope.txt slopeshade.tif 287 | 288 | __Resample (resize) raster__ 289 | 290 | gdalwarp -ts -r cubic dem.tif resampled_dem.tif 291 | 292 | Entering 0 for either width or height guesses based on current dimensions. 293 | 294 | Alternatively, 295 | 296 | gdal_translate -outsize 10% 10% -r cubic dem.tif resampled_dem.tif 297 | 298 | For both of these, `-r cubic` specifies cubic interpolation: when resampling continuous data (like a DEM), the default nearest neighbor interpolation can result in "stair step" artifacts. 299 | 300 | __Burn vector into raster__ 301 | 302 | gdal_rasterize -b 1 -i -burn -32678 -l layername input.shp input.tif 303 | 304 | __Extract polygons from raster__ 305 | 306 | gdal_polygonize.py input.tif -f "GeoJSON" output.json 307 | 308 | __Create contours from DEM__ 309 | 310 | gdal_contour -a elev -i 50 input_dem.tif output_contours.shp 311 | 312 | __Get values for a specific location in a raster__ 313 | 314 | gdallocationinfo -xml -wgs84 input.tif 315 | 316 | __Convert GRIB band to .tif__ 317 | 318 | Assumes data for entire globe in WGS84. Be sure to specify band, or you may end up with a nonsense RGB image composed of the first three. 319 | 320 | gdal_translate input.grib -a_ullr -180 -90 180 90 -a_srs EPSG:4326 -b 1 output_band1.tif 321 | 322 | 323 | Other 324 | --- 325 | __Convert KML points to CSV (simple)__ 326 | 327 | ogr2ogr -f CSV output.csv input.kmz -lco GEOMETRY=AS_XY 328 | 329 | __Convert KML to CSV (WKT)__ 330 | First list layers in the KML file 331 | 332 | ogrinfo -so input.kml 333 | 334 | Convert the desired KML layer to CSV 335 | 336 | ogr2ogr -f CSV output.csv input.kml -sql "select *,OGR_GEOM_WKT from some_kml_layer" 337 | 338 | __CSV points to SHP__ 339 | 340 | Given `input.csv`: 341 | 342 | lon,lat,value 343 | -81,31,13 344 | -80,32,14 345 | -81,33,15 346 | 347 | Create a shapefile, using Spatialite functions to generate the point: 348 | 349 | ogr2ogr output.shp input.csv -dialect sqlite \ 350 | -sql "SELECT MakePoint(CAST(lon as REAL), CAST(lat as REAL), 4326) Geometry, * FROM input" 351 | 352 | Note the 4326, which refers to a spatial reference (in this case [`EPSG:4326`](http://epsg.io/4326)). Use the correct code for your data. 353 | 354 | __MODIS operations__ 355 | 356 | First, download relevant .hdf tiles from the MODIS ftp site: ; use the [MODIS sinusoidal grid](http://www.geohealth.ou.edu/modis_v5/modis.shtml) for reference. 357 | 358 | List MODIS Subdatasets in a given HDF (conf. the [MODIS products table](https://lpdaac.usgs.gov/products/modis_products_table/)) 359 | 360 | gdalinfo longFileName.hdf | grep SUBDATASET 361 | 362 | Make TIFs from each file in list; replace 'MOD12Q1:Land_Cover_Type_1' with desired Subdataset name 363 | 364 | mkdir output 365 | find . '*.hdf' -exec gdalwarp -of GTiff 'HDF4_EOS:EOS_GRID:"{}":MOD12Q1:Land_Cover_Type_1' output/{}.tif \; 366 | 367 | Merge all .tifs in output directory into single file 368 | 369 | gdal_merge.py -o output/Merged_Landcover.tif output/*.tif 370 | 371 | __BASH functions__ 372 | _Size Functions_ 373 | This size function echos the pixel dimensions of a given file in the format expected by gdalwarp. 374 | 375 | function gdal_size() { 376 | SIZE=$(gdalinfo $1 |\ 377 | grep 'Size is ' |\ 378 | cut -d\ -f3-4 |\ 379 | sed 's/,//g') 380 | echo -n "$SIZE" 381 | } 382 | 383 | This can be used to easily resample one raster to the dimensions of another: 384 | 385 | gdalwarp -ts $(gdal_size bigraster.tif) -r cubicspline smallraster.tif resampled_smallraster.tif 386 | 387 | _Extent Functions_ 388 | These extent functions echo the extent of the given file in the order/format expected by gdal_translate -projwin. 389 | (Originally from [Linfiniti](http://linfiniti.com/2009/09/clipping-rasters-with-gdal-using-polygons/)). 390 | 391 | function gdal_extent() { 392 | if [ -z "$1" ]; then 393 | echo "Missing arguments. Syntax:" 394 | echo " gdal_extent " 395 | return 396 | fi 397 | EXTENT=$(gdalinfo $1 |\ 398 | grep "Upper Left\|Lower Right" |\ 399 | sed "s/Upper Left //g;s/Lower Right //g;s/).*//g" |\ 400 | tr "\n" " " |\ 401 | sed 's/ *$//g' |\ 402 | tr -d "[(,]") 403 | echo -n "$EXTENT" 404 | } 405 | 406 | function ogr_extent() { 407 | if [ -z "$1" ]; then 408 | echo "Missing arguments. Syntax:" 409 | echo " ogr_extent " 410 | return 411 | fi 412 | EXTENT=$(ogrinfo -al -so $1 |\ 413 | grep Extent |\ 414 | sed 's/Extent: //g' |\ 415 | sed 's/(//g' |\ 416 | sed 's/)//g' |\ 417 | sed 's/ - /, /g') 418 | EXTENT=`echo $EXTENT | awk -F ',' '{print $1 " " $4 " " $3 " " $2}'` 419 | echo -n "$EXTENT" 420 | } 421 | 422 | function ogr_layer_extent() { 423 | if [ -z "$2" ]; then 424 | echo "Missing arguments. Syntax:" 425 | echo " ogr_extent " 426 | return 427 | fi 428 | EXTENT=$(ogrinfo -so $1 $2 |\ 429 | grep Extent |\ 430 | sed 's/Extent: //g' |\ 431 | sed 's/(//g' |\ 432 | sed 's/)//g' |\ 433 | sed 's/ - /, /g') 434 | EXTENT=`echo $EXTENT | awk -F ',' '{print $1 " " $4 " " $3 " " $2}'` 435 | echo -n "$EXTENT" 436 | } 437 | 438 | Extents can be passed directly into a gdal_translate command like so: 439 | 440 | gdal_translate -projwin $(ogr_extent boundingbox.shp) input.tif clipped_output.tif 441 | 442 | or 443 | 444 | gdal_translate -projwin $(gdal_extent target_crop.tif) input.tif clipped_output.tif 445 | 446 | This can be a useful way to quickly crop one raster to the same extent as another. Add these to your ~/.bash_profile file for easy terminal access. 447 | 448 | 449 | Sources 450 | --- 451 | 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 468 | 469 | 470 | 471 | 472 | 473 | 474 | 475 | 476 | 477 | 478 | 479 | 480 | 481 | 482 | 483 | 484 | 485 | 486 | --------------------------------------------------------------------------------