└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # gdal/ogr cheatsheet 2 | 3 | ## GDAL 4 | 5 | #### Cloud Optimized GeoTif - [COG](https://www.cogeo.org/) 6 | 7 | gdal_translate in.tif out.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=LZW 8 | 9 | - gdaladdo should be run before to create internal overviews 10 | 11 | Source: https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF#HowtogenerateitwithGDAL 12 | 13 | alternatively use ```-of COG``` : 14 | 15 | gdal_translate world.tif world_webmerc_cog.tif -of COG -co TILING_SCHEME=GoogleMapsCompatible -co COMPRESS=JPEG 16 | 17 | Source: https://gdal.org/drivers/raster/cog.html 18 | 19 | *as of GDAL>=2.4 [WebP](https://gdal.org/drivers/raster/webp.html) seems to be the best compression available, 20 | although it seem QGIS 3.4 (GDAL 2.2.2, released 2017/09/15) does not read this type of compression yet 21 | 22 | -CO COMPRESS=WEBP 23 | 24 | #### Compress imagery 25 | 26 | as seen on http://blog.cleverelephant.ca/2015/02/geotiff-compression-for-dummies.html 27 | 28 | gdal_translate \ 29 | -co COMPRESS=JPEG \ 30 | -co PHOTOMETRIC=YCBCR \ 31 | -co TILED=YES \ 32 | -co BIGTIFF=YES \ 33 | 5255C.tif 5255C_JPEG_YCBCR.tif 34 | 35 | 36 | * If dealing with IR bands the recommended options are -co PHOTOMETRIC=RGB -co ALPHA=NO 37 | source: https://lists.osgeo.org/pipermail/gdal-dev/2011-April/028455.html 38 | With these 2 options you should get the 4th band defined as 'Undefined'. Otherwise you get a 4th band defined as Alpha. 39 | 40 | 41 | Create overviews 42 | 43 | gdaladdo \ 44 | --config COMPRESS_OVERVIEW JPEG \ 45 | --config PHOTOMETRIC_OVERVIEW YCBCR \ 46 | --config INTERLEAVE_OVERVIEW PIXEL \ 47 | -r average \ 48 | 5255C_JPEG_YCBCR.tif \ 49 | 2 4 8 16 50 | 51 | Compress a bunch of tifs... 52 | 53 | for FILE in *.tif \ 54 | do \ 55 | BASE=`basename $FILE .tif` \ 56 | NEWFILE=${BASE}_c.tif \ 57 | gdal_translate -b 1 -b 2 -b 3 -co COMPRESS=JPEG -co TILED=YES -co PHOTOMETRIC=YCBCR $FILE $NEWFILE \ 58 | done \ 59 | 60 | 61 | 62 | 63 | --- 64 | #### GDALWARP: 65 | 66 | *see below for using gdalwarp to merge tifs 67 | 68 | gdalwarp -s_srs EPSG:4326 -t_srs EPSG:27700 home_wgs84.bmp home_OSGB36.tif 69 | 70 | Crops image based on shapefile select polygon using -cwhere 71 | 72 | gdalwarp -cutline shpfile.shp -cwhere "fieldname = 'fieldvalue'" -crop_to_cutline inimage.tif outimage.tif 73 | 74 | Mask raster to cutline, use NO_GEOTRANSFORM for un-georeferenced images 75 | 76 | gdalwarp -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM -cutline cut_line.csv in.tif out.tif 77 | 78 | * Format of cut_line.csv 79 | id,WKT 80 | 1,"POLYGON((9756 9321, 9756 360, 816 360, 816 9321, 9756 9321))" 81 | 82 | * options: 83 | -dstalpha to create an alpha band, masking nodata pixels 84 | -cblend to feather eadges of imagery for a better seamless image 85 | -multi to multi-threaded processing 86 | 87 | 88 | Subset with -te 89 | 90 | gdalwarp -te -7.35 48.48 3.79 59.51 merged_DEM.tif subset_DEM.tif 91 | 92 | Change Resolution w/ -tr (target resolution) 93 | 94 | gdalwarp -tr 100.0 100.0 -r cubic -of GTiff USGS_13_n45w068_20230302_6350.tif ./warped/USGS_13_n45w068_20230302_6350_100x100.tif 95 | 96 | Mosaic 97 | 98 | gdalwarp --config GDAL_CACHEMAX 3000 -wm 3000 *.tif final_mosaic.tif 99 | 100 | * Note that it is usually a good idea to "optimise" the resulting image with gdal_translate. 101 | 102 | 103 | #### gdalbuildvrt 104 | 105 | gdalbuildvrt -input_file_list newfile output.vrt 106 | 107 | then create overviews 108 | 109 | gdaladdo -ro --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL --config BIGTIFF_OVERVIEW YES output.vrt 2 4 8 16 110 | 111 | * prepend ```/vsicurl/``` to your COG url in S3 to create VRT - https://www.cogeo.org/qgis-tutorial.html 112 | 113 | 114 | --- 115 | #### gdal_translate 116 | 117 | 118 | Compress tif 119 | 120 | gdal_translate -of GTiff -co COMPRESS=DEFLATE -co TILED=NO image1.tif image1_compressed.tif 121 | 122 | * optional argument to resize image -outsize 50% 50% 123 | *for GEOTIFF compression option -co NUM_THREADS=ALL_CPUS is available for better preformance 124 | 125 | 126 | Convert Multi-band GeoTiff file to JPEG: 127 | 128 | 129 | gdal_translate -of JPEG 884084-utm.tif 884084-utm.jpg 130 | 131 | 132 | If GDAL complains about strange tags used in a tif file (http://www.gdal.org/frmt_gtiff.html) 133 | 134 | 135 | gdal_translate -co < PROFILE=BASELINE > or input.tif output.tif 136 | 137 | Save a single band from a multi-band image 138 | 139 | gdal_translate -b 1 input7.tif output.tif 140 | 141 | Re-order bands 142 | source: https://medium.com/planet-stories/a-gentle-introduction-to-gdal-part-4-working-with-satellite-data-d3835b5e2971 143 | 144 | # 5 band image to 3 band RGB image \ 145 | gdal_translate 1155205_2017-03-31_RE3_3A.tif \ 146 | 1155205_2017-03-31_RE3_3A_rgb.tif \ 147 | -b 3 -b 2 -b 1 \ 148 | -co COMPRESS=DEFLATE -co PHOTOMETRIC=RGB 149 | 150 | 151 | --- 152 | #### gdal_contour: 153 | 154 | 155 | gdal_contour -a elev dem.tif contour.shp -i 10.0 156 | 157 | 158 | 159 | 160 | --- 161 | #### gdal_merge 162 | 163 | Merge DEMs 164 | 165 | gdal_merge srtm_35_01.tif srtm_35_02.tif srtm_35_03.tif srtm_36_01.tif srtm_36_02.tif srtm_37_02.tif -o merged_DEM.tif 166 | 167 | Merge Rasters: 168 | 169 | gdal_merge -o Theale_merged.tif Theale1_cal.bmp Theale2_cal.bmp Theale3_cal.bmp Theale4_cal.bmp 170 | 171 | Merge Separate Bands into a single file: (band order matters) 172 | source: (https://medium.com/planet-stories/a-gentle-introduction-to-gdal-part-4-working-with-satellite-data-d3835b5e2971) 173 | 174 | gdal_merge.py -o final_image.tif -separate image_B4.TIF iamge_B3.TIF image_B2.TIF -co PHOTOMETRIC=RGB -co COMPRESS=DEFLATE 175 | 176 | 177 | --- 178 | Copy all tifs to new location 179 | 180 | for %I in (image1.tif image2.tif image3.tif image4.tif) \ 181 | do \ 182 | copy %I test\folder\ 183 | 184 | 185 | --- 186 | #### gdal2tiles - Create web map tiles 187 | 188 | gdal2tiles.py --zoom=11-15 --title=maptitle in.tif output_folder_name 189 | 190 | --- 191 | #### gdal polygonize 192 | 193 | gdal_polygonize Project_clip1.tif -f "ESRI Shapefile" vector.shp crit2 Value 194 | 195 | --- 196 | #### gdal calc 197 | source: http://www.gdal.org/gdal_calc.html 198 | 199 | Average 2 bands together 200 | gdal_calc.py -A input.tif -B input2.tif --outfile=result.tif --calc="(A+B)/2" 201 | 202 | --- 203 | 204 | --- 205 | 206 | ## OGR 207 | 208 | #### SUBSET SHAPEFILE: 209 | 210 | 211 | ogr2ogr -spat -1.5 51 -0.5 52 -f "ESRI Shapefile" watersubset.shp waterways.shp 212 | 213 | 214 | --- 215 | #### MEND SHAPEFILE: 216 | 217 | 218 | ogr2ogr -skipfailures -f "ESRI Shapefile" mended.shp broken.shp 219 | 220 | 221 | --- 222 | #### KML to SHP: 223 | 224 | 225 | ogr2ogr -f "ESRI Shapefile" thames.shp thames.kml 226 | 227 | 228 | --- 229 | #### CONVERT SHAPE TO KML and change proj: 230 | 231 | 232 | ogr2ogr -f "KML" -s_srs "EPSG:27700" -t_srs "EPSG:4326" rail.kml rail_OSGB36.shp 233 | 234 | 235 | --- 236 | #### Convert shp to kml w/ descriptions: 237 | 238 | 239 | ogr2ogr -f "KML" sample.kml sample.shp -dsco NameField=Field1 -dsco DescriptionField=field2 240 | 241 | 242 | --- 243 | #### Convert shp to kml - using "where" as query featuures: 244 | 245 | 246 | ogr2ogr -f "KML" -where "NBRHOOD='Telegraph Hill'" realtor_neighborhoods.kml realtor_neighborhoods.shp 247 | 248 | 249 | --- 250 | #### Convert shp to kml - using "select" to add specific attributes: 251 | 252 | 253 | ogr2ogr –SELECT “field1 field2 field3” -t_srs EPSG:4326 -f "KML" outPutFileName.kml inPutFileName.shp 254 | 255 | 256 | --- 257 | #### REPROJECT SHAPE FILE: 258 | 259 | 260 | ogr2ogr -s_srs "EPSG:4326" -t_srs "EPSG:27700" buildings_OS.shp buildings.shp 261 | 262 | 263 | --- 264 | #### Get information about a shapefile: (List Fields and type) 265 | 266 | 267 | ogrinfo -al ssurgo_geo.shp **This Lists ALL geometry which can be a pain 268 | 269 | ogrinfo -al -geom=NO ssurgo_geo.shp **Will not list millions of pages of geometry 270 | 271 | 272 | --- 273 | #### Extract all polygons from a SSURGO shapefile where the mapunit symbol is 'ScA': 274 | 275 | 276 | ogr2ogr -where "musym = 'ScA' " ssurgo_ScA.shp ssurgo_utm.shp 277 | 278 | 279 | --- 280 | #### GPX files 281 | source: http://www.gdal.org/ogr/drv_gpx.html 282 | 283 | ogr2ogr --config GPX_SHORT_NAMES YES out input.gpx track_points 284 | 285 | * GPX_SHORT_NAMES YES = converts long column names to shorter names to prevent non-unique names 286 | * out = is the ouput location and folder 287 | * track_points = the feature type you are converting/extracting from the file. Other options are: waypoints, route_points, routes, tracks. If nothing is specified then all are extracted. An empty shp file is created for those with no features. 288 | 289 | 290 | 291 | --- 292 | #### iterate over many files linux/unix (source: http://gothos.info/tag/gdal-ogr/) 293 | 294 | 295 | #!/bin/bash 296 | from Sherman (2008) Desktop GIS Mapping the Planet With Open Source Tools pp 243-44 297 | 298 | for shp in *.shp \ 299 | do \ 300 | echo “Processing $shp” \ 301 | ogr2ogr -f “ESRI Shapefile” -t_srs EPSG:4326 geo/$shp $shp \ 302 | done \ 303 | 304 | 305 | --- 306 | #### Ogr with SQL 307 | 308 | source: http://www.sarasafavi.com/intro-to-ogr-part-i-exploring-data.html 309 | 310 | ogrinfo city_of_austin_parks.shp -sql "SELECT COUNT(*) FROM city_of_austin_parks" 311 | 312 | * add '-so' for summary only 313 | 314 | 315 | SQL and look at only one feature 316 | 317 | ogrinfo -q city_of_austin_parks.shp -sql "SELECT * FROM city_of_austin_parks" -fid 1 318 | 319 | * '-q' = quiet 320 | 321 | 322 | Same using all SQL 323 | 324 | ogrinfo -q city_of_austin_parks.shp -sql "SELECT * FROM city_of_austin_parks WHERE fid IN (1,3)" 325 | 326 | --- 327 | 328 | #### Import into Postgis (from PostGIS Cookbook, 2014) 329 | 330 | ogr2ogr -f PostgreSQL -sql "SELECT ISO2, NAME AS country_name \ 331 | FROM 'TM_WORLD_BORDERS-0.3' WHERE REGION=2" \ 332 | -nlt MULTIPOLYGON \ 333 | PG:"host=000.000.000.000 dbname='postgis_cookbook' user='me' password='mypassword'" \ 334 | -nln africa_countries \ 335 | -lco SCHEMA=chp01 \ 336 | -lco GEOMETRY_NAME=the_geom \ 337 | TM_WORLD_BORDERS-0.3.shp 338 | 339 | #### Export from Postgis (from PostGIS Cookbook, 2014) 340 | 341 | ogr2ogr -f GeoJSON -t_srs EPSG:4326 warmest_hs.geojson \ 342 | PG:"host=000.000.000.000 dbname='postgis_cookbook' user='me' password='mypassword'" \ 343 | -sql "SELECT f.the_geom as the_geom, f.bright_t31, ac.iso2, ac.country_name \ 344 | FROM chp01.global_24h as f \ 345 | JOIN chp01.africa_countries as ac \ 346 | ON ST_Contains(ac.the_geom, ST_Transform(f.the_geom, 4326)) \ 347 | ORDER BY f.bright_t31 DESC LIMIT 100" 348 | 349 | --- 350 | 351 | #### Using OGR with GNU Parallel 352 | 353 | Source:http://blog.faraday.io/how-to-crunch-lots-of-geodata-in-parallel/ 354 | 355 | mkdir wgs84 356 | ls *.shp | parallel ogr2ogr -t_srs 'EPSG:4326' wgs84/{} {} 357 | 358 | 359 | --------------------------------------------------------------------------------