├── 01.Filtering-Image-Collection.md ├── 02.Atm-correction.md ├── 03.cloudmaskTOA.md ├── 04.topo_correction.md ├── 05.brdf_correction.md ├── 07.reprojection.md ├── 08.image_registration.md ├── 09.band_adjustment.md ├── 10.time_series.md ├── 11.crop_mapping.md ├── README.md └── jupyter_notebooks ├── 02.Atm-corr-Landsat7.ipynb ├── 02.Atm-corr-Landsat8.ipynb ├── 02.Atm-corr-Sentinel2.ipynb └── python_guide.md /01.Filtering-Image-Collection.md: -------------------------------------------------------------------------------- 1 | # [geeguide](README.md) 2 | 3 | # 01.Filtering Image Collection 4 | https://code.earthengine.google.com/f06fe3e5961eb031cb16ca81d115d873 5 | ## Objective 6 | Filtering image collection based on: 7 | - Time of interest 8 | - Area of interest 9 | - Metadata (cloud cover, path, row, etc.) 10 | 11 | ## Core script 12 | ``` 13 | var geometry = ee.Geometry.Point([108.94553918036411,11.566755394830713]) 14 | var start = '2018-01-01'; 15 | var end = '2018-12-31'; 16 | var L8_col = ee.ImageCollection('LANDSAT/LC08/C01/T1_TOA') 17 | .filterBounds(geometry) 18 | .filterDate(start,end) 19 | .filter(ee.Filter.eq('WRS_PATH',123)) 20 | .filter(ee.Filter.eq('WRS_ROW',52)) 21 | .filter(ee.Filter.lt('CLOUD_COVER',95)) 22 | .sort('CLOUD_COVER') //first image will be the less cloudy image in the collection 23 | 24 | Map.centerObject(geometry,10) 25 | 26 | ``` 27 | 28 | ## Visualization and Checking 29 | ``` 30 | print(L8_col.size(),'L8_col size') //.size() is the number of images 31 | print(L8_col.first(),'first image') //first image will be the less cloudy image 32 | ``` 33 | -------------------------------------------------------------------------------- /02.Atm-correction.md: -------------------------------------------------------------------------------- 1 | # [geeguide](README.md) 2 | 3 | # 02.Atmospheric Correction 4 | ## Objective 5 | Atmospheric correction of Landsat7,8 and Sentinel 2 in Google Earth Engine 6 | - Both Landsat and Sentinel 2 images were atmospheric corrected using one Py6S model 7 | - TOA --> BOA using [Py6S](http://rtwilson.com/academic/Wilson_2012_Py6S_Paper.pdf) via GEE python API 8 | - Generate BOA dataset to your GEE asset 9 | 10 | ## General Instruction 11 | - GEE support both [Web-based Code Editor](https://developers.google.com/earth-engine/playground) and [Python API](https://www.earthdatascience.org/tutorials/intro-google-earth-engine-python-api/). Each environment has its own advantage and disadvantage, we will use only Python API when applying atmospheric correction, all the others tasks will be Code Editor based 12 | - First, please follow this instruction by [Samsammurphy](https://github.com/samsammurphy/gee-atmcorr-S2) on atmospheric correction of a single Sentinel 2 image using [Py6S](http://rtwilson.com/academic/Wilson_2012_Py6S_Paper.pdf). You will be guided through how to setup GEE python environment, authentication, [docker](https://www.docker.com/products/container-runtime), etc 13 | - Second, I have modified [Samsammurphy](https://github.com/samsammurphy/gee-atmcorr-S2)'s version so that we can apply it to Landsat 7, Landsat 8 and iterate to a whole image collection. You should pay attention to the part about turning on/off the Exporting to GEE Asset. Making some tests before turning on full scale is recommended. 14 | - If you still want to do atmospheric correction in GEE Code Editor, you can use [SIAC_GEE by MarcYin](https://github.com/MarcYin/SIAC_GEE). 15 | ## Core Script 16 | - Sentinel 2 Atmospheric Correction 17 | ``` 18 | https://github.com/ndminhhus/geeguide/blob/master/jupyter_notebooks/02.Atm-corr-Sentinel2.ipynb 19 | ``` 20 | - Landsat 8 Atmospheric Correction 21 | ``` 22 | https://github.com/ndminhhus/geeguide/blob/master/jupyter_notebooks/02.Atm-corr-Landsat8.ipynb 23 | ``` 24 | - Landsat 7 Atmospheric Correction 25 | ``` 26 | https://github.com/ndminhhus/geeguide/blob/master/jupyter_notebooks/02.Atm-corr-Landsat7.ipynb 27 | ``` 28 | ## Visualization and Checking 29 | 30 | - Landsat 8 before-after 31 | 32 | ![Landsat 8](https://user-images.githubusercontent.com/40456844/61792009-f1777a80-ae1b-11e9-9839-101f6279ffb6.png) 33 | 34 | - Sentinel 2 before-after 35 | 36 | ![Sentinel 2](https://user-images.githubusercontent.com/40456844/61792002-efadb700-ae1b-11e9-9319-d1182fe51caa.png) 37 | 38 | 39 | # References 40 | 1. Mission specific for atmospheric correction 41 | https://github.com/samsammurphy/ee-atmcorr-timeseries/blob/master/atmcorr/mission_specifics.py 42 | 2. Atmospheric correction of Sentinel 2 imagery in Google Earth Engine using Py6S. 43 | https://github.com/samsammurphy/gee-atmcorr-S2 44 | 3. Introduction to the Google Earth Engine Python API 45 | https://www.earthdatascience.org/tutorials/intro-google-earth-engine-python-api/ 46 | 4. Py6S: A Python interface to the 6S Radiative Transfer Model 47 | http://rtwilson.com/academic/Wilson_2012_Py6S_Paper.pdf 48 | 5. 6S radiative transfer model 49 | https://doi.org/10.1109/36.581987 50 | 6. Basic writing and formatting syntax 51 | https://help.github.com/en/articles/basic-writing-and-formatting-syntax#relative-links 52 | -------------------------------------------------------------------------------- /03.cloudmaskTOA.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 03. Cloud Masking of Landsat and Sentinel2 TOA Products 3 | 4 | ## Objective 5 | - Landsat TOA Cloud and Shadow Mask 6 | - Sentinel2 TOA Cloud and Shadow Mask 7 | ## General Information 8 | - Cloud contamination will lower NDVI values, measures like the timing of ‘green up’ or peak maturity would appear later than they actually occurred ([USGS, 2019](https://www.usgs.gov/land-resources/nli/landsat/landsat-collection-1-level-1-quality-assessment-band?qt-science_support_page_related_con=0#qt-science_support_page_related_con)). Therefore, cloud should be masked effectively and 'aggressively' when the images are used to analysis crop’s phenology. 9 | - For Landsat TOA product, BQA band can be used to mask out cloud with decent performance. BQA band was generated using [CFMask Algorithm](https://www.usgs.gov/land-resources/nli/landsat/cfmask-algorithm). 10 | Or ```ee.Algorithms.Landsat.simpleCloudScore()``` is a built-in GEE function can provide a shortcut to a rudimentary cloud scoring algorithm [GEE Documentation](https://developers.google.com/earth-engine/landsat). Behide the scene of the simpleCloudScore algorithm is described in this [gee script](https://code.earthengine.google.com/dc5611259d9ccab952526b3c2d05ce07). 11 | - However, without a thermal band, current cloud detection methods for Sentinel 2 (eg.Fmask, Sen2Cor, MAJA, etc) could not provide satisfy results. Bright land surface such as white roof, sand beach, bare land, high turbidity surface water, etc. could be miss identified as cloud. Sentinel-2 optimal cloud mask is the main issue in [hls.gsfc.nasa.gov](https://hls.gsfc.nasa.gov/) initiative ([Claverie et.al, 2018](https://doi.org/10.1016/j.rse.2018.09.002)) 12 | - I have been working on a S2 cloud mask based on [Supervised Classification](https://developers.google.com/earth-engine/classification) and QA60 was used as training data. The results look promissing but it may be updated later. 13 | 14 | ## Core script 15 | ### Cloud Mask Landsat8 16 | - Based on [Quality Assessment Band](https://www.usgs.gov/land-resources/nli/landsat/landsat-collection-1-level-1-quality-assessment-band?qt-science_support_page_related_con=0#qt-science_support_page_related_con) 17 | ``` 18 | // Landsat 8 Cloud Masking Example 19 | 20 | var RADIX = 2; // Radix for binary (base 2) data. 21 | 22 | // Reference a sample Landsat 8 TOA image. 23 | var image = ee.Image('LANDSAT/LC08/C01/T1_TOA/LC08_043034_20180202'); 24 | // Extract the QA band. 25 | var image_qa = image.select('BQA'); 26 | 27 | var extractQABits = function (qaBand, bitStart, bitEnd) { 28 | var numBits = bitEnd - bitStart + 1; 29 | var qaBits = qaBand.rightShift(bitStart).mod(Math.pow(RADIX, numBits)); 30 | //Map.addLayer(qaBits, {min:0, max:(Math.pow(RADIX, numBits)-1)}, 'qaBits'); 31 | return qaBits; 32 | }; 33 | 34 | // Create a mask for the dual QA bit "Cloud Confidence". 35 | var bitStartCloudConfidence = 5; 36 | var bitEndCloudConfidence = 6; 37 | var qaBitsCloudConfidence = extractQABits(image_qa, bitStartCloudConfidence, bitEndCloudConfidence); 38 | // Test for clouds, based on the Cloud Confidence value. 39 | var testCloudConfidence = qaBitsCloudConfidence.gte(2); 40 | 41 | // Create a mask for the dual QA bit "Cloud Shadow Confidence". 42 | var bitStartShadowConfidence = 7; 43 | var bitEndShadowConfidence = 8; 44 | var qaBitsShadowConfidence = extractQABits(image_qa, bitStartShadowConfidence, bitEndShadowConfidence); 45 | // Test for shadows, based on the Cloud Shadow Confidence value. 46 | var testShadowConfidence = qaBitsShadowConfidence.gte(2); 47 | 48 | // Calculate a composite mask and apply it to the image. 49 | var maskComposite = (testCloudConfidence.or(testShadowConfidence)).not(); 50 | var imageMasked = image.updateMask(maskComposite); 51 | 52 | Map.addLayer(image, {bands:"B4,B3,B2", min:0, max:0.3}, 'original image', false); 53 | Map.addLayer(image.select('BQA'), {min:0, max:10000}, 'BQA', false); 54 | Map.addLayer(testCloudConfidence.mask(testCloudConfidence), {min:0, max:1, palette:'grey,white'}, 'testCloudConfidence'); 55 | Map.addLayer(testShadowConfidence.mask(testShadowConfidence), {min:0, max:1, palette:'grey,black'}, 'testShadowConfidence'); 56 | Map.addLayer(maskComposite.mask(maskComposite), {min:0, max:1, palette:'green'}, 'maskComposite', false); 57 | Map.addLayer(imageMasked, {bands:"B4,B3,B2", min:0, max:0.3}, 'imageMasked'); 58 | ``` 59 | - Based on [ee.Algorithms.Landsat.simpleCloudScore](https://developers.google.com/earth-engine/landsat) 60 | ``` 61 | //This function even calculates percentage of cloud coverage over a particular region. Useful for screening too cloudy images 62 | 63 | function scoreL8_TOA (image) { 64 | var cloud = ee.Algorithms.Landsat.simpleCloudScore(image).select('cloud').rename('cloudScore'); 65 | var cloudiness = cloud.reduceRegion({ 66 | reducer: 'mean', 67 | geometry: region, 68 | scale: 30, 69 | }); 70 | return image.set(cloudiness).addBands(cloud); 71 | } 72 | 73 | function maskL8_TOA (img){ 74 | var mask = img.select('cloudScore').lt(20); 75 | return img.updateMask(mask) 76 | } 77 | ``` 78 | ### Cloud Mask Sentinel 2 79 | 80 | ``` 81 | //CloudScore originally written by Matt Hancher and adapted for S2 data by Ian Housman 82 | ////////////////////////////////////////////////////////// 83 | ///https://code.earthengine.google.com/c0b316ba2b56121b82318ffd1e5c42de 84 | //User Params 85 | var startYear = 2016; 86 | var endYear = 2017; 87 | var startJulian = 190; 88 | var endJulian = 250; 89 | var cloudThresh =20;//Ranges from 1-100.Lower value will mask more pixels out. Generally 10-30 works well with 20 being used most commonly 90 | var cloudHeights = ee.List.sequence(200,10000,250);//Height of clouds to use to project cloud shadows 91 | var irSumThresh =0.35;//Sum of IR bands to include as shadows within TDOM and the shadow shift method (lower number masks out less) 92 | var dilatePixels = 2; //Pixels to dilate around clouds 93 | var contractPixels = 1;//Pixels to reduce cloud mask and dark shadows by to reduce inclusion of single-pixel comission errors 94 | 95 | ////////////////////////////////////////////////////////// 96 | var vizParams = {'min': 0.05,'max': [0.3,0.6,0.35], 'bands':'swir1,nir,red'}; 97 | // var vizParams = {bands: ['red', 'green', 'blue'], min: 0, max: 0.3}; 98 | ////////////////////////////////////////////////////////////////////////// 99 | var rescale = function(img, exp, thresholds) { 100 | return img.expression(exp, {img: img}) 101 | .subtract(thresholds[0]).divide(thresholds[1] - thresholds[0]); 102 | }; 103 | 104 | //////////////////////////////////////// 105 | //////////////////////////////////////// 106 | // Cloud masking algorithm for Sentinel2 107 | //Built on ideas from Landsat cloudScore algorithm 108 | //Currently in beta and may need tweaking for individual study areas 109 | function sentinelCloudScore(img) { 110 | 111 | 112 | // Compute several indicators of cloudyness and take the minimum of them. 113 | var score = ee.Image(1); 114 | 115 | // Clouds are reasonably bright in the blue and cirrus bands. 116 | score = score.min(rescale(img, 'img.blue', [0.1, 0.5])); 117 | score = score.min(rescale(img, 'img.cb', [0.1, 0.3])); 118 | score = score.min(rescale(img, 'img.cb + img.cirrus', [0.15, 0.2])); 119 | 120 | // Clouds are reasonably bright in all visible bands. 121 | score = score.min(rescale(img, 'img.red + img.green + img.blue', [0.2, 0.8])); 122 | 123 | 124 | //Clouds are moist 125 | var ndmi = img.normalizedDifference(['nir','swir1']); 126 | score=score.min(rescale(ndmi, 'img', [-0.1, 0.1])); 127 | 128 | // However, clouds are not snow. 129 | var ndsi = img.normalizedDifference(['green', 'swir1']); 130 | score=score.min(rescale(ndsi, 'img', [0.8, 0.6])); 131 | 132 | score = score.multiply(100).byte(); 133 | 134 | return img.addBands(score.rename('cloudScore')); 135 | } 136 | ////////////////////////////////////////////////////////////////////////// 137 | // Function to mask clouds using the Sentinel-2 QA band. 138 | function maskS2clouds(image) { 139 | var qa = image.select('QA60').int16(); 140 | 141 | // Bits 10 and 11 are clouds and cirrus, respectively. 142 | var cloudBitMask = Math.pow(2, 10); 143 | var cirrusBitMask = Math.pow(2, 11); 144 | 145 | // Both flags should be set to zero, indicating clear conditions. 146 | var mask = qa.bitwiseAnd(cloudBitMask).eq(0).and( 147 | qa.bitwiseAnd(cirrusBitMask).eq(0)); 148 | 149 | // Return the masked and scaled data. 150 | return image.updateMask(mask); 151 | } 152 | ////////////////////////////////////////////////////// 153 | ////////////////////////////////////////////////////////////////////////// 154 | //Function for finding dark outliers in time series 155 | //Masks pixels that are dark, and dark outliers 156 | function simpleTDOM2(c){ 157 | var shadowSumBands = ['nir','swir1']; 158 | var irSumThresh = 0.4; 159 | var zShadowThresh = -1.2; 160 | //Get some pixel-wise stats for the time series 161 | var irStdDev = c.select(shadowSumBands).reduce(ee.Reducer.stdDev()); 162 | var irMean = c.select(shadowSumBands).mean(); 163 | var bandNames = ee.Image(c.first()).bandNames(); 164 | print('bandNames',bandNames); 165 | //Mask out dark dark outliers 166 | c = c.map(function(img){ 167 | var z = img.select(shadowSumBands).subtract(irMean).divide(irStdDev); 168 | var irSum = img.select(shadowSumBands).reduce(ee.Reducer.sum()); 169 | var m = z.lt(zShadowThresh).reduce(ee.Reducer.sum()).eq(2).and(irSum.lt(irSumThresh)).not(); 170 | 171 | return img.updateMask(img.mask().and(m)); 172 | }); 173 | 174 | return c.select(bandNames); 175 | } 176 | //////////////////////////////////////////////////////// 177 | ///////////////////////////////////////////// 178 | /*** 179 | * Implementation of Basic cloud shadow shift 180 | * 181 | * Author: Gennadii Donchyts 182 | * License: Apache 2.0 183 | */ 184 | function projectShadows(cloudMask,image,cloudHeights){ 185 | var meanAzimuth = image.get('MEAN_SOLAR_AZIMUTH_ANGLE'); 186 | var meanZenith = image.get('MEAN_SOLAR_ZENITH_ANGLE'); 187 | /////////////////////////////////////////////////////// 188 | // print('a',meanAzimuth); 189 | // print('z',meanZenith) 190 | 191 | //Find dark pixels 192 | var darkPixels = image.select(['nir','swir1','swir2']).reduce(ee.Reducer.sum()).lt(irSumThresh) 193 | .focal_min(contractPixels).focal_max(dilatePixels) 194 | ;//.gte(1); 195 | 196 | 197 | //Get scale of image 198 | var nominalScale = cloudMask.projection().nominalScale(); 199 | //Find where cloud shadows should be based on solar geometry 200 | //Convert to radians 201 | var azR =ee.Number(meanAzimuth).add(180).multiply(Math.PI).divide(180.0); 202 | var zenR =ee.Number(meanZenith).multiply(Math.PI).divide(180.0); 203 | 204 | 205 | 206 | //Find the shadows 207 | var shadows = cloudHeights.map(function(cloudHeight){ 208 | cloudHeight = ee.Number(cloudHeight); 209 | 210 | var shadowCastedDistance = zenR.tan().multiply(cloudHeight);//Distance shadow is cast 211 | var x = azR.sin().multiply(shadowCastedDistance).divide(nominalScale);//X distance of shadow 212 | var y = azR.cos().multiply(shadowCastedDistance).divide(nominalScale);//Y distance of shadow 213 | // print(x,y) 214 | 215 | return cloudMask.changeProj(cloudMask.projection(), cloudMask.projection().translate(x, y)); 216 | 217 | 218 | }); 219 | 220 | 221 | var shadowMask = ee.ImageCollection.fromImages(shadows).max(); 222 | // Map.addLayer(cloudMask.updateMask(cloudMask),{'min':1,'max':1,'palette':'88F'},'Cloud mask'); 223 | // Map.addLayer(shadowMask.updateMask(shadowMask),{'min':1,'max':1,'palette':'880'},'Shadow mask'); 224 | 225 | //Create shadow mask 226 | shadowMask = shadowMask.and(cloudMask.not()); 227 | shadowMask = shadowMask.and(darkPixels).focal_min(contractPixels).focal_max(dilatePixels); 228 | 229 | var cloudShadowMask = shadowMask.or(cloudMask); 230 | 231 | image = image.updateMask(cloudShadowMask.not()).addBands(shadowMask.rename(['cloudShadowMask'])); 232 | return image; 233 | } 234 | ////////////////////////////////////////////////////// 235 | //Function to bust clouds from S2 image 236 | function bustClouds(img){ 237 | img = sentinelCloudScore(img); 238 | img = img.updateMask(img.select(['cloudScore']).gt(cloudThresh).focal_min(contractPixels).focal_max(dilatePixels).not()); 239 | return img; 240 | } 241 | ////////////////////////////////////////////////////// 242 | //Function for wrapping the entire process to be applied across collection 243 | function wrapIt(img){ 244 | img = sentinelCloudScore(img); 245 | var cloudMask = img.select(['cloudScore']).gt(cloudThresh) 246 | .focal_min(contractPixels).focal_max(dilatePixels) 247 | 248 | img = projectShadows(cloudMask,img,cloudHeights); 249 | 250 | return img; 251 | } 252 | ////////////////////////////////////////////////////// 253 | //Function to find unique values of a field in a collection 254 | function uniqueValues(collection,field){ 255 | var values =ee.Dictionary(collection.reduceColumns(ee.Reducer.frequencyHistogram(),[field]).get('histogram')).keys(); 256 | 257 | return values; 258 | } 259 | ////////////////////////////////////////////////////// 260 | //Function to simplify data into daily mosaics 261 | function dailyMosaics(imgs){ 262 | //Simplify date to exclude time of day 263 | imgs = imgs.map(function(img){ 264 | var d = ee.Date(img.get('system:time_start')); 265 | var day = d.get('day'); 266 | var m = d.get('month'); 267 | var y = d.get('year'); 268 | var simpleDate = ee.Date.fromYMD(y,m,day); 269 | return img.set('simpleTime',simpleDate.millis()); 270 | }); 271 | 272 | //Find the unique days 273 | var days = uniqueValues(imgs,'simpleTime'); 274 | 275 | imgs = days.map(function(d){ 276 | d = ee.Number.parse(d); 277 | d = ee.Date(d); 278 | var t = imgs.filterDate(d,d.advance(1,'day')); 279 | var f = ee.Image(t.first()); 280 | t = t.mosaic(); 281 | t = t.set('system:time_start',d.millis()); 282 | t = t.copyProperties(f); 283 | return t; 284 | }); 285 | imgs = ee.ImageCollection.fromImages(imgs); 286 | 287 | return imgs; 288 | } 289 | ////////////////////////////////////////////////////// 290 | //Get some s2 data 291 | var s2s = ee.ImageCollection('COPERNICUS/S2') 292 | .filter(ee.Filter.calendarRange(startYear,endYear,'year')) 293 | .filter(ee.Filter.calendarRange(startJulian,endJulian)) 294 | .filterBounds(geometry) 295 | .map(function(img){ 296 | 297 | var t = img.select([ 'B1','B2','B3','B4','B5','B6','B7','B8','B8A', 'B9','B10', 'B11','B12']).divide(10000);//Rescale to 0-1 298 | t = t.addBands(img.select(['QA60'])); 299 | var out = t.copyProperties(img).copyProperties(img,['system:time_start']); 300 | return out; 301 | }) 302 | .select(['QA60', 'B1','B2','B3','B4','B5','B6','B7','B8','B8A', 'B9','B10', 'B11','B12'],['QA60','cb', 'blue', 'green', 'red', 're1','re2','re3','nir', 'nir2', 'waterVapor', 'cirrus','swir1', 'swir2']); 303 | 304 | //Convert to daily mosaics to avoid redundent observations in MGRS overlap areas and edge artifacts for shadow masking 305 | s2s = dailyMosaics(s2s); 306 | print(s2s); 307 | ////////////////////////////////////////////////// 308 | //Optional- View S2 daily mosaics 309 | // days.getInfo().map(function(d){ 310 | // var ds = new Date(parseInt(d)) 311 | // d = ee.Date(ee.Number.parse(d)) 312 | 313 | // print(ds) 314 | // var s2sT = ee.Image(s2s.filterDate(d,d.advance(1,'minute')).first()); 315 | 316 | // Map.addLayer(s2sT,vizParams,ds,false); 317 | // }) 318 | ///////////////////////////////////////////////////// 319 | //Look at individual image 320 | // var s2 = ee.Image(s2s.first()); 321 | // Map.addLayer(s2,vizParams,'Before Masking',false); 322 | 323 | // s2 = sentinelCloudScore(s2); 324 | // var cloudScore = s2.select('cloudScore'); 325 | // Map.addLayer(cloudScore,{'min':0,'max':100},'CloudScore',false); 326 | 327 | 328 | // var s2CloudMasked = bustClouds(s2); 329 | // Map.addLayer(s2CloudMasked,vizParams,'After CloudScore Masking',false); 330 | 331 | // var s2CloudShadowMasked = wrapIt(s2); 332 | // Map.addLayer(s2CloudShadowMasked,vizParams,'After CloudScore and Shadow Masking',false); 333 | 334 | // var s2MaskedQA = maskS2clouds(s2) ; 335 | // print(s2) 336 | // Map.addLayer(s2MaskedQA,vizParams,'After QA Masking',false); 337 | 338 | 339 | // var s2TDOMMasked = ee.Image(simpleTDOM2(s2s.map(bustClouds)).first()); 340 | // Map.addLayer(s2TDOMMasked,vizParams,'After CloudScore and TDOM Masking',false); 341 | 342 | 343 | ///////////////////////////////////////////// 344 | //Look at mosaics 345 | //Get the raw mosaic and median 346 | Map.addLayer(s2s.mosaic(),vizParams,'Raw Mosaic', false); 347 | Map.addLayer(s2s.median(),vizParams,'Raw Median', false); 348 | 349 | 350 | //Bust clouds using BQA method 351 | var s2MaskedQA = s2s.map(maskS2clouds); 352 | Map.addLayer(s2MaskedQA.mosaic(),vizParams,'QA Cloud Masked Mosaic',false); 353 | Map.addLayer(s2MaskedQA.median(),vizParams,'QA Cloud Masked Median',false); 354 | 355 | 356 | 357 | //Bust clouds using cloudScore method 358 | var s2sMosaic = s2s.map(bustClouds); 359 | Map.addLayer(s2sMosaic.mosaic(),vizParams,'CloudScore Masked Mosaic', false); 360 | Map.addLayer(s2sMosaic.median(),vizParams,'CloudScore Masked Median', false); 361 | 362 | 363 | //Bust clouds using cloudScore and shadows using TDOM 364 | var s2TDOM = simpleTDOM2(s2sMosaic); 365 | Map.addLayer(s2TDOM.mosaic(),vizParams,'CloudScore and TDOM Masked Mosaic', false); 366 | Map.addLayer(s2TDOM.median(),vizParams,'CloudScore and TDOM Masked Median', false); 367 | 368 | //Bust clouds using cloudScore and shadows using shadow shift method 369 | var s2sMosaicCloudsProjectedDark = s2s.map(wrapIt) 370 | Map.addLayer(s2sMosaicCloudsProjectedDark.mosaic(), vizParams, 'Cloud Masked + Projected Shadows + Dark + Mosaic ',false); 371 | Map.addLayer(s2sMosaicCloudsProjectedDark.median(), vizParams, 'Cloud Masked + Projected Shadows + Dark + Median ',false); 372 | 373 | ``` 374 | ## Visualization and Checking 375 | 376 | ## References 377 | 1. [Cloud masking Sentinel 2, Google Earth Engine Developers](https://groups.google.com/forum/#!msg/google-earth-engine-developers/i63DS-Dg8Sg/kbPBxA3ZAQAJ). 378 | 2. [Landsat 8 BQA - using cloud confidence to create a cloud mask, Stack Exchange](https://gis.stackexchange.com/questions/292835/landsat-8-bqa-using-cloud-confidence-to-create-a-cloud-mask) 379 | 380 | 381 | -------------------------------------------------------------------------------- /04.topo_correction.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 04. Topographic and BRDF Correction 3 | https://code.earthengine.google.com/9b98dc9cb9635a230149fe61cf5c18b5 4 | ## Objective 5 | - Topographic and BRDF normalziation for both Landsat and Sentinel 2 images 6 | ## General Information 7 | ### Topographic correction 8 | - Accounts for variations in reflectance (of similar features) due to slope, aspect and elevation. Topographic correction is not always required, but sometimes can be more important than atmospheric correction in mountainous or rugged terrain ([Colby, 1991](https://esajournals.onlinelibrary.wiley.com/servlet/linkout?suffix=null&dbid=128&doi=10.1002%2Fecy.1730&key=A1991FJ61200006), [Vanonckelen et al. 2013](https://doi.org/10.1016/j.jag.2013.02.003)). 9 | - Implementation of this topographic correction model in GEE by ([Poortinga et al., 2019](https://doi.org/10.3390/rs11070831)). The method based on the modified Sun-Canopy-Sensor Topographic Correction as described by ([Soenen et al., 2005](https://doi.org/10.1109/TGRS.2005.852480)). 10 | 11 | ### BRDF Correction 12 | - View and illumination angles (BRDF) adjustment account for differing solar and view angles associated with Landsat 8 and Sentinel-2 ([Claverie el al., 2018](https://doi.org/10.1016/j.rse.2018.09.002)). 13 | - Implementation in GEE developed by ([Poortinga et al., 2019](https://doi.org/10.3390/rs11070831)) based on ([Roy et al., 2017](https://doi.org/10.1016/j.rse.2017.06.019)) and ([Roy et al., 2016](https://doi.org/10.1016/j.rse.2016.01.023)) results. 14 | - This BRDF correction model called MODIS BRDF-based c-factor uses fixed coefficients originally developed for Landsat but proven to be working for S2 as well ([Roy et al., 2017](https://doi.org/10.1016/j.rse.2017.06.019) , [Roy et al., 2016](https://doi.org/10.1016/j.rse.2016.01.023), [Zhang et al., 2018](https://doi.org/10.1016/j.rse.2018.04.031)) 15 | 16 | ## Core script 17 | - 18 | ``` 19 | //This script corrects the BRDF and Topography effect of a L8 and S2 image, surface reflectance product 20 | //Adapted from: Poortinga et al.,2018 https://doi.org/10.3390/rs11070831 21 | 22 | 23 | var bandIn = ['B2','B3','B4','B5','B6','B7']; 24 | var bandOut = ['blue','green','red','nir','swir1','swir2']; 25 | 26 | //Example images of S2 and L8 27 | var imgS2SR = ee.Image('COPERNICUS/S2_SR/20181107T082129_20181107T082732_T36SYC').select(bandIn,bandOut); 28 | var imgL8SR = ee.Image('LANDSAT/LC08/C01/T1_SR/LC08_174036_20181107').select(bandIn,bandOut); 29 | 30 | // Step 1: BRDF correction 31 | var PI = ee.Number(3.14159265359); 32 | var MAX_SATELLITE_ZENITH = 7.5; 33 | var MAX_DISTANCE = 1000000; 34 | var UPPER_LEFT = 0; 35 | var LOWER_LEFT = 1; 36 | var LOWER_RIGHT = 2; 37 | var UPPER_RIGHT = 3; 38 | 39 | var imgS2_SR_BRDF = applyBRDF(imgS2SR);//BRDF S2 40 | var imgL8_SR_BRDF = applyBRDF_L8(imgL8SR); //BRDF L8 41 | 42 | //Step 2: Topographic correction 43 | var scale = 10; 44 | var dem = ee.Image("USGS/SRTMGL1_003") 45 | var degree2radian = 0.01745; 46 | 47 | var imgL8_Topo_Cond = illuminationCondition(imgL8_SR_BRDF) 48 | var imgL8_Topo_Corr = illuminationCorrection(imgL8_Topo_Cond) 49 | 50 | var imgS2_Topo_Cond = illuminationConditionS2(imgS2_SR_BRDF) 51 | var imgS2_Topo_Corr = illuminationCorrection(imgS2_Topo_Cond); 52 | 53 | //Topo corrected without BRDF to L8 and S2 54 | var imgL8_Topo_Cond_noBRDF = illuminationCondition(imgL8SR) 55 | var imgL8_Topo_Corr_noBRDF = illuminationCorrection(imgL8_Topo_Cond_noBRDF) 56 | 57 | var imgS2_Topo_Cond_noBRDF = illuminationConditionS2(imgS2SR) 58 | var imgS2_Topo_Corr_noBRDF = illuminationCorrection(imgS2_Topo_Cond_noBRDF); 59 | 60 | Map.centerObject(imgS2_Topo_Corr_noBRDF,10) 61 | 62 | 63 | //Global functions 64 | //BRDF correction 65 | //Source: https://doi.org/10.3390/rs11070831 66 | function applyBRDF_L8(image){ 67 | var date = ee.Date('2018-11-07'); 68 | var footprint = ee.List(image.geometry().bounds().bounds().coordinates().get(0)); 69 | var angles = getsunAngles(date, footprint); 70 | var sunAz = angles[0]; 71 | var sunZen = angles[1]; 72 | 73 | var viewAz = azimuth(footprint); 74 | var viewZen = zenith(footprint); 75 | 76 | 77 | var kval = _kvol(sunAz, sunZen, viewAz, viewZen); 78 | var kvol = kval[0]; 79 | var kvol0 = kval[1]; 80 | var result = _applyL8(image, kvol.multiply(PI), kvol0.multiply(PI)); 81 | 82 | return result; 83 | } 84 | function applyBRDF(image){ 85 | var date = image.date(); 86 | var footprint = ee.List(image.geometry().bounds().bounds().coordinates().get(0)); 87 | var angles = getsunAngles(date, footprint); 88 | var sunAz = angles[0]; 89 | var sunZen = angles[1]; 90 | 91 | var viewAz = azimuth(footprint); 92 | var viewZen = zenith(footprint); 93 | 94 | 95 | var kval = _kvol(sunAz, sunZen, viewAz, viewZen); 96 | var kvol = kval[0]; 97 | var kvol0 = kval[1]; 98 | var result = _apply(image, kvol.multiply(PI), kvol0.multiply(PI)); 99 | 100 | return result; 101 | } 102 | function getsunAngles(date, footprint){ 103 | var jdp = date.getFraction('year'); 104 | var seconds_in_hour = 3600; 105 | var hourGMT = ee.Number(date.getRelative('second', 'day')).divide(seconds_in_hour); 106 | 107 | var latRad = ee.Image.pixelLonLat().select('latitude').multiply(PI.divide(180)); 108 | var longDeg = ee.Image.pixelLonLat().select('longitude'); 109 | 110 | // Julian day proportion in radians 111 | var jdpr = jdp.multiply(PI).multiply(2); 112 | 113 | var a = ee.List([0.000075, 0.001868, 0.032077, 0.014615, 0.040849]); 114 | var meanSolarTime = longDeg.divide(15.0).add(ee.Number(hourGMT)); 115 | var localSolarDiff1 = value(a, 0) 116 | .add(value(a, 1).multiply(jdpr.cos())) 117 | .subtract(value(a, 2).multiply(jdpr.sin())) 118 | .subtract(value(a, 3).multiply(jdpr.multiply(2).cos())) 119 | .subtract(value(a, 4).multiply(jdpr.multiply(2).sin())); 120 | 121 | var localSolarDiff2 = localSolarDiff1.multiply(12 * 60); 122 | 123 | var localSolarDiff = localSolarDiff2.divide(PI); 124 | var trueSolarTime = meanSolarTime 125 | .add(localSolarDiff.divide(60)) 126 | .subtract(12.0); 127 | 128 | // Hour as an angle; 129 | var ah = trueSolarTime.multiply(ee.Number(MAX_SATELLITE_ZENITH * 2).multiply(PI.divide(180))) ; 130 | var b = ee.List([0.006918, 0.399912, 0.070257, 0.006758, 0.000907, 0.002697, 0.001480]); 131 | var delta = value(b, 0) 132 | .subtract(value(b, 1).multiply(jdpr.cos())) 133 | .add(value(b, 2).multiply(jdpr.sin())) 134 | .subtract(value(b, 3).multiply(jdpr.multiply(2).cos())) 135 | .add(value(b, 4).multiply(jdpr.multiply(2).sin())) 136 | .subtract(value(b, 5).multiply(jdpr.multiply(3).cos())) 137 | .add(value(b, 6).multiply(jdpr.multiply(3).sin())); 138 | 139 | var cosSunZen = latRad.sin().multiply(delta.sin()) 140 | .add(latRad.cos().multiply(ah.cos()).multiply(delta.cos())); 141 | var sunZen = cosSunZen.acos(); 142 | 143 | // sun azimuth from south, turning west 144 | var sinSunAzSW = ah.sin().multiply(delta.cos()).divide(sunZen.sin()); 145 | sinSunAzSW = sinSunAzSW.clamp(-1.0, 1.0); 146 | 147 | var cosSunAzSW = (latRad.cos().multiply(-1).multiply(delta.sin()) 148 | .add(latRad.sin().multiply(delta.cos()).multiply(ah.cos()))) 149 | .divide(sunZen.sin()); 150 | var sunAzSW = sinSunAzSW.asin(); 151 | 152 | sunAzSW = where(cosSunAzSW.lte(0), sunAzSW.multiply(-1).add(PI), sunAzSW); 153 | sunAzSW = where(cosSunAzSW.gt(0).and(sinSunAzSW.lte(0)), sunAzSW.add(PI.multiply(2)), sunAzSW); 154 | 155 | var sunAz = sunAzSW.add(PI); 156 | // # Keep within [0, 2pi] range 157 | sunAz = where(sunAz.gt(PI.multiply(2)), sunAz.subtract(PI.multiply(2)), sunAz); 158 | 159 | var footprint_polygon = ee.Geometry.Polygon(footprint); 160 | sunAz = sunAz.clip(footprint_polygon); 161 | sunAz = sunAz.rename(['sunAz']); 162 | sunZen = sunZen.clip(footprint_polygon).rename(['sunZen']); 163 | 164 | return [sunAz, sunZen]; 165 | } 166 | function azimuth(footprint){ 167 | function x(point){return ee.Number(ee.List(point).get(0))} 168 | function y(point){return ee.Number(ee.List(point).get(1))} 169 | 170 | var upperCenter = line_from_coords(footprint, UPPER_LEFT, UPPER_RIGHT).centroid().coordinates(); 171 | var lowerCenter = line_from_coords(footprint, LOWER_LEFT, LOWER_RIGHT).centroid().coordinates(); 172 | var slope = ((y(lowerCenter)).subtract(y(upperCenter))).divide((x(lowerCenter)).subtract(x(upperCenter))); 173 | var slopePerp = ee.Number(-1).divide(slope); 174 | var azimuthLeft = ee.Image(PI.divide(2).subtract((slopePerp).atan())); 175 | return azimuthLeft.rename(['viewAz']); 176 | } 177 | function zenith(footprint){ 178 | var leftLine = line_from_coords(footprint, UPPER_LEFT, LOWER_LEFT); 179 | var rightLine = line_from_coords(footprint, UPPER_RIGHT, LOWER_RIGHT); 180 | var leftDistance = ee.FeatureCollection(leftLine).distance(MAX_DISTANCE); 181 | var rightDistance = ee.FeatureCollection(rightLine).distance(MAX_DISTANCE); 182 | var viewZenith = rightDistance.multiply(ee.Number(MAX_SATELLITE_ZENITH * 2)) 183 | .divide(rightDistance.add(leftDistance)) 184 | .subtract(ee.Number(MAX_SATELLITE_ZENITH)) 185 | .clip(ee.Geometry.Polygon(footprint)) 186 | .rename(['viewZen']); 187 | return viewZenith.multiply(PI.divide(180)); 188 | } 189 | function _apply(image, kvol, kvol0){ 190 | var f_iso = 0; 191 | var f_geo = 0; 192 | var f_vol = 0; 193 | var blue = _correct_band(image, 'blue', kvol, kvol0, f_iso=0.0774, f_geo=0.0079, f_vol=0.0372); 194 | var green = _correct_band(image, 'green', kvol, kvol0, f_iso=0.1306, f_geo=0.0178, f_vol=0.0580); 195 | var red = _correct_band(image, 'red', kvol, kvol0, f_iso=0.1690, f_geo=0.0227, f_vol=0.0574); 196 | var re1 = _correct_band(image, 're1', kvol, kvol0, f_iso=0.2085, f_geo=0.0256, f_vol=0.0845); 197 | var re2 = _correct_band(image, 're2', kvol, kvol0, f_iso=0.2316, f_geo=0.0273, f_vol=0.1003); 198 | var re3 = _correct_band(image, 're3', kvol, kvol0, f_iso=0.2599, f_geo=0.0294, f_vol=0.1197); 199 | var nir = _correct_band(image, 'nir', kvol, kvol0, f_iso=0.3093, f_geo=0.0330, f_vol=0.1535); 200 | var re4 = _correct_band(image, 're4', kvol, kvol0, f_iso=0.2907, f_geo=0.0410, f_vol=0.1611); 201 | var swir1 = _correct_band(image, 'swir1', kvol, kvol0, f_iso=0.3430, f_geo=0.0453, f_vol=0.1154); 202 | var swir2 = _correct_band(image, 'swir2', kvol, kvol0, f_iso=0.2658, f_geo=0.0387, f_vol=0.0639); 203 | return image.select([]).addBands([blue, green, red, nir,re1,re2,re3,nir,re4,swir1, swir2]); 204 | } 205 | function _applyL8(image, kvol, kvol0){ 206 | var f_iso = 0; 207 | var f_geo = 0; 208 | var f_vol = 0; 209 | var blue = _correct_band(image, 'blue', kvol, kvol0, f_iso=0.0774, f_geo=0.0079, f_vol=0.0372); 210 | var green = _correct_band(image, 'green', kvol, kvol0, f_iso=0.1306, f_geo=0.0178, f_vol=0.0580); 211 | var red = _correct_band(image, 'red', kvol, kvol0, f_iso=0.1690, f_geo=0.0227, f_vol=0.0574); 212 | var nir = _correct_band(image, 'nir', kvol, kvol0, f_iso=0.3093, f_geo=0.0330, f_vol=0.1535); 213 | var swir1 = _correct_band(image, 'swir1', kvol, kvol0, f_iso=0.3430, f_geo=0.0453, f_vol=0.1154); 214 | var swir2 = _correct_band(image, 'swir2', kvol, kvol0, f_iso=0.2658, f_geo=0.0387, f_vol=0.0639); 215 | return image.select([]).addBands([blue, green, red, nir, swir1, swir2]); 216 | } 217 | function _correct_band(image, band_name, kvol, kvol0, f_iso, f_geo, f_vol){ 218 | //"""fiso + fvol * kvol + fgeo * kgeo""" 219 | var iso = ee.Image(f_iso); 220 | var geo = ee.Image(f_geo); 221 | var vol = ee.Image(f_vol); 222 | var pred = vol.multiply(kvol).add(geo.multiply(kvol)).add(iso).rename(['pred']); 223 | var pred0 = vol.multiply(kvol0).add(geo.multiply(kvol0)).add(iso).rename(['pred0']); 224 | var cfac = pred0.divide(pred).rename(['cfac']); 225 | var corr = image.select(band_name).multiply(cfac).rename([band_name]); 226 | return corr; 227 | } 228 | function _kvol(sunAz, sunZen, viewAz, viewZen){ 229 | //"""Calculate kvol kernel. 230 | //From Lucht et al. 2000 231 | //Phase angle = cos(solar zenith) cos(view zenith) + sin(solar zenith) sin(view zenith) cos(relative azimuth)""" 232 | 233 | var relative_azimuth = sunAz.subtract(viewAz).rename(['relAz']); 234 | var pa1 = viewZen.cos().multiply(sunZen.cos()); 235 | var pa2 = viewZen.sin().multiply(sunZen.sin()).multiply(relative_azimuth.cos()); 236 | var phase_angle1 = pa1.add(pa2); 237 | var phase_angle = phase_angle1.acos(); 238 | var p1 = ee.Image(PI.divide(2)).subtract(phase_angle); 239 | var p2 = p1.multiply(phase_angle1); 240 | var p3 = p2.add(phase_angle.sin()); 241 | var p4 = sunZen.cos().add(viewZen.cos()); 242 | var p5 = ee.Image(PI.divide(4)); 243 | 244 | var kvol = p3.divide(p4).subtract(p5).rename(['kvol']); 245 | 246 | var viewZen0 = ee.Image(0); 247 | var pa10 = viewZen0.cos().multiply(sunZen.cos()); 248 | var pa20 = viewZen0.sin().multiply(sunZen.sin()).multiply(relative_azimuth.cos()); 249 | var phase_angle10 = pa10.add(pa20); 250 | var phase_angle0 = phase_angle10.acos(); 251 | var p10 = ee.Image(PI.divide(2)).subtract(phase_angle0); 252 | var p20 = p10.multiply(phase_angle10); 253 | var p30 = p20.add(phase_angle0.sin()); 254 | var p40 = sunZen.cos().add(viewZen0.cos()); 255 | var p50 = ee.Image(PI.divide(4)); 256 | 257 | var kvol0 = p30.divide(p40).subtract(p50).rename(['kvol0']); 258 | 259 | return [kvol, kvol0]} 260 | function line_from_coords(coordinates, fromIndex, toIndex){ 261 | return ee.Geometry.LineString(ee.List([ 262 | coordinates.get(fromIndex), 263 | coordinates.get(toIndex)])); 264 | } 265 | function where(condition, trueValue, falseValue){ 266 | var trueMasked = trueValue.mask(condition); 267 | var falseMasked = falseValue.mask(invertMask(condition)); 268 | return trueMasked.unmask(falseMasked); 269 | } 270 | function invertMask(mask){ 271 | return mask.multiply(-1).add(1); 272 | } 273 | function value(list,index){ 274 | return ee.Number(list.get(index)); 275 | } 276 | 277 | 278 | /////Topographic correction//// 279 | //Source: https://doi.org/10.3390/rs11070831 280 | function illuminationCondition(img){ 281 | 282 | // Extract image metadata about solar position 283 | var SZ_rad = ee.Image.constant(ee.Number(img.get('SOLAR_ZENITH_ANGLE'))).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 284 | var SA_rad = ee.Image.constant(ee.Number(img.get('SOLAR_AZIMUTH_ANGLE')).multiply(3.14159265359).divide(180)).clip(img.geometry().buffer(10000)); 285 | // Creat terrain layers 286 | var slp = ee.Terrain.slope(dem).clip(img.geometry().buffer(10000)); 287 | var slp_rad = ee.Terrain.slope(dem).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 288 | var asp_rad = ee.Terrain.aspect(dem).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 289 | 290 | // Calculate the Illumination Condition (IC) 291 | // slope part of the illumination condition 292 | var cosZ = SZ_rad.cos(); 293 | var cosS = slp_rad.cos(); 294 | var slope_illumination = cosS.expression("cosZ * cosS", 295 | {'cosZ': cosZ, 296 | 'cosS': cosS.select('slope')}); 297 | // aspect part of the illumination condition 298 | var sinZ = SZ_rad.sin(); 299 | var sinS = slp_rad.sin(); 300 | var cosAziDiff = (SA_rad.subtract(asp_rad)).cos(); 301 | var aspect_illumination = sinZ.expression("sinZ * sinS * cosAziDiff", 302 | {'sinZ': sinZ, 303 | 'sinS': sinS, 304 | 'cosAziDiff': cosAziDiff}); 305 | // full illumination condition (IC) 306 | var ic = slope_illumination.add(aspect_illumination); 307 | 308 | // Add IC to original image 309 | var img_plus_ic = ee.Image(img.addBands(ic.rename('IC')).addBands(cosZ.rename('cosZ')).addBands(cosS.rename('cosS')).addBands(slp.rename('slope'))); 310 | return img_plus_ic; 311 | } 312 | function illuminationCorrection(img){ 313 | var props = img.toDictionary(); 314 | var st = img.get('system:time_start'); 315 | 316 | var img_plus_ic = img; 317 | var mask1 = img_plus_ic.select('nir').gt(-0.1); 318 | var mask2 = img_plus_ic.select('slope').gte(5) 319 | .and(img_plus_ic.select('IC').gte(0)) 320 | .and(img_plus_ic.select('nir').gt(-0.1)); 321 | var img_plus_ic_mask2 = ee.Image(img_plus_ic.updateMask(mask2)); 322 | 323 | // Specify Bands to topographically correct 324 | var bandList = ['blue','green','red','nir','swir1','swir2']; 325 | var compositeBands = img.bandNames(); 326 | var nonCorrectBands = img.select(compositeBands.removeAll(bandList)); 327 | 328 | var geom = ee.Geometry(img.get('system:footprint')).bounds().buffer(10000); 329 | 330 | function apply_SCSccorr(band){ 331 | var method = 'SCSc'; 332 | var out = img_plus_ic_mask2.select('IC', band).reduceRegion({ 333 | reducer: ee.Reducer.linearFit(), // Compute coefficients: a(slope), b(offset), c(b/a) 334 | geometry: ee.Geometry(img.geometry().buffer(-100)), // trim off the outer edges of the image for linear relationship 335 | scale: 10, 336 | maxPixels: 1000000000 337 | }); 338 | 339 | if (out === null || out === undefined ){ 340 | return img_plus_ic_mask2.select(band); 341 | } 342 | 343 | else{ 344 | var out_a = ee.Number(out.get('scale')); 345 | var out_b = ee.Number(out.get('offset')); 346 | var out_c = out_b.divide(out_a); 347 | // Apply the SCSc correction 348 | var SCSc_output = img_plus_ic_mask2.expression( 349 | "((image * (cosB * cosZ + cvalue)) / (ic + cvalue))", { 350 | 'image': img_plus_ic_mask2.select(band), 351 | 'ic': img_plus_ic_mask2.select('IC'), 352 | 'cosB': img_plus_ic_mask2.select('cosS'), 353 | 'cosZ': img_plus_ic_mask2.select('cosZ'), 354 | 'cvalue': out_c 355 | }); 356 | 357 | return SCSc_output; 358 | } 359 | 360 | } 361 | 362 | var img_SCSccorr = ee.Image(bandList.map(apply_SCSccorr)).addBands(img_plus_ic.select('IC')); 363 | var bandList_IC = ee.List([bandList, 'IC']).flatten(); 364 | img_SCSccorr = img_SCSccorr.unmask(img_plus_ic.select(bandList_IC)).select(bandList); 365 | 366 | return img_SCSccorr.addBands(nonCorrectBands) 367 | .setMulti(props) 368 | .set('system:time_start',st); 369 | } 370 | function illuminationConditionS2(img){ 371 | 372 | // Extract image metadata about solar position 373 | var SZ_rad = ee.Image.constant(ee.Number(img.get('MEAN_SOLAR_ZENITH_ANGLE'))).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 374 | var SA_rad = ee.Image.constant(ee.Number(img.get('MEAN_SOLAR_AZIMUTH_ANGLE')).multiply(3.14159265359).divide(180)).clip(img.geometry().buffer(10000)); 375 | // Creat terrain layers 376 | var slp = ee.Terrain.slope(dem).clip(img.geometry().buffer(10000)); 377 | var slp_rad = ee.Terrain.slope(dem).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 378 | var asp_rad = ee.Terrain.aspect(dem).multiply(3.14159265359).divide(180).clip(img.geometry().buffer(10000)); 379 | 380 | // Calculate the Illumination Condition (IC) 381 | // slope part of the illumination condition 382 | var cosZ = SZ_rad.cos(); 383 | var cosS = slp_rad.cos(); 384 | var slope_illumination = cosS.expression("cosZ * cosS", 385 | {'cosZ': cosZ, 386 | 'cosS': cosS.select('slope')}); 387 | // aspect part of the illumination condition 388 | var sinZ = SZ_rad.sin(); 389 | var sinS = slp_rad.sin(); 390 | var cosAziDiff = (SA_rad.subtract(asp_rad)).cos(); 391 | var aspect_illumination = sinZ.expression("sinZ * sinS * cosAziDiff", 392 | {'sinZ': sinZ, 393 | 'sinS': sinS, 394 | 'cosAziDiff': cosAziDiff}); 395 | // full illumination condition (IC) 396 | var ic = slope_illumination.add(aspect_illumination); 397 | 398 | // Add IC to original image 399 | var img_plus_ic = ee.Image(img.addBands(ic.rename('IC')).addBands(cosZ.rename('cosZ')).addBands(cosS.rename('cosS')).addBands(slp.rename('slope'))); 400 | return img_plus_ic; 401 | } 402 | 403 | 404 | ``` 405 | 406 | ## Visualization and Checking 407 | ``` 408 | Map.centerObject(imgS2_Topo_Corr_noBRDF,10) 409 | var viz ={min:0,max:3500,bands:['red','green','blue']} 410 | Map.addLayer(imgL8SR,viz,'L8 Original') 411 | Map.addLayer(imgL8_SR_BRDF,viz,'L8 BRDF') 412 | Map.addLayer(ee.Image(imgL8_Topo_Corr),viz,'L8 Topo corrected') 413 | ``` 414 | 415 | ## References 416 | 1. Vanonckelen, S., Lhermitte, S., Van Rompaey, A., 2013. The effect of atmospheric and topographic correction methods on land cover classification accuracy. International Journal of Applied Earth Observation and Geoinformation 24, 9–21. https://doi.org/10.1016/j.jag.2013.02.003 417 | 2. Poortinga, A., Tenneson, K., Shapiro, A., Nquyen, Q., San Aung, K., Chishtie, F., Saah, D., 2019. Mapping Plantations in Myanmar by Fusing Landsat-8, Sentinel-2 and Sentinel-1 Data along with Systematic Error Quantification. Remote Sensing 11, 831. https://doi.org/10.3390/rs11070831 418 | 3. Soenen, S.A., Peddle, D.R., Coburn, C.A., 2005. SCS+C: a modified Sun-canopy-sensor topographic correction in forested terrain. IEEE Transactions on Geoscience and Remote Sensing 43, 2148–2159. https://doi.org/10.1109/tgrs.2005.852480 419 | 4. Roy, D.P., Li, J., Zhang, H.K., Yan, L., Huang, H., Li, Z., 2017. Examination of Sentinel-2A multi-spectral instrument (MSI) reflectance anisotropy and the suitability of a general method to normalize MSI reflectance to nadir BRDF adjusted reflectance. Remote Sensing of Environment 199, 25–38. https://doi.org/10.1016/j.rse.2017.06.019 420 | 5. Claverie, M., Ju, J., Masek, J.G., Dungan, J.L., Vermote, E.F., Roger, J.-C., Skakun, S.V., Justice, C., 2018. The Harmonized Landsat and Sentinel-2 surface reflectance data set. Remote Sensing of Environment 219, 145–161. https://doi.org/10.1016/j.rse.2018.09.002 421 | 6. Roy, D.P., Zhang, H.K., Ju, J., Gomez-Dans, J.L., Lewis, P.E., Schaaf, C.B., Sun, Q., Li, J., Huang, H., Kovalskyy, V., 2016. A general method to normalize Landsat reflectance data to nadir BRDF adjusted reflectance. Remote Sensing of Environment 176, 255–271. https://doi.org/10.1016/j.rse.2016.01.023 422 | 7. Zhang, H.K., Roy, D.P., Yan, L., Li, Z., Huang, H., Vermote, E., Skakun, S., Roger, J.-C., 2018. Characterization of Sentinel-2A and Landsat-8 top of atmosphere, surface, and nadir BRDF adjusted reflectance and NDVI differences. Remote Sensing of Environment 215, 482–494. https://doi.org/10.1016/j.rse.2018.04.031 423 | -------------------------------------------------------------------------------- /05.brdf_correction.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # xx. Name of Task 3 | 4 | ## Objective 5 | - 6 | - 7 | ## General Instruction 8 | - 9 | - 10 | ## Core script 11 | - 12 | ``` 13 | ``` 14 | 15 | ## Visualization and Checking 16 | 17 | ## References 18 | 1. 19 | 2. 20 | 3. 21 | -------------------------------------------------------------------------------- /07.reprojection.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 07. Re-projection and Re-sampling Images 3 | 4 | ## Objective 5 | - A harmonized dataset should has uniform spatial resolution and projection across all bands 6 | - All shared bands between Landsat and Sentinel2 (blue,green,red,nir,swir1,swir2) are transformed to 30m, WGS84 UTM 7 | 8 | ## General Infomation 9 | - To resample pixel resolution, bicubic interpolation was chosen over bilinear or nearest-neighbor interpolation because it gives a smoother surface ([R. Keys, 1981](https://doi.org/10.1109/TASSP.1981.1163711)). 10 | - It is quite easy to do resampling and reprojection in GEE ([GEE documentation](https://developers.google.com/earth-engine/projections)). 11 | ## Core script 12 | 13 | ``` 14 | var bound = ee.Geometry.Polygon([ 15 | [[35.96060746534647,33.815740435596645], [36.00472443922342,33.815740435596645], [36.00472443922342,33.85366939280121], [35.96060746534647,33.85366939280121], [35.96060746534647,33.815740435596645]] 16 | ]); 17 | 18 | var bandIn = ['B2','B3','B4','B5','B6','B7']; 19 | var bandOut = ['blue','green','red','nir','swir1','swir2']; 20 | 21 | //Example images of S2 and L8 22 | var imgS2SR = ee.Image('COPERNICUS/S2_SR/20181107T082129_20181107T082732_T36SYC').select(bandIn,bandOut).clip(bound); 23 | var imgL8SR = ee.Image('LANDSAT/LC08/C01/T1_SR/LC08_174036_20181107').select(bandIn,bandOut).clip(bound); 24 | 25 | //check projection of B4(red) bands 26 | print('S2 CRS:', imgS2SR.select('red').projection().crs()); 27 | print('L8 CRS:', imgL8SR.select('red').projection().crs()); 28 | 29 | // Reproject and rescale L8 image according to S2. Keep the system:time_start in metadata 30 | var L8SR_10m = imgL8SR.resample('bicubic').reproject({ 31 | crs: imgS2SR.select('red').projection().crs(), 32 | scale: 10 33 | }).set('system:time_start',imgL8SR.date()); 34 | 35 | ``` 36 | 37 | ## Visualization and Checking 38 | ``` 39 | Map.centerObject(bound,12) 40 | var visParams = {bands: ['red','green','blue'], min:0, max: 3500}; 41 | Map.addLayer(imgL8SR, visParams, 'imgL8SR 30m'); 42 | Map.addLayer(imgS2SR, visParams, 'imgS2SR 10m'); 43 | Map.addLayer(L8SR_10m, visParams, 'l8SR 10m'); 44 | 45 | ``` 46 | - L8 Original (30m). 47 | 48 | ![30mL8](https://user-images.githubusercontent.com/40456844/62151792-586cc600-b32b-11e9-91f6-12c4ce8a374d.png). 49 | 50 | - L8 rescale to 10m (bicubic). 51 | 52 | ![rescale](https://user-images.githubusercontent.com/40456844/62151685-1b083880-b32b-11e9-98b9-1606ca17238f.png). 53 | ## References 54 | 1. [Cubic convolution interpolation for digital image processing](https://doi.org/10.1109/TASSP.1981.1163711) (R. Keys 1981) 55 | 2. [Projections in Google Earth Engine](https://developers.google.com/earth-engine/projections) (GEE Documentation) 56 | -------------------------------------------------------------------------------- /08.image_registration.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 08. L8-S2 Co-Registration (Re-alignment) 3 | 4 | ## Objective 5 | - Find the miss-aligment between L8-S2 6 | - Re-align L8 according to S2 7 | - Plot the offset difference before and after 8 | ## General Information 9 | - Miss alignment (or miss registration) between L8 and S2 images varied with location and can exceed one Landsat pixel (>30 meters) ([Storey et al., 2016](https://doi.org/10.1016/j.rse.2016.08.025)). It is due to the residual geolocation errors in Landsat-8 framework which based upon the Global Land Survey images. 10 | - In general, S2 has better absolute geodetic accuracy than L8 (Storey et al.,2016) therefore we are going to re-align Landsat images according to a reference S2 image ([Gao et al., 2009](https://doi.org/10.1117/1.3104620)) 11 | - In GEE, we are going to find a pair of L8-S2 which were captured at the same time over the same location. Then ``` displacement() ``` is used to calculate the displacement vector (X and Y) at each pixel between the two images. Then ```displace() ``` will be used to displace or wrap the L8 image aligned with the base S2 image ([GEE Documenation](https://developers.google.com/earth-engine/register)). 12 | - The re-alignment described here is purely image processing technique. It is differed from geo-referencing or geo-correcting which involves aligning images to the correct geographic location through ground control points. 13 | ## Core script 14 | ``` 15 | var bound = ee.Geometry.Polygon([ 16 | [[35.96060746534647,33.815740435596645], [36.00472443922342,33.815740435596645], [36.00472443922342,33.85366939280121], [35.96060746534647,33.85366939280121], [35.96060746534647,33.815740435596645]] 17 | ]); 18 | 19 | var bandIn = ['B2','B3','B4','B5','B6','B7']; 20 | var bandOut = ['blue','green','red','nir','swir1','swir2']; 21 | 22 | //Example images of S2 and L8 23 | var imgS2SR = ee.Image('COPERNICUS/S2_SR/20181107T082129_20181107T082732_T36SYC').select(bandIn,bandOut).clip(bound); 24 | var imgL8SR = ee.Image('LANDSAT/LC08/C01/T1_SR/LC08_174036_20181107').select(bandIn,bandOut).clip(bound); 25 | 26 | // Choose to register using only the 'Red' band. 27 | var L8RedBand = imgL8SR.select('red'); 28 | var S2RedBand = imgS2SR.select('red'); 29 | 30 | // Determine the displacement by matching only the 'Red' bands. 31 | var displacement = L8RedBand.displacement({ 32 | referenceImage: S2RedBand, 33 | maxOffset: 50.0,//The maximum offset allowed when attempting to align the input images, in meters 34 | patchWidth: 100.0 // Small enough to capture texture and large enough that ignorable 35 | //objects are small within the patch. Automatically ditermined if not provided 36 | }); 37 | 38 | //wrap the imgL8SR image 39 | var l8SR_aligned = imgL8SR.displace(displacement); 40 | 41 | ``` 42 | 43 | ## Visualization and Checking 44 | - Visualizing on the map 45 | ``` 46 | Map.centerObject(bound,12) 47 | var visParams = {bands: ['red','green','blue'], min:0, max: 3500}; 48 | Map.addLayer(imgL8SR, visParams, 'imgL8SR 30m'); 49 | Map.addLayer(imgS2SR, visParams, 'imgS2SR 10m'); 50 | Map.addLayer(l8SR_aligned, visParams, 'l8SR 10m'); 51 | ``` 52 | - Histogram of the offset difference before the alignment 53 | ``` 54 | // Compute image offset and direction. 55 | var offset = displacement.select('dx').hypot(displacement.select('dy'));//calculates the magnitude of the 2D vector [x, y] 56 | var angle = displacement.select('dx').atan2(displacement.select('dy'));// calculates the angle formed by the 2D vector [x, y]. 57 | //histogram of offset and agle 58 | var histogramOffset = ui.Chart.image.histogram(offset, bound, 10).setOptions({title: 'Offset Differences'}); 59 | var histogramAngle = ui.Chart.image.histogram(angle, bound, 10).setOptions({title: 'Angle Differences'}); 60 | print(histogramOffset,'Offset Differences Before') 61 | ``` 62 | - Histogram of the offset difference after the alignment 63 | ``` 64 | // Determine the displacement after registration (Red band) 65 | var L8RedBand_aligned = l8SR_aligned.select('red'); 66 | var displacement = L8RedBand_aligned.displacement({ 67 | referenceImage: S2RedBand, 68 | maxOffset: 50.0, 69 | patchWidth: 100.0 70 | }); 71 | var offset_after = displacement.select('dx').hypot(displacement.select('dy')); 72 | var angle_after = displacement.select('dx').atan2(displacement.select('dy')); 73 | var histogramOffset_after = ui.Chart.image.histogram(offset_after, bound, 10).setOptions({title: 'Offset Differences'}); 74 | var histogramAngle_after = ui.Chart.image.histogram(angle_after, bound, 10).setOptions({title: 'Angle Differences'}); 75 | 76 | print(histogramOffset_after,'Offset Differences After'); 77 | ``` 78 | ## References 79 | 1. [Registering Images in Google Earth Engine](https://developers.google.com/earth-engine/register) 80 | 2. [Masek, J., 2009. Automated registration and orthorectification package for Landsat and Landsat-like data processing. Journal of Applied Remote Sensing 3, 33515. https://doi.org/10.1117/1.3104620](https://doi.org/10.1117/1.3104620) 81 | 3. [Storey, J., Roy, D.P., Masek, J., Gascon, F., Dwyer, J., Choate, M., 2016. A note on the temporary misregistration of Landsat-8 Operational Land Imager (OLI) and Sentinel-2 Multi Spectral Instrument (MSI) imagery. Remote Sensing of Environment 186, 121–122. https://doi.org/10.1016/j.rse.2016.08.025](https://doi.org/10.1016/j.rse.2016.08.025) 82 | -------------------------------------------------------------------------------- /09.band_adjustment.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 09. Band Adjustment 3 | 4 | ## Objective 5 | 6 | ## General Instruction 7 | - Both the Landsat and Sentinel-2 calibration team have been putting effort on radiometric and geometric calibration so that their bands 8 | are compatible ([Barsi et al., 2017](https://doi.org/10.1080/22797254.2018.1507613)). However small adjustment is still needed ([Barsi et al., 2017](https://doi.org/10.1080/22797254.2018.1507613), [Chastain et al., 2019](https://doi.org/10.1016/j.rse.2018.11.012), [Claverie el al., 2018](https://doi.org/10.1016/j.rse.2018.09.002)). 9 | - Transformation coefficients based on ([Chastain et al., 2019](https://doi.org/10.1016/j.rse.2018.11.012)) 10 | 11 | ## Core script 12 | ``` 13 | var bound = ee.Geometry.Polygon([ 14 | [[35.96060746534647,33.815740435596645], [36.00472443922342,33.815740435596645], [36.00472443922342,33.85366939280121], [35.96060746534647,33.85366939280121], [35.96060746534647,33.815740435596645]] 15 | ]); 16 | var bandIn = ['B2','B3','B4','B5','B6','B7']; 17 | var bandOut = ['blue','green','red','nir','swir1','swir2']; 18 | //Example images of S2 and L8 19 | var imgS2SR = ee.Image('COPERNICUS/S2_SR/20181107T082129_20181107T082732_T36SYC').select(bandIn,bandOut).clip(bound); 20 | var imgL8SR = ee.Image('LANDSAT/LC08/C01/T1_SR/LC08_174036_20181107').select(bandIn,bandOut).clip(bound); 21 | 22 | var interceptsL8 = [-0.0107,0.0026,-0.0015,0.0033,0.0065,0.0046]; 23 | var slopesL8 = [1.0946,1.0043,1.0524,0.8954,1.0049,1.0002]; 24 | 25 | var imgL8SR_bandadj = ee.Image(imgL8SR.multiply(slopesL8).add(interceptsL8).float().copyProperties(imgL8SR)).set('system:time_start',imgL8SR.get('system:time_start')) 26 | ``` 27 | 28 | ## Visualization and Checking 29 | ``` 30 | print(imgL8SR_bandadj) 31 | Map.centerObject(bound,12) 32 | var visParams = {bands: ['red','green','blue'], min:0, max: 3500}; 33 | Map.addLayer(imgL8SR, visParams, 'imgL8SR 30m'); 34 | Map.addLayer(imgS2SR, visParams, 'imgS2SR 10m'); 35 | Map.addLayer(imgL8SR_bandadj, visParams, 'imgL8SR_bandadj'); 36 | ``` 37 | - Before Band Adjustment. 38 | 39 | ![beforeBandadj](https://user-images.githubusercontent.com/40456844/62152098-05474300-b32c-11e9-9235-84711410b9a9.png). 40 | 41 | - After Band Adjustment. 42 | 43 | ![afterBandadj](https://user-images.githubusercontent.com/40456844/62152117-1001d800-b32c-11e9-80b5-e84734def447.png). 44 | 45 | 46 | ## References 47 | 1. Barsi, J.A., Alhammoud, B., Czapla-Myers, J., Gascon, F., Haque, M.O., Kaewmanee, M., Leigh, L., Markham, B.L., 2018. Sentinel-2A MSI and Landsat-8 OLI radiometric cross comparison over desert sites. European Journal of Remote Sensing 51, 822–837. https://doi.org/10.1080/22797254.2018.1507613 48 | 2. Chastain, R., Housman, I., Goldstein, J., Finco, M., Tenneson, K., 2019. Empirical cross sensor comparison of Sentinel-2A and 2B MSI, Landsat-8 OLI, and Landsat-7 ETM+ top of atmosphere spectral characteristics over the conterminous United States. Remote Sensing of Environment 221, 274–285. https://doi.org/10.1016/j.rse.2018.11.012 49 | 3. Claverie, M., Ju, J., Masek, J.G., Dungan, J.L., Vermote, E.F., Roger, J.-C., Skakun, S.V., Justice, C., 2018. The Harmonized Landsat and Sentinel-2 surface reflectance data set. Remote Sensing of Environment 219, 145–161. https://doi.org/10.1016/j.rse.2018.09.002 50 | -------------------------------------------------------------------------------- /10.time_series.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 10. Images Compositing and Time series 3 | https://code.earthengine.google.com/0aab36f0d920a0e9d294f72d9c6012e6 4 | ## Objective 5 | - Yearly, seasonal, monthly, decadal time series (NDVI) 6 | ## General Information 7 | 8 | ## Core script 9 | - Define initial conditions 10 | 11 | ``` 12 | var hls_col = ee.ImageCollection('users/ndminhhus/eLEAF/nt/hls_v01').map(ndviF); 13 | var paddy_nt = ee.Image('users/ndminhhus/NTmask/paddy'); 14 | var geometry = ee.Geometry.Point([108.94553918036411,11.566755394830713]) 15 | //function ndvi 16 | function ndviF(img){ 17 | var ndvi = img.normalizedDifference(['nir','red']).rename('NDVI'); 18 | return img.addBands(ndvi) 19 | } 20 | ``` 21 | - Seasonal Composite 22 | ``` 23 | var years = ee.List.sequence(2018, 2019) 24 | var seasonalNDVI = ee.ImageCollection.fromImages( 25 | years.map(function (y) { 26 | var w = hls_col.filter(ee.Filter.calendarRange(y, y, 'year')) 27 | //spring crop (Jan -> Apr), summer crop (May to Aug), rainy season (Sep to Dec) 28 | .filter(ee.Filter.calendarRange(1, 4, 'month')) 29 | // .filter(ee.Filter.calendarRange(5, 8, 'month')) 30 | // .filter(ee.Filter.calendarRange(9, 12, 'month')) 31 | .mean(); 32 | return w.set('year', y) 33 | .set('system:time_start',ee.Date.fromYMD(y,1,1)); 34 | }).flatten()); 35 | print(seasonalNDVI) 36 | ``` 37 | - Monthly Composite 38 | ``` 39 | var months = ee.List.sequence(1,12); 40 | var monthlyNDVI = ee.ImageCollection.fromImages( 41 | months.map(function(m){ 42 | var w = hls_col.select('NDVI').filter(ee.Filter.calendarRange(m, m, 'month')) 43 | .mean(); 44 | return w.set('year', 2018) 45 | .set('month', m) 46 | .set('date', ee.Date.fromYMD(2018,m,1)) 47 | .set('system:time_start',ee.Date.fromYMD(2018,m,1)) 48 | 49 | }).flatten()); 50 | print(monthlyNDVI) 51 | ``` 52 | - Decadal Composite 53 | ``` 54 | var start = '2018-01-01'; 55 | var end = '2018-12-31'; 56 | var interval = 10; 57 | var increment = 'day'; 58 | var start = start; 59 | // make a list 60 | var startDate = ee.Date(start); 61 | var secondDate = startDate.advance(interval, increment).millis(); 62 | var increase = secondDate.subtract(startDate.millis()); 63 | var list = ee.List.sequence(startDate.millis(), ee.Date(end).millis(), increase); 64 | var tenDays = ee.ImageCollection.fromImages(list.map(function(date){ 65 | return hls_col.select('NDVI') 66 | .filterDate(ee.Date(date), ee.Date(date).advance(interval, increment)) 67 | //.min() //take the darkest pixel from L7, L8 or S2. reduce cloud contamination 68 | .mean() 69 | .set('system:time_start',ee.Date(date).millis()); 70 | })); 71 | print(tenDays,'tenDays'); 72 | ``` 73 | 74 | 75 | ## Visualization and Checking 76 | - Time Series 77 | ``` 78 | // Predefine the chart titles. 79 | var title = { 80 | title: ' NDVI Series ', 81 | hAxis: {title: 'Date'},max:1, 82 | vAxis: {title: 'NDVI'}, 83 | series: {0:{color:'red'},1:{color:'blue'}}, 84 | }; 85 | 86 | var chart_ndvi_season = ui.Chart.image.series(seasonalNDVI.select('NDVI'), geometry, ee.Reducer.mean(), 30).setOptions(title).setChartType('ScatterChart'); 87 | print(chart_ndvi_season) 88 | 89 | var chart_ndvi_10days = ui.Chart.image.series(tenDays.select('NDVI'), geometry, ee.Reducer.mean(), 30).setOptions(title).setChartType('ScatterChart'); 90 | print(chart_ndvi_10days) 91 | 92 | ``` 93 | - Decadal (10 days) composite images timeseries 94 | ![decadal](https://user-images.githubusercontent.com/40456844/62111875-bffb2500-b2db-11e9-8a9a-d0bec9714b65.png) 95 | ## References 96 | 97 | -------------------------------------------------------------------------------- /11.crop_mapping.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | # 11. Crop Mapping (Level 3 Land Cover) 3 | 4 | ## Objective 5 | - 6 | - 7 | 8 | ## General Instruction 9 | - Temporal signature 10 | - Spectral signature 11 | 12 | ## Core script 13 | ``` 14 | 15 | ``` 16 | 17 | ## Visualization and Checking 18 | ``` 19 | 20 | ``` 21 | 22 | ## References 23 | 1. 24 | 2. 25 | 3. 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [geeguide](/README.md) 2 | 3 | # Harmonization of Landsat and Sentinel 2 in Google Earth Engine, documentation and scripts 4 | 5 | The objective of this work is to document the technical part of my master thesis, named: "Harmonization of Landsat and Sentinel 2 for Crop Monitoring, a complete stream processing in Google Earth Engine". A complete stream processing entirely in Google Earth Engine is developed to generate seamless surface reflectances of harmonized L7,L8 and Sentinel2. 6 | 7 | To inspect the result of the hamonized dataset via NDVI time series and the cropland classification in Ninh Thuan, Vietnam please go to this [GEE App](https://ndminhhus.users.earthengine.app/view/cropninhthuan2019) 8 | 9 | I am glad to announce that this work has been peer reviewed and now published on the Journal of Remote sensing: [Minh et al., 2020](https://doi.org/10.3390/rs12020281). We were also invited to speak about this work at the 6 th International Conference on Satellite & Space Missions on July 15-16, 2020 in London, UK. [More information](https://satellite.insightconferences.com) 10 | 11 | I will put here many scripts that I have used, either collected from various sources or the ones I developed myself. Citation is considered seriously. The remote sensing content also will be discussed. 12 | 13 | The scripts are organized not in order of increasing complexity but according to the processing workflow. Also, the scripts can be used separately or in combination depending on user-specific applications. Each session has five parts including Objective, General Instruction, Core Script, Visualisation Checking, and References 14 | 15 | Some experiences with GEE are needed. General introduction about GEE and how to use it can be found in many other places. 16 | 17 | Table of Contents 18 | Part 1: Preprocessing and Data analysis. 19 | 1. [Filtering Image Collection](01.Filtering-Image-Collection.md) 20 | 2. [Atmospheric correction](02.Atm-correction.md) 21 | 3. [Cloud masking](03.cloudmaskTOA.md) 22 | 3A.[Cloud masking improved](https://medium.com/eelab/improve-cloud-detection-and-removal-with-machine-learning-in-google-earth-engine-ac0a2f759022) 23 | 4. [Shadow masking](03.cloudmaskTOA.md) 24 | 5. [Topographic correction](04.topo_correction.md) 25 | 6. [BRDF correction](04.topo_correction.md) 26 | 7. [Reprojection and resampling](07.reprojection.md) 27 | 8. [Image registration](08.image_registration.md) 28 | 9. [Band adjustment](09.band_adjustment.md) 29 | 10. [Composite time series](10.time_series.md) 30 | 11. Exporting data 31 | 12. Crop mapping with spectral signature & temporal signature 32 | 33 | ![fig3](https://user-images.githubusercontent.com/40456844/65582427-9ee74580-dfa7-11e9-9eae-0dd0a3cdc6bb.jpg) 34 | ![timeseries](https://user-images.githubusercontent.com/40456844/61792616-76af5f00-ae1d-11e9-8c08-2a43613724eb.png) 35 | 36 | 37 | -------------------------------------------------------------------------------- /jupyter_notebooks/02.Atm-corr-Landsat7.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 3, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import ee\n", 12 | "from Py6S import *\n", 13 | "import datetime\n", 14 | "import math\n", 15 | "import os\n", 16 | "import sys\n", 17 | "sys.path.append(os.path.join(os.path.dirname(os.getcwd()),'bin'))\n", 18 | "from atmospheric import Atmospheric\n", 19 | "\n", 20 | "ee.Initialize()" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 28, 26 | "metadata": { 27 | "collapsed": true 28 | }, 29 | "outputs": [], 30 | "source": [ 31 | "def func1(img):\n", 32 | " img = ee.Image(img)\n", 33 | " info = img.getInfo()['properties']\n", 34 | " scene_date = datetime.datetime.utcfromtimestamp(info['system:time_start']/1000)\n", 35 | " solar_elv = img.getInfo()['properties']['SUN_ELEVATION']\n", 36 | " solar_z = ee.Number(90).subtract(solar_elv).getInfo()\n", 37 | " h2o = Atmospheric.water(geom,img.date()).getInfo()\n", 38 | " o3 = Atmospheric.ozone(geom,img.date()).getInfo()\n", 39 | " aot = Atmospheric.aerosol(geom,img.date()).getInfo()\n", 40 | " SRTM = ee.Image('CGIAR/SRTM90_V4')# Shuttle Radar Topography mission covers *most* of the Earth\n", 41 | " alt = SRTM.reduceRegion(reducer = ee.Reducer.mean(),geometry = geom.centroid()).get('elevation').getInfo()\n", 42 | " km = alt/1000 # i.e. Py6S uses units of kilometers\n", 43 | " # Instantiate\n", 44 | " s = SixS()\n", 45 | " # Atmospheric constituents\n", 46 | " s.atmos_profile = AtmosProfile.UserWaterAndOzone(h2o,o3)\n", 47 | " s.aero_profile = AeroProfile.Continental\n", 48 | " s.aot550 = aot\n", 49 | " # Earth-Sun-satellite geometry\n", 50 | " s.geometry = Geometry.User()\n", 51 | " s.geometry.view_z = 0 # always NADIR (I think..)\n", 52 | " #s.geometry.solar_z = solar_z # solar zenith angle\n", 53 | " s.geometry.solar_z = solar_z # solar zenith angle\n", 54 | " s.geometry.month = scene_date.month # month and day used for Earth-Sun distance\n", 55 | " s.geometry.day = scene_date.day # month and day used for Earth-Sun distance\n", 56 | " s.altitudes.set_sensor_satellite_level()\n", 57 | " s.altitudes.set_target_custom_altitude(km)\n", 58 | " #Mission specific for atmospheric correction\n", 59 | " #https://github.com/samsammurphy/ee-atmcorr-timeseries/blob/master/atmcorr/mission_specifics.py\n", 60 | " def spectralResponseFunction(bandname): \n", 61 | " bandSelect = {\n", 62 | " 'B1':PredefinedWavelengths.LANDSAT_ETM_B1,\n", 63 | " 'B2':PredefinedWavelengths.LANDSAT_ETM_B2,\n", 64 | " 'B3':PredefinedWavelengths.LANDSAT_ETM_B3,\n", 65 | " 'B4':PredefinedWavelengths.LANDSAT_ETM_B4,\n", 66 | " 'B5':PredefinedWavelengths.LANDSAT_ETM_B5,\n", 67 | " 'B7':PredefinedWavelengths.LANDSAT_ETM_B7,\n", 68 | " }\n", 69 | " return Wavelength(bandSelect[bandname])\n", 70 | " def toa_to_rad(bandname):\n", 71 | " ESUN_L8 = [1895.33, 2004.57, 1820.75, 1549.49, 951.76, 247.55, 85.46, 1723.8, 366.97]\n", 72 | " ESUN_L7 = [1997, 1812, 1533, 1039, 230.8, 84.9] # PAN = 1362 (removed to match Py6S)\n", 73 | " ESUN_BAND = {\n", 74 | " 'B1':ESUN_L7[0],\n", 75 | " 'B2':ESUN_L7[1],\n", 76 | " 'B3':ESUN_L7[2],\n", 77 | " 'B4':ESUN_L7[3],\n", 78 | " 'B5':ESUN_L7[4], \n", 79 | " 'B7':ESUN_L7[5],\n", 80 | " }\n", 81 | " solar_angle_correction = math.cos(math.radians(solar_z))\n", 82 | " # Earth-Sun distance (from day of year)\n", 83 | " doy = scene_date.timetuple().tm_yday\n", 84 | " d = 1 - 0.01672 * math.cos(0.9856 * (doy-4))# http://physics.stackexchange.com/questions/177949/earth-sun-distance-on-a-given-day-of-the-year\n", 85 | " # conversion factor\n", 86 | " multiplier = ESUN_BAND[bandname]*solar_angle_correction/(math.pi*d**2)\n", 87 | " # at-sensor radiance\n", 88 | " rad = img.select(bandname).multiply(multiplier)\n", 89 | " return rad\n", 90 | " def surface_reflectance(bandname):\n", 91 | " # run 6S for this waveband\n", 92 | " s.wavelength = spectralResponseFunction(bandname)\n", 93 | " s.run()\n", 94 | "\n", 95 | " # extract 6S outputs\n", 96 | " Edir = s.outputs.direct_solar_irradiance #direct solar irradiance\n", 97 | " Edif = s.outputs.diffuse_solar_irradiance #diffuse solar irradiance\n", 98 | " Lp = s.outputs.atmospheric_intrinsic_radiance #path radiance\n", 99 | " absorb = s.outputs.trans['global_gas'].upward #absorption transmissivity\n", 100 | " scatter = s.outputs.trans['total_scattering'].upward #scattering transmissivity\n", 101 | " tau2 = absorb*scatter #total transmissivity\n", 102 | "\n", 103 | " # radiance to surface reflectance\n", 104 | " rad = toa_to_rad(bandname)\n", 105 | " ref = rad.subtract(Lp).multiply(math.pi).divide(tau2*(Edir+Edif))\n", 106 | " return ref\n", 107 | " \n", 108 | " \n", 109 | " blue = surface_reflectance('B1')\n", 110 | " green = surface_reflectance('B2')\n", 111 | " red = surface_reflectance('B3')\n", 112 | " nir = surface_reflectance('B4')\n", 113 | " swir1 = surface_reflectance('B5')\n", 114 | " swir2 = surface_reflectance('B7')\n", 115 | " \n", 116 | " ref = img.select('BQA').addBands(blue).addBands(green).addBands(red).addBands(nir).addBands(swir1).addBands(swir2)\n", 117 | " \n", 118 | " dateString = scene_date.strftime(\"%Y-%m-%d\")\n", 119 | " ref = ref.copyProperties(img).set({ \n", 120 | " 'AC_date':dateString,\n", 121 | " 'AC_aerosol_optical_thickness':aot,\n", 122 | " 'AC_water_vapour':h2o,\n", 123 | " 'AC_version':'py6S',\n", 124 | " 'AC_contact':'ndminhhus@gmail.com',\n", 125 | " 'AC_ozone':o3})\n", 126 | " \n", 127 | "\n", 128 | " # define YOUR assetID \n", 129 | " # export\n", 130 | "\n", 131 | " fname = ee.String(img.get('system:index')).getInfo()\n", 132 | " export = ee.batch.Export.image.toAsset(\\\n", 133 | " image=ref,\n", 134 | " description= 'L7_BOA_'+fname,\n", 135 | " assetId = 'users/ndminhhus/eLEAF/nt/L7_py6S/'+'L7_BOA'+fname,\n", 136 | " region = region,\n", 137 | " scale = 30,\n", 138 | " maxPixels = 1e13)\n", 139 | "\n", 140 | " # # uncomment to run the export\n", 141 | " export.start()\n", 142 | " print('exporting ' +fname + '--->done')" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": 29, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [ 151 | "startDate = ee.Date('2018-01-01')\n", 152 | "endDate = ee.Date('2020-01-01')\n", 153 | "geom = ee.Geometry.Point(108.91220182000018,11.700863529688942)# Ninh Thuan region\n", 154 | "region = geom.buffer(60000).bounds().getInfo()['coordinates']" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 30, 160 | "metadata": { 161 | "collapsed": true 162 | }, 163 | "outputs": [], 164 | "source": [ 165 | "# The Landsat 8 image collection\n", 166 | "L7_col = ee.ImageCollection('LANDSAT/LE07/C01/T1_TOA').filterBounds(geom).filterDate(startDate,endDate)" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 31, 172 | "metadata": {}, 173 | "outputs": [ 174 | { 175 | "name": "stdout", 176 | "output_type": "stream", 177 | "text": [ 178 | "26\n" 179 | ] 180 | } 181 | ], 182 | "source": [ 183 | "print(L7_col.size().getInfo())" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": 25, 189 | "metadata": { 190 | "collapsed": true 191 | }, 192 | "outputs": [], 193 | "source": [ 194 | "L7_list = L7_col.toList(L7_col.size().getInfo())\n", 195 | "img1 = ee.Image(L7_list.get(9))" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 26, 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "toa = img1\n", 205 | "boa = ee.Image(func1(toa))" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 33, 211 | "metadata": {}, 212 | "outputs": [ 213 | { 214 | "name": "stdout", 215 | "output_type": "stream", 216 | "text": [ 217 | "2018-07-05\n" 218 | ] 219 | } 220 | ], 221 | "source": [ 222 | "print(boa.getInfo()['properties']['AC_date'])" 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": 106, 228 | "metadata": {}, 229 | "outputs": [ 230 | { 231 | "data": { 232 | "text/html": [ 233 | "" 234 | ], 235 | "text/plain": [ 236 | "" 237 | ] 238 | }, 239 | "metadata": {}, 240 | "output_type": "display_data" 241 | }, 242 | { 243 | "data": { 244 | "text/html": [ 245 | "" 246 | ], 247 | "text/plain": [ 248 | "" 249 | ] 250 | }, 251 | "metadata": {}, 252 | "output_type": "display_data" 253 | } 254 | ], 255 | "source": [ 256 | "from IPython.display import display, Image\n", 257 | "\n", 258 | "channels = ['B3','B2','B1']\n", 259 | "\n", 260 | "original = Image(url=toa.select(channels).getThumbUrl({\n", 261 | " 'region':region,\n", 262 | " 'min':0,\n", 263 | " 'max':0.25\n", 264 | " }))\n", 265 | "\n", 266 | "corrected = Image(url=ee.Image(boa).select(channels).getThumbUrl({\n", 267 | " 'region':region,\n", 268 | " 'min':0,\n", 269 | " 'max':0.25\n", 270 | " }))\n", 271 | "\n", 272 | "display(original, corrected)" 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": 32, 278 | "metadata": {}, 279 | "outputs": [ 280 | { 281 | "name": "stdout", 282 | "output_type": "stream", 283 | "text": [ 284 | "exporting LE07_123052_20180110--->done\n", 285 | "exporting LE07_123052_20180126--->done\n", 286 | "exporting LE07_123052_20180211--->done\n", 287 | "exporting LE07_123052_20180227--->done\n", 288 | "exporting LE07_123052_20180315--->done\n", 289 | "exporting LE07_123052_20180331--->done\n", 290 | "exporting LE07_123052_20180502--->done\n", 291 | "exporting LE07_123052_20180518--->done\n", 292 | "exporting LE07_123052_20180619--->done\n", 293 | "exporting LE07_123052_20180705--->done\n", 294 | "exporting LE07_123052_20180721--->done\n", 295 | "exporting LE07_123052_20180822--->done\n", 296 | "exporting LE07_123052_20180907--->done\n", 297 | "exporting LE07_123052_20180923--->done\n", 298 | "exporting LE07_123052_20181009--->done\n", 299 | "exporting LE07_123052_20181025--->done\n", 300 | "exporting LE07_123052_20181110--->done\n", 301 | "exporting LE07_123052_20181212--->done\n", 302 | "exporting LE07_123052_20190113--->done\n", 303 | "exporting LE07_123052_20190214--->done\n", 304 | "exporting LE07_123052_20190302--->done\n", 305 | "exporting LE07_123052_20190318--->done\n", 306 | "exporting LE07_123052_20190403--->done\n", 307 | "exporting LE07_123052_20190419--->done\n", 308 | "exporting LE07_123052_20190505--->done\n", 309 | "exporting LE07_123052_20190521--->done\n" 310 | ] 311 | } 312 | ], 313 | "source": [ 314 | "col_length = L7_col.size().getInfo()\n", 315 | "#print(col_length)\n", 316 | "\n", 317 | "for i in range(0,col_length):\n", 318 | " #print(i)\n", 319 | " list = L7_col.toList(col_length)\n", 320 | " img = ee.Image(list.get(i))\n", 321 | " func1(img)\n" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": null, 327 | "metadata": { 328 | "collapsed": true 329 | }, 330 | "outputs": [], 331 | "source": [] 332 | } 333 | ], 334 | "metadata": { 335 | "kernelspec": { 336 | "display_name": "Python 3", 337 | "language": "python", 338 | "name": "python3" 339 | }, 340 | "language_info": { 341 | "codemirror_mode": { 342 | "name": "ipython", 343 | "version": 3 344 | }, 345 | "file_extension": ".py", 346 | "mimetype": "text/x-python", 347 | "name": "python", 348 | "nbconvert_exporter": "python", 349 | "pygments_lexer": "ipython3", 350 | "version": "3.6.0" 351 | } 352 | }, 353 | "nbformat": 4, 354 | "nbformat_minor": 2 355 | } 356 | -------------------------------------------------------------------------------- /jupyter_notebooks/02.Atm-corr-Landsat8.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 114, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import ee\n", 12 | "from Py6S import *\n", 13 | "import datetime\n", 14 | "import math\n", 15 | "import os\n", 16 | "import sys\n", 17 | "sys.path.append(os.path.join(os.path.dirname(os.getcwd()),'bin'))\n", 18 | "from atmospheric import Atmospheric\n", 19 | "\n", 20 | "ee.Initialize()" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 116, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "def func1(img):\n", 30 | " img = ee.Image(img)\n", 31 | " info = img.getInfo()['properties']\n", 32 | " scene_date = datetime.datetime.utcfromtimestamp(info['system:time_start']/1000)\n", 33 | " solar_elv = img.getInfo()['properties']['SUN_ELEVATION']\n", 34 | " solar_z = ee.Number(90).subtract(solar_elv).getInfo()\n", 35 | " h2o = Atmospheric.water(geom,img.date()).getInfo()\n", 36 | " o3 = Atmospheric.ozone(geom,img.date()).getInfo()\n", 37 | " aot = Atmospheric.aerosol(geom,img.date()).getInfo()\n", 38 | " SRTM = ee.Image('CGIAR/SRTM90_V4')# Shuttle Radar Topography mission covers *most* of the Earth\n", 39 | " alt = SRTM.reduceRegion(reducer = ee.Reducer.mean(),geometry = geom.centroid()).get('elevation').getInfo()\n", 40 | " km = alt/1000 # i.e. Py6S uses units of kilometers\n", 41 | " # Instantiate\n", 42 | " s = SixS()\n", 43 | " # Atmospheric constituents\n", 44 | " s.atmos_profile = AtmosProfile.UserWaterAndOzone(h2o,o3)\n", 45 | " s.aero_profile = AeroProfile.Continental\n", 46 | " s.aot550 = aot\n", 47 | " # Earth-Sun-satellite geometry\n", 48 | " s.geometry = Geometry.User()\n", 49 | " s.geometry.view_z = 0 # always NADIR (I think..)\n", 50 | " #s.geometry.solar_z = solar_z # solar zenith angle\n", 51 | " s.geometry.solar_z = solar_z # solar zenith angle\n", 52 | " s.geometry.month = scene_date.month # month and day used for Earth-Sun distance\n", 53 | " s.geometry.day = scene_date.day # month and day used for Earth-Sun distance\n", 54 | " s.altitudes.set_sensor_satellite_level()\n", 55 | " s.altitudes.set_target_custom_altitude(km)\n", 56 | " def spectralResponseFunction(bandname): \n", 57 | " bandSelect = {\n", 58 | " 'B1':PredefinedWavelengths.LANDSAT_OLI_B1,\n", 59 | " 'B2':PredefinedWavelengths.LANDSAT_OLI_B2,\n", 60 | " 'B3':PredefinedWavelengths.LANDSAT_OLI_B3,\n", 61 | " 'B4':PredefinedWavelengths.LANDSAT_OLI_B4,\n", 62 | " 'B5':PredefinedWavelengths.LANDSAT_OLI_B5,\n", 63 | " 'B6':PredefinedWavelengths.LANDSAT_OLI_B6,\n", 64 | " 'B7':PredefinedWavelengths.LANDSAT_OLI_B7,\n", 65 | " 'B8':PredefinedWavelengths.LANDSAT_OLI_B8,\n", 66 | " 'B9':PredefinedWavelengths.LANDSAT_OLI_B9,\n", 67 | " }\n", 68 | " return Wavelength(bandSelect[bandname])\n", 69 | " def toa_to_rad(bandname):\n", 70 | " ESUN_L8 = [1895.33, 2004.57, 1820.75, 1549.49, 951.76, 247.55, 85.46, 1723.8, 366.97]\n", 71 | " ESUN_BAND = {\n", 72 | " 'B1':ESUN_L8[0],\n", 73 | " 'B2':ESUN_L8[1],\n", 74 | " 'B3':ESUN_L8[2],\n", 75 | " 'B4':ESUN_L8[3],\n", 76 | " 'B5':ESUN_L8[4],\n", 77 | " 'B6':ESUN_L8[5],\n", 78 | " 'B7':ESUN_L8[6],\n", 79 | " 'B8':ESUN_L8[7],\n", 80 | " 'B9':ESUN_L8[8],\n", 81 | " }\n", 82 | " solar_angle_correction = math.cos(math.radians(solar_z))\n", 83 | " # Earth-Sun distance (from day of year)\n", 84 | " doy = scene_date.timetuple().tm_yday\n", 85 | " d = 1 - 0.01672 * math.cos(0.9856 * (doy-4))# http://physics.stackexchange.com/questions/177949/earth-sun-distance-on-a-given-day-of-the-year\n", 86 | " # conversion factor\n", 87 | " multiplier = ESUN_BAND[bandname]*solar_angle_correction/(math.pi*d**2)\n", 88 | " # at-sensor radiance\n", 89 | " rad = img.select(bandname).multiply(multiplier)\n", 90 | " return rad\n", 91 | " def surface_reflectance(bandname):\n", 92 | " # run 6S for this waveband\n", 93 | " s.wavelength = spectralResponseFunction(bandname)\n", 94 | " s.run()\n", 95 | "\n", 96 | " # extract 6S outputs\n", 97 | " Edir = s.outputs.direct_solar_irradiance #direct solar irradiance\n", 98 | " Edif = s.outputs.diffuse_solar_irradiance #diffuse solar irradiance\n", 99 | " Lp = s.outputs.atmospheric_intrinsic_radiance #path radiance\n", 100 | " absorb = s.outputs.trans['global_gas'].upward #absorption transmissivity\n", 101 | " scatter = s.outputs.trans['total_scattering'].upward #scattering transmissivity\n", 102 | " tau2 = absorb*scatter #total transmissivity\n", 103 | "\n", 104 | " # radiance to surface reflectance\n", 105 | " rad = toa_to_rad(bandname)\n", 106 | " ref = rad.subtract(Lp).multiply(math.pi).divide(tau2*(Edir+Edif))\n", 107 | " return ref\n", 108 | " \n", 109 | " ca = surface_reflectance('B1')\n", 110 | " blue = surface_reflectance('B2')\n", 111 | " green = surface_reflectance('B3')\n", 112 | " red = surface_reflectance('B4')\n", 113 | " nir = surface_reflectance('B5')\n", 114 | " swir1 = surface_reflectance('B6')\n", 115 | " swir2 = surface_reflectance('B7')\n", 116 | " \n", 117 | " ref = img.select('BQA').addBands(ca).addBands(blue).addBands(green).addBands(red).addBands(nir).addBands(swir1).addBands(swir2)\n", 118 | " \n", 119 | " dateString = scene_date.strftime(\"%Y-%m-%d\")\n", 120 | " ref = ref.copyProperties(img).set({ \n", 121 | " 'AC_date':dateString,\n", 122 | " 'AC_aerosol_optical_thickness':aot,\n", 123 | " 'AC_water_vapour':h2o,\n", 124 | " 'AC_version':'py6S',\n", 125 | " 'AC_contact':'ndminhhus@gmail.com',\n", 126 | " 'AC_ozone':o3})\n", 127 | " \n", 128 | "\n", 129 | " # define YOUR assetID \n", 130 | " # in my case it was something like this..\n", 131 | " #assetID = 'users/samsammurphy/shared/sentinel2/6S/ESRIN_'+dateString\n", 132 | " #assetID = 'users/ndminhhus/eLEAF/nt/s2_SIAC/'+fname,\n", 133 | " # # export\n", 134 | " fname = ee.String(img.get('system:index')).getInfo()\n", 135 | " export = ee.batch.Export.image.toAsset(\\\n", 136 | " image=ref,\n", 137 | " description= 'L8_BOA_'+fname,\n", 138 | " assetId = 'users/ndminhhus/eLEAF/nt/L8_py6S/'+'L8_BOA'+fname,\n", 139 | " region = region,\n", 140 | " scale = 30,\n", 141 | " maxPixels = 1e13)\n", 142 | "\n", 143 | " # # uncomment to run the export\n", 144 | " export.start()\n", 145 | " print('exporting ' +fname + '--->done')" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": 120, 151 | "metadata": { 152 | "collapsed": true 153 | }, 154 | "outputs": [], 155 | "source": [ 156 | "startDate = ee.Date('2018-01-01')\n", 157 | "endDate = ee.Date('2020-01-01')\n", 158 | "geom = ee.Geometry.Point(108.91220182000018,11.700863529688942)# Ninh Thuan region\n", 159 | "region = geom.buffer(60000).bounds().getInfo()['coordinates']" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": 121, 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "# The Landsat 8 image collection\n", 169 | "L8_col = ee.ImageCollection('LANDSAT/LC08/C01/T1_TOA').filterBounds(geom).filterDate(startDate,endDate).sort('CLOUD_COVER')" 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": 122, 175 | "metadata": {}, 176 | "outputs": [ 177 | { 178 | "name": "stdout", 179 | "output_type": "stream", 180 | "text": [ 181 | "28\n" 182 | ] 183 | } 184 | ], 185 | "source": [ 186 | "print(L8_col.size().getInfo())" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": 91, 192 | "metadata": {}, 193 | "outputs": [], 194 | "source": [ 195 | "L8_list = L8_col.toList(L8_col.size().getInfo())\n", 196 | "img1 = ee.Image(L8_list.get(9))" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": 92, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "toa = img1\n", 206 | "boa = ee.Image(func1(toa))" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": 124, 212 | "metadata": {}, 213 | "outputs": [ 214 | { 215 | "ename": "KeyError", 216 | "evalue": "'AC_date'", 217 | "output_type": "error", 218 | "traceback": [ 219 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 220 | "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", 221 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mboa\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetInfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'properties'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'AC_date'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", 222 | "\u001b[0;31mKeyError\u001b[0m: 'AC_date'" 223 | ] 224 | } 225 | ], 226 | "source": [ 227 | "print(boa.getInfo()['properties']['AC_date'])" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": 106, 233 | "metadata": {}, 234 | "outputs": [ 235 | { 236 | "data": { 237 | "text/html": [ 238 | "" 239 | ], 240 | "text/plain": [ 241 | "" 242 | ] 243 | }, 244 | "metadata": {}, 245 | "output_type": "display_data" 246 | }, 247 | { 248 | "data": { 249 | "text/html": [ 250 | "" 251 | ], 252 | "text/plain": [ 253 | "" 254 | ] 255 | }, 256 | "metadata": {}, 257 | "output_type": "display_data" 258 | } 259 | ], 260 | "source": [ 261 | "from IPython.display import display, Image\n", 262 | "\n", 263 | "channels = ['B4','B3','B2']\n", 264 | "\n", 265 | "original = Image(url=toa.select(channels).getThumbUrl({\n", 266 | " 'region':region,\n", 267 | " 'min':0,\n", 268 | " 'max':0.25\n", 269 | " }))\n", 270 | "\n", 271 | "corrected = Image(url=ee.Image(boa).select(channels).getThumbUrl({\n", 272 | " 'region':region,\n", 273 | " 'min':0,\n", 274 | " 'max':0.25\n", 275 | " }))\n", 276 | "\n", 277 | "display(original, corrected)" 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": 123, 283 | "metadata": {}, 284 | "outputs": [ 285 | { 286 | "name": "stdout", 287 | "output_type": "stream", 288 | "text": [ 289 | "exporting LC08_123052_20190614--->done\n", 290 | "exporting LC08_123052_20181017--->done\n", 291 | "exporting LC08_123052_20190310--->done\n", 292 | "exporting LC08_123052_20180830--->done\n", 293 | "exporting LC08_123052_20180424--->done\n", 294 | "exporting LC08_123052_20180510--->done\n", 295 | "exporting LC08_123052_20181102--->done\n", 296 | "exporting LC08_123052_20180219--->done\n", 297 | "exporting LC08_123052_20180323--->done\n", 298 | "exporting LC08_123052_20180307--->done\n", 299 | "exporting LC08_123052_20190529--->done\n", 300 | "exporting LC08_123052_20181204--->done\n", 301 | "exporting LC08_123052_20181220--->done\n", 302 | "exporting LC08_123052_20190513--->done\n", 303 | "exporting LC08_123052_20190206--->done\n", 304 | "exporting LC08_123052_20190222--->done\n", 305 | "exporting LC08_123052_20190105--->done\n", 306 | "exporting LC08_123052_20190326--->done\n", 307 | "exporting LC08_123052_20190427--->done\n", 308 | "exporting LC08_123052_20180526--->done\n", 309 | "exporting LC08_123052_20190411--->done\n", 310 | "exporting LC08_123052_20190121--->done\n", 311 | "exporting LC08_123052_20180102--->done\n", 312 | "exporting LC08_123052_20180203--->done\n", 313 | "exporting LC08_123052_20180627--->done\n", 314 | "exporting LC08_123052_20180729--->done\n", 315 | "exporting LC08_123052_20180408--->done\n", 316 | "exporting LC08_123052_20180915--->done\n" 317 | ] 318 | } 319 | ], 320 | "source": [ 321 | "col_length = L8_col.size().getInfo()\n", 322 | "#print(col_length)\n", 323 | "\n", 324 | "for i in range(0,col_length):\n", 325 | " #print(i)\n", 326 | " list = L8_col.toList(col_length)\n", 327 | " img = ee.Image(list.get(i))\n", 328 | " func1(img)\n" 329 | ] 330 | }, 331 | { 332 | "cell_type": "code", 333 | "execution_count": null, 334 | "metadata": { 335 | "collapsed": true 336 | }, 337 | "outputs": [], 338 | "source": [] 339 | } 340 | ], 341 | "metadata": { 342 | "kernelspec": { 343 | "display_name": "Python 3", 344 | "language": "python", 345 | "name": "python3" 346 | }, 347 | "language_info": { 348 | "codemirror_mode": { 349 | "name": "ipython", 350 | "version": 3 351 | }, 352 | "file_extension": ".py", 353 | "mimetype": "text/x-python", 354 | "name": "python", 355 | "nbconvert_exporter": "python", 356 | "pygments_lexer": "ipython3", 357 | "version": "3.6.0" 358 | } 359 | }, 360 | "nbformat": 4, 361 | "nbformat_minor": 2 362 | } 363 | -------------------------------------------------------------------------------- /jupyter_notebooks/02.Atm-corr-Sentinel2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 2, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import ee\n", 12 | "from Py6S import *\n", 13 | "import datetime\n", 14 | "import math\n", 15 | "import os\n", 16 | "import sys\n", 17 | "sys.path.append(os.path.join(os.path.dirname(os.getcwd()),'bin'))\n", 18 | "from atmospheric import Atmospheric\n", 19 | "\n", 20 | "ee.Initialize()" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 33, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "def func1(img):\n", 30 | " img = ee.Image(img)\n", 31 | " info = img.getInfo()['properties']\n", 32 | " scene_date = datetime.datetime.utcfromtimestamp(info['system:time_start']/1000)\n", 33 | " solar_z = img.getInfo()['properties']['MEAN_SOLAR_ZENITH_ANGLE']\n", 34 | " h2o = Atmospheric.water(geom,img.date()).getInfo()\n", 35 | " o3 = Atmospheric.ozone(geom,img.date()).getInfo()\n", 36 | " aot = Atmospheric.aerosol(geom,img.date()).getInfo()\n", 37 | " SRTM = ee.Image('CGIAR/SRTM90_V4')# Shuttle Radar Topography mission covers *most* of the Earth\n", 38 | " alt = SRTM.reduceRegion(reducer = ee.Reducer.mean(),geometry = geom.centroid()).get('elevation').getInfo()\n", 39 | " km = alt/1000 # i.e. Py6S uses units of kilometers\n", 40 | " # Instantiate\n", 41 | " s = SixS()\n", 42 | " # Atmospheric constituents\n", 43 | " s.atmos_profile = AtmosProfile.UserWaterAndOzone(h2o,o3)\n", 44 | " s.aero_profile = AeroProfile.Continental\n", 45 | " s.aot550 = aot\n", 46 | " # Earth-Sun-satellite geometry\n", 47 | " s.geometry = Geometry.User()\n", 48 | " s.geometry.view_z = 0 # always NADIR (I think..)\n", 49 | " s.geometry.solar_z = solar_z # solar zenith angle\n", 50 | " s.geometry.month = scene_date.month # month and day used for Earth-Sun distance\n", 51 | " s.geometry.day = scene_date.day # month and day used for Earth-Sun distance\n", 52 | " s.altitudes.set_sensor_satellite_level()\n", 53 | " s.altitudes.set_target_custom_altitude(km)\n", 54 | " def spectralResponseFunction(bandname): \n", 55 | " \n", 56 | " bandSelect = {\n", 57 | " 'B1':PredefinedWavelengths.S2A_MSI_01,\n", 58 | " 'B2':PredefinedWavelengths.S2A_MSI_02,\n", 59 | " 'B3':PredefinedWavelengths.S2A_MSI_03,\n", 60 | " 'B4':PredefinedWavelengths.S2A_MSI_04,\n", 61 | " 'B5':PredefinedWavelengths.S2A_MSI_05,\n", 62 | " 'B6':PredefinedWavelengths.S2A_MSI_06,\n", 63 | " 'B7':PredefinedWavelengths.S2A_MSI_07,\n", 64 | " 'B8':PredefinedWavelengths.S2A_MSI_08,\n", 65 | " 'B8A':PredefinedWavelengths.S2A_MSI_09,\n", 66 | " 'B9':PredefinedWavelengths.S2A_MSI_10,\n", 67 | " 'B10':PredefinedWavelengths.S2A_MSI_11,\n", 68 | " 'B11':PredefinedWavelengths.S2A_MSI_12,\n", 69 | " 'B12':PredefinedWavelengths.S2A_MSI_13,\n", 70 | " }\n", 71 | " return Wavelength(bandSelect[bandname])\n", 72 | " def toa_to_rad(bandname):\n", 73 | " ESUN = info['SOLAR_IRRADIANCE_'+bandname]\n", 74 | " solar_angle_correction = math.cos(math.radians(solar_z))\n", 75 | " # Earth-Sun distance (from day of year)\n", 76 | " doy = scene_date.timetuple().tm_yday\n", 77 | " d = 1 - 0.01672 * math.cos(0.9856 * (doy-4))# http://physics.stackexchange.com/questions/177949/earth-sun-distance-on-a-given-day-of-the-year\n", 78 | " # conversion factor\n", 79 | " multiplier = ESUN*solar_angle_correction/(math.pi*d**2)\n", 80 | " # at-sensor radiance\n", 81 | " rad = img.select(bandname).multiply(multiplier)\n", 82 | " return rad\n", 83 | " def surface_reflectance(bandname):\n", 84 | " # run 6S for this waveband\n", 85 | " s.wavelength = spectralResponseFunction(bandname)\n", 86 | " s.run()\n", 87 | "\n", 88 | " # extract 6S outputs\n", 89 | " Edir = s.outputs.direct_solar_irradiance #direct solar irradiance\n", 90 | " Edif = s.outputs.diffuse_solar_irradiance #diffuse solar irradiance\n", 91 | " Lp = s.outputs.atmospheric_intrinsic_radiance #path radiance\n", 92 | " absorb = s.outputs.trans['global_gas'].upward #absorption transmissivity\n", 93 | " scatter = s.outputs.trans['total_scattering'].upward #scattering transmissivity\n", 94 | " tau2 = absorb*scatter #total transmissivity\n", 95 | "\n", 96 | " # radiance to surface reflectance\n", 97 | " rad = toa_to_rad(bandname)\n", 98 | " ref = rad.subtract(Lp).multiply(math.pi).divide(tau2*(Edir+Edif))\n", 99 | " return ref\n", 100 | " \n", 101 | " ca = surface_reflectance('B1')\n", 102 | " blue = surface_reflectance('B2')\n", 103 | " green = surface_reflectance('B3')\n", 104 | " red = surface_reflectance('B4')\n", 105 | " nir = surface_reflectance('B8')\n", 106 | " swir1 = surface_reflectance('B11')\n", 107 | " swir2 = surface_reflectance('B12')\n", 108 | " \n", 109 | " ref = img.select('QA60').addBands(ca).addBands(blue).addBands(green).addBands(red).addBands(nir).addBands(swir1).addBands(swir2)\n", 110 | " \n", 111 | " dateString = scene_date.strftime(\"%Y-%m-%d\")\n", 112 | " ref = ref.copyProperties(img).set({ \n", 113 | " 'AC_date':dateString,\n", 114 | " 'AC_aerosol_optical_thickness':aot,\n", 115 | " 'AC_water_vapour':h2o,\n", 116 | " 'AC_version':'py6S',\n", 117 | " 'AC_contact':'ndminhhus@gmail.com',\n", 118 | " 'AC_ozone':o3})\n", 119 | " \n", 120 | "\n", 121 | " # define YOUR assetID \n", 122 | " # in my case it was something like this..\n", 123 | " #assetID = 'users/samsammurphy/shared/sentinel2/6S/ESRIN_'+dateString\n", 124 | " #assetID = 'users/ndminhhus/eLEAF/nt/s2_SIAC/'+fname,\n", 125 | " # # export\n", 126 | " fname = ee.String(img.get('system:index')).getInfo()\n", 127 | " export = ee.batch.Export.image.toAsset(\\\n", 128 | " image=ref,\n", 129 | " description= 'S2_BOA_'+fname,\n", 130 | " assetId = 'users/ndminhhus/eLEAF/nt/S2_py6S/'+'S2_BOA'+fname,\n", 131 | " region = region,\n", 132 | " scale = 10,\n", 133 | " maxPixels = 1e13)\n", 134 | "\n", 135 | " # # uncomment to run the export\n", 136 | " export.start()\n", 137 | " print('exporting ' +fname + '--->done')" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 34, 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "startDate = ee.Date('2018-01-01')\n", 147 | "endDate = ee.Date('2020-01-01')\n", 148 | "geom = ee.Geometry.Point(108.91220182000018,11.700863529688942)\n", 149 | "geom1 = ee.Geometry.Point(108.8619544253213,11.40533502657536)# Ninh Thuan region for lower part (T49PBN)\n", 150 | "geom2 = ee.Geometry.Point(108.96124878094292,11.799830246236146)# Ninh Thuan region for upper part (T49PBP)\n", 151 | "\n", 152 | "region = geom.buffer(55000).bounds().getInfo()['coordinates']" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": 35, 158 | "metadata": {}, 159 | "outputs": [], 160 | "source": [ 161 | "# The Sentinel 2 image collection\n", 162 | "#S2_col = ee.ImageCollection('COPERNICUS/S2').filterBounds(geom1).filterDate(startDate,endDate).filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',50))\n", 163 | "S2_col = ee.ImageCollection('COPERNICUS/S2').filterBounds(geom2).filterDate(startDate,endDate).filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',50))" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": 36, 169 | "metadata": {}, 170 | "outputs": [ 171 | { 172 | "name": "stdout", 173 | "output_type": "stream", 174 | "text": [ 175 | "60\n" 176 | ] 177 | } 178 | ], 179 | "source": [ 180 | "print(S2_col.size().getInfo())" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": 17, 186 | "metadata": { 187 | "collapsed": true 188 | }, 189 | "outputs": [], 190 | "source": [ 191 | "S2_list = S2_col.toList(S2_col.size().getInfo())\n", 192 | "img1 = ee.Image(S2_list.get(9))" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 18, 198 | "metadata": { 199 | "collapsed": true 200 | }, 201 | "outputs": [], 202 | "source": [ 203 | "toa = img1\n", 204 | "boa = ee.Image(func1(toa))" 205 | ] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "execution_count": 20, 210 | "metadata": {}, 211 | "outputs": [ 212 | { 213 | "name": "stdout", 214 | "output_type": "stream", 215 | "text": [ 216 | "2018-03-09\n" 217 | ] 218 | } 219 | ], 220 | "source": [ 221 | "print(boa.getInfo()['properties']['AC_date'])" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 22, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "data": { 231 | "text/html": [ 232 | "" 233 | ], 234 | "text/plain": [ 235 | "" 236 | ] 237 | }, 238 | "metadata": {}, 239 | "output_type": "display_data" 240 | }, 241 | { 242 | "data": { 243 | "text/html": [ 244 | "" 245 | ], 246 | "text/plain": [ 247 | "" 248 | ] 249 | }, 250 | "metadata": {}, 251 | "output_type": "display_data" 252 | } 253 | ], 254 | "source": [ 255 | "from IPython.display import display, Image\n", 256 | "\n", 257 | "channels = ['B4','B3','B2']\n", 258 | "\n", 259 | "original = Image(url=toa.select(channels).getThumbUrl({\n", 260 | " 'region':region,\n", 261 | " 'min':0,\n", 262 | " 'max':3500\n", 263 | " }))\n", 264 | "\n", 265 | "corrected = Image(url=ee.Image(boa).select(channels).getThumbUrl({\n", 266 | " 'region':region,\n", 267 | " 'min':0,\n", 268 | " 'max':3500\n", 269 | " }))\n", 270 | "\n", 271 | "display(original, corrected)" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": null, 277 | "metadata": {}, 278 | "outputs": [ 279 | { 280 | "name": "stdout", 281 | "output_type": "stream", 282 | "text": [ 283 | "exporting 20180108T031109_20180108T032345_T49PBN--->done\n", 284 | "exporting 20180123T032321_20180123T032316_T49PBN--->done\n", 285 | "exporting 20180202T030931_20180202T031511_T49PBN--->done\n", 286 | "exporting 20180207T030859_20180207T031350_T49PBN--->done\n", 287 | "exporting 20180212T030831_20180212T032401_T49PBN--->done\n", 288 | "exporting 20180217T030759_20180217T031916_T49PBN--->done\n", 289 | "exporting 20180222T030731_20180222T031550_T49PBN--->done\n", 290 | "exporting 20180227T030649_20180227T031252_T49PBN--->done\n", 291 | "exporting 20180304T030621_20180304T031229_T49PBN--->done\n", 292 | "exporting 20180309T030539_20180309T032200_T49PBN--->done\n", 293 | "exporting 20180319T030539_20180319T031100_T49PBN--->done\n", 294 | "exporting 20180329T030539_20180329T032339_T49PBN--->done\n", 295 | "exporting 20180403T030541_20180403T031957_T49PBN--->done\n", 296 | "exporting 20180413T030541_20180413T031057_T49PBN--->done\n", 297 | "exporting 20180423T030541_20180423T031503_T49PBN--->done\n", 298 | "exporting 20180428T030539_20180428T032437_T49PBN--->done\n", 299 | "exporting 20180508T030539_20180508T031153_T49PBN--->done\n", 300 | "exporting 20180518T030539_20180518T032230_T49PBN--->done\n", 301 | "exporting 20180607T030539_20180607T032215_T49PBN--->done\n", 302 | "exporting 20180622T030541_20180622T031034_T49PBN--->done\n", 303 | "exporting 20180707T030539_20180707T031902_T49PBN--->done\n", 304 | "exporting 20180801T030541_20180801T032012_T49PBN--->done\n", 305 | "exporting 20180831T030541_20180831T031152_T49PBN--->done\n", 306 | "exporting 20180905T030539_20180905T031854_T49PBN--->done\n", 307 | "exporting 20180925T030539_20180925T032406_T49PBN--->done\n", 308 | "exporting 20180930T030541_20180930T031150_T49PBN--->done\n", 309 | "exporting 20181010T030631_20181010T031209_T49PBN--->done\n", 310 | "exporting 20181025T030809_20181025T031436_T49PBN--->done\n", 311 | "exporting 20181030T030831_20181030T031435_T49PBN--->done\n", 312 | "exporting 20181104T030909_20181104T031306_T49PBN--->done\n", 313 | "exporting 20181129T031051_20181129T031648_T49PBN--->done\n", 314 | "exporting 20181204T031109_20181204T032310_T49PBN--->done\n", 315 | "exporting 20181209T031111_20181209T031714_T49PBN--->done\n", 316 | "exporting 20181219T031131_20181219T031727_T49PBN--->done\n", 317 | "exporting 20190108T031111_20190108T032310_T49PBN--->done\n", 318 | "exporting 20190113T031059_20190113T031703_T49PBN--->done\n", 319 | "exporting 20190128T031001_20190128T031602_T49PBN--->done\n", 320 | "exporting 20190202T030939_20190202T031413_T49PBN--->done\n", 321 | "exporting 20190207T030911_20190207T031859_T49PBN--->done\n", 322 | "exporting 20190212T030839_20190212T032044_T49PBN--->done\n", 323 | "exporting 20190217T030801_20190217T032232_T49PBN--->done\n", 324 | "exporting 20190222T030729_20190222T031329_T49PBN--->done\n", 325 | "exporting 20190227T030651_20190227T031435_T49PBN--->done\n", 326 | "exporting 20190304T030619_20190304T032047_T49PBN--->done\n", 327 | "exporting 20190309T030541_20190309T032019_T49PBN--->done\n", 328 | "exporting 20190314T030539_20190314T032229_T49PBN--->done\n" 329 | ] 330 | } 331 | ], 332 | "source": [ 333 | "col_length = S2_col.size().getInfo()\n", 334 | "#print(col_length)\n", 335 | "\n", 336 | "for i in range(0,col_length):\n", 337 | " #print(i)\n", 338 | " list = S2_col.toList(col_length)\n", 339 | " img = ee.Image(list.get(i))\n", 340 | " func1(img)\n" 341 | ] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "execution_count": null, 346 | "metadata": { 347 | "collapsed": true 348 | }, 349 | "outputs": [], 350 | "source": [] 351 | } 352 | ], 353 | "metadata": { 354 | "kernelspec": { 355 | "display_name": "Python 3", 356 | "language": "python", 357 | "name": "python3" 358 | }, 359 | "language_info": { 360 | "codemirror_mode": { 361 | "name": "ipython", 362 | "version": 3 363 | }, 364 | "file_extension": ".py", 365 | "mimetype": "text/x-python", 366 | "name": "python", 367 | "nbconvert_exporter": "python", 368 | "pygments_lexer": "ipython3", 369 | "version": "3.6.0" 370 | } 371 | }, 372 | "nbformat": 4, 373 | "nbformat_minor": 2 374 | } 375 | -------------------------------------------------------------------------------- /jupyter_notebooks/python_guide.md: -------------------------------------------------------------------------------- 1 | 2 | --------------------------------------------------------------------------------